A low-altitude bird detection method and system based on radar optoelectronic information fusion

Abstract

The present invention discloses a low-altitude bird detection method and system based on radar optoelectronic information fusion, which simultaneously uses radar equipment and optoelectronic equipment to track low-altitude bird targets in real time, and matches and fuses the radar track and the optoelectronic track to obtain a fused track, that is, the bird track. First, the radar track and the optoelectronic track are aligned in time and converted to the same coordinate system, and then the radar track and the optoelectronic track are cyclically found to match and fuse the radar track and the optoelectronic track that meet the matching judgment conditions to obtain a fused track. Then, as time goes by, the fused track is updated according to the radar track and the optoelectronic track at the next moment, that is, the time t+1, to complete the real-time tracking of low-altitude bird targets. For the two tracks at the next moment, that is, the time t+1, there will be a situation where there is no matching fusion, and these remaining tracks are re-matched and fused to generate a new fused track to ensure the continuity and real-time nature of the fused track information.

Description

Technical Field

[0001] The present invention relates to the field of radar optoelectronic cooperative detection technology, and in particular to a low-altitude bird detection method and system based on radar optoelectronic information fusion. Background Art

[0002] Low-flying birds are a threat to the safety of flight takeoff and landing. Bird strikes caused by them bring huge economic losses every year. As the number of flights continues to grow, the pressure on bird strike prevention is increasing. Existing bird detection technologies include acoustic detection, photoelectric detection, infrared detection, radar detection, etc. Photoelectric equipment is low in price, flexible in station layout, high in data rate, and not susceptible to electromagnetic interference. It can improve the ability of threat assessment and target identification, but photoelectric equipment cannot provide accurate positioning and speed information of the target, and is greatly affected by weather and cannot work all day and all weather. Radar has a wide detection range, flexible target tracking, good weather adaptability, and high positioning accuracy, but it cannot collect more abundant types of information about the target. With the diversified development of detection technology, collaborative detection technology that makes full use of multi-sensor resources and information advantages is gradually emerging, and the complementary advantages of radar photoelectric equipment and coordinated use are used to achieve collaborative detection of bird targets.

[0003] The existing radar and optoelectronic collaborative detection technology generally has radar and optoelectronics track and process independently. The radar searches and tracks the target and sends the tracking information to the optoelectronics device, which captures the target and tracks it based on the guidance information. When the system works in a complex environment, the radar is susceptible to interference and clutter, which leads to a decrease in the quality of the tracking track, mis-association of tracks, and false tracks. The optoelectronics are also susceptible to obstruction, and the target detection and recognition capabilities are insufficient, resulting in target loss and other problems. Summary of the invention

[0004] In view of this, the present invention provides a low-altitude bird detection method and system based on radar optoelectronic information fusion, which can ensure the effective tracking of bird targets in complex environments and avoid the problem of mistracking or loss of tracking by a single radar or a single optoelectronic during the tracking process, resulting in the unusable tracking system.

[0005] The specific technical solution adopted by the present invention is as follows:

[0006] A low-altitude bird detection method based on radar optoelectronic information fusion, which uses radar equipment and optoelectronic equipment to track low-altitude bird targets in real time, and matches and fuses the radar track and the optoelectronic track to obtain a fused track, namely the bird track;

[0007] The matching fusion process is as follows: one of the radar track set and the optoelectronic track set is recorded as set A, and the other is recorded as set B; set A contains M tracks [a1, a2, ..., a M ], a m ∈[a1,a2,…,aM ], m∈[1,2,…,M], set B contains N tracks [b1,b2,…,b N ], b n ∈[b1,b2,…,b N ], n∈[1,2,…,N]; at time t, traverse the M tracks in set A, and for each track a m , search for the same track as a in the N tracks in set B m Track b that meets the matching judgment conditions n Matching fusion is to take the union of the track data contained in the track to obtain the fused track C q , track b n with a m After matching and fusion, it is no longer consistent with track a m+1 Matching fusion; Q fused tracks form a fused track set [C1, C2, …, C Q ], C q ∈[C1,C2,…,C Q ], q∈[1,2,…,Q], where Q is the maximum value of M and N, M, N, Q are all positive integers, a m , b n , C q Indicates the track number; fused track C q Track number C q =[c q ,a m ,b n ], where c q It is the track label of the fused track;

[0008] If there is no b n with a m If the matching judgment condition is met, then according to a m Generate a corresponding independent fusion track, the track number is C q =[c q ,a m ,-1],-1 is b n = -1 means no match to b n ; After traversing all the tracks a m Afterwards, if there is a remaining track b n , then according to b n Generate a corresponding independent fusion track, the track number is C q =[c q ,-1,b n ], where -1 is a m = -1 means no a m ;

[0009] After matching and fusion at time t, the fused track is updated according to the set A and set B at time t+1: the track number of each fused track obtained at time t is extracted respectively, and the track number of the fused track is updated according to the track number of a m and b n The value of , searches for the corresponding track in set A and set B at time t+1, replaces the track data in the fused track, and obtains a new fused track;

[0010] For an independent fused track, if the track number is C q =[c q ,a m ,-1], then traverse the b in set B that has not been matched and fused n With track number a m The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, a m unchanged; if the match fails, the track number at time t+1 is a m The track generates a new independent fusion track, and the track number is C q =[c q ,a m ,-1] unchanged;

[0011] If the track number is C q =[c q ,-1,b n ], then traverse the a in set A that has not been matched and fused m With track number b n The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, b n unchanged; if the match fails, the track number at time t+1 is b n The track generates a new independent fusion track, and the track number is C q =[c q ,-1,b n ]constant;

[0012] After the fused track is updated, the remaining tracks in set A and tracks in set B that have not been matched and fused are traversed, and matching and fusion are performed according to the matching and fusion process to generate a new fused track;

[0013] The process of updating the fused track is repeated to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

[0014] Furthermore, when traversing a m In the process, for each track a m , if there are multiple tracks b n If the matching condition is met, select b n The track coordinates and a m The track b with the smallest absolute distance of track coordinates n , match and fuse, and generate the fused track with the track number C p =[c p ,a m ,b n ], where c p It is the track label of the fused track.

[0015] Further, the matching judgment condition is that the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than a matching threshold set by the system.

[0016] Furthermore, before the matching and fusion is performed, it is necessary to time-align the radar track and the optoelectronic track, and transform the spatial coordinates of the time-aligned radar track and the optoelectronic track into the same coordinate system.

[0017] Furthermore, the time-aligned radar track and optoelectronic track are spatially converted to the same coordinate system by converting the radar track coordinates from the radar rectangular coordinate system to the pixel coordinate system or converting the pixel coordinate system of the optoelectronic track coordinates to the radar rectangular coordinate system.

[0018] A low-altitude bird detection system based on radar optoelectronic information fusion comprises a radar module, an optoelectronic module and a fusion module; the radar module and the optoelectronic module simultaneously track low-altitude bird targets in real time to obtain radar tracks and optoelectronic tracks; the fusion module matches and fuses the radar tracks and optoelectronic tracks to obtain fused tracks, i.e., bird tracks;

[0019] The fusion module includes a time alignment unit, a coordinate conversion unit, a matching fusion unit and an updating unit;

[0020] The time alignment unit is used to align the time of the radar track and the optoelectronic track; the coordinate conversion unit is used to convert the spatial coordinates of the radar track and the optoelectronic track to the same coordinate system; the matching fusion unit is used to match and fuse the radar track and the optoelectronic track that meet the matching judgment conditions, that is, to take the union of the radar track data and the optoelectronic track data, obtain the fused track and store it; the updating unit is used to update the fused track according to the radar track and the optoelectronic track at the next moment, and obtain the real-time bird track.

[0021] The matching fusion unit performs matching fusion in the following manner: one of the radar track set and the optoelectronic track set is recorded as set A, and the other is recorded as set B; set A contains M tracks [a1, a2, ..., a M ], a m ∈[a1,a2,…,a M ], m∈[1,2,…,M], set B contains N tracks [b1,b2,…,b N ], b n ∈[b1,b2,…,b N ], n∈[1,2,…,N]; at time t, traverse the M tracks in set A, and for each track a m , search for the same track as a in the N tracks in set B m Track b that meets the matching judgment conditions n Matching fusion is to take the union of the track data contained in the track to obtain the fused track C q , track b n with a m After matching and fusion, it is no longer consistent with track a m+1 Matching fusion; Q fused tracks form a fused track set [C1, C2, …, C Q ], C q ∈[C1,C2,…,C Q ], q∈[1,2,…,Q], where Q is the maximum value of M and N, M, N, Q are all positive integers, a m , b n , C q Indicates the track number; fused track C q Track number C q =[c q ,a m ,b n ], where c q It is the track label of the fused track;

[0022] If there is no b n with a m If the matching judgment condition is met, then according to a m Generate a corresponding independent fusion track, the track number is Cq =[c q ,a m ,-1],-1 is b n = -1 means no b n ; After traversing all the tracks a m Afterwards, if there is a remaining track b n , then according to b n Generate a corresponding independent fusion track, the track number is C q =[c q ,-1,b n ], where -1 is a m = -1 means no a m ;

[0023] After matching and fusion at time t, according to set A and set B at time t+1, the fused track is updated in the updating unit: the track number of each fused track obtained at time t is extracted respectively, and the track number of the fused track is updated according to a m and b n , searching for the corresponding track in set A and set B at time t+1, replacing the track data in the fused track, obtaining a new fused track, and storing it in the matching fusion unit;

[0024] For an independent fused track, if the track number is C q =[c q ,a m ,-1], then traverse the b in set B that has not been matched and fused n With track number a m The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, a m unchanged; if the match fails, the track number at time t+1 is a m The track generates a new independent fusion track, and the track number is C q =[c q ,a m ,-1] unchanged;

[0025] If the track number is C q =[c q ,-1,b n ], then traverse the a in set A that has not been matched and fused m With track number b n The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[cq ,a m ,b n ], where c q unchanged, b n unchanged; if the match fails, the track number at time t+1 is b n The track generates a new independent fusion track, and the track number is C q =[c q ,-1,b n ]constant;

[0026] After the fused track is updated, the remaining tracks in set A and tracks in set B that have not been matched and fused are traversed, and matching and fusion are performed in the matching and fusion unit according to the matching and fusion process to generate a new fused track;

[0027] The process of updating the fused track is repeated to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

[0028] Furthermore, before the matching fusion unit performs matching fusion, the time alignment unit needs to time align the radar track and the optoelectronic track, and the coordinate conversion unit converts the spatial coordinates of the time-aligned radar track and optoelectronic track to the same coordinate system; after the matching fusion unit performs matching fusion, the fused track is updated by the updating unit.

[0029] Furthermore, when traversing a m In the process, for each track a m , if there are multiple tracks b n If the matching condition is met, select b n The track coordinates and a m The track b with the smallest absolute distance of track coordinates n , match and fuse, and generate the fused track with the track number C p =[c p ,a m ,b n ], where c p It is the track label of the fused track.

[0030] Further, the matching judgment condition is that the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than a matching threshold set by the system.

[0031] Beneficial effects:

[0032] (1) The present invention uses radar equipment and optoelectronic equipment to track low-altitude bird targets in real time, which can ensure the effective tracking of bird targets in complex environments. When the number of targets indicated by the radar track is inaccurate, the accurate number can be identified by optoelectronics. When the optoelectronic target is blocked and the track is interrupted, the radar track can maintain continuous tracking, avoiding the problem of false tracking or loss of tracking by a single radar or a single optoelectronic device during the tracking process, which makes the tracking system unusable. The radar track and the optoelectronic track are matched and fused to obtain a fused track, which can maintain the continuity of the bird track tracking process and track data. After obtaining the fused track, the fused track is updated according to the radar track and optoelectronic track obtained by real-time tracking to ensure the real-time nature of the fused track. The method of the present invention is used to obtain the fused track while retaining the original radar track and optoelectronic track, which has good applicability and versatility, and users can select the required track data according to their needs.

[0033] (2) If there are multiple optoelectronic tracks or radar tracks that meet the matching judgment conditions, the radar track or optoelectronic track with the smallest absolute distance between the radar track coordinates at the current moment and the optoelectronic track coordinates at the current moment is selected for matching and fusion, which ensures the smooth progress of the matching and fusion process and avoids fusion confusion and repeated fusion.

[0034] (3) After the cycle, the radar tracks and optoelectronic tracks that are not matched and fused will generate radar independent tracks and optoelectronic independent tracks, which can ensure that the fused track completely contains all radar track data and optoelectronic track data, thereby ensuring the consistency and accuracy of the fused track.

[0035] (4) After the fused track is updated, there will still be radar tracks and optoelectronic tracks that have not been selected for matching and fusion. The remaining radar tracks and optoelectronic tracks that have not been matched and fused are circulated again to perform matching and fusion to generate new fused tracks. This ensures that the fused track obtained as it is updated over time can still contain all the updated radar track data and optoelectronic track data, thereby further ensuring continuity and accuracy while ensuring real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the time alignment process of radar track and photoelectric track of the present invention;

[0037] Figure 2 It is a schematic diagram of time alignment interpolation of radar track and optoelectronic track of the present invention;

[0038] Figure 3 This is a schematic diagram of converting the radar coordinate system to the pixel coordinate system;

[0039] Figure 4 This is a schematic diagram of the camera imaging principle;

[0040] Figure 5 Schematic diagram of image coordinate system and pixel coordinate system;

[0041] Figure 6 The present invention is a flow chart of the fusion processing of radar track data and optoelectronic track data. DETAILED DESCRIPTION

[0042] The present invention provides a low-altitude bird detection method and system based on radar optoelectronic information fusion, which simultaneously uses radar equipment and optoelectronic equipment to track low-altitude bird targets in real time, and matches and fuses the radar track and the optoelectronic track to obtain a fused track, that is, the bird track. First, the radar track and the optoelectronic track are aligned in time and converted to the same coordinate system, and then the radar track and the optoelectronic track are cyclically found to match and fuse the radar track and the optoelectronic track that meet the matching judgment conditions to obtain a fused track. Then, as time goes by, the fused track is updated according to the radar track and the optoelectronic track at the next moment, that is, the time t+1, to complete the real-time tracking of low-altitude bird targets. For the two tracks at the next moment, that is, the time t+1, there will be a situation where there is no matching fusion. These remaining tracks are re-matched and fused to generate a new fused track to ensure the continuity and real-time nature of the fused track information.

[0043] Among them, radar equipment will obtain multiple radar tracks when tracking bird targets. One radar track corresponds to one bird target. Multiple radar tracks constitute a radar track set. The radar track contains radar track data. The radar track data includes radar track coordinates and bird flight speed, etc.; optoelectronic equipment will obtain multiple optoelectronic tracks when tracking bird targets. One optoelectronic track corresponds to one bird target. Multiple optoelectronic tracks constitute an optoelectronic track set. The optoelectronic track contains optoelectronic track data. The optoelectronic track data includes optoelectronic track coordinates and the type and color of the bird, etc. Radar tracks and optoelectronic tracks correspond to bird targets. The specific track number of the radar track and the track number of the optoelectronic track can correspond to the number of the bird target. There is no restriction on the specific correspondence. They can be equal or have different representations or counting forms, as long as one target corresponds to one track number. The fused track contains radar track data and optoelectronic track data. Multiple fused tracks constitute a fused track set. The fused track includes optoelectronic radar fusion track track F,RV , radar independent track F,R Heguang independent track F,V Therefore, the fused track set includes the photoelectric radar fusion track set Track F,RV , Radar independent track set Track F,R Heguang independent track set Track F,VThe fused track number of the fused track includes the track label of the fused track, the track number of the radar track, and the track number of the photoelectric track; the track labels of different types of fused tracks are set continuously according to the order of matching and fusion. Once the fused track is generated, the cyclic update will only update the data information of the fused track, and its track label remains unchanged. If a new fused track is generated, a new track label will continue to be generated in sequence. The track number of the radar track in the radar track set and the photoelectric track in the photoelectric track set are consistent with the target sequence number of the bird target, and remain unchanged after matching and fusion.

[0044] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0045] A low-altitude bird detection method based on radar optoelectronic information fusion, which uses radar equipment and optoelectronic equipment to track low-altitude bird targets in real time, and matches and fuses the radar track and the optoelectronic track to obtain a fused track, namely the bird track;

[0046] The matching fusion process is as follows:

[0047] One of the radar track set and the optoelectronic track set is denoted as set A, and the other is denoted as set B; set A contains M tracks [a1, a2, …, a M ], a m ∈[a1,a2,…,a M ], m∈[1,2,…,M], set B contains N tracks [b1,b2,…,b N ], b n ∈[b1,b2,…,b N ], n∈[1,2,…,N]; at time t, traverse the M tracks in set A, and for each track a m , search for the same track as a in the N tracks in set B m Track b that meets the matching judgment conditions n Matching fusion is to take the union of the track data contained in the track to obtain the fused track C q , track b n with a m After matching and fusion, it is no longer consistent with track a m+1 Matching fusion; Q fused tracks form a fused track set [C1, C2, …, C Q ], C q ∈[C1,C2,…,C Q ], q∈[1,2,…,Q], where Q is the maximum value of M and N, M, N, Q are all positive integers, a m , b n , C q Indicates the track number; fused track C q Track number Cq =[c q ,a m ,b n ], where c q It is the track label of the fused track;

[0048] If there is no b n with a m If the matching judgment condition is met, then according to a m Generate corresponding independent fused tracks, that is, the fused track only contains a m The corresponding track data, the track number of the fused track is C q =[c q ,a m ,-1],-1 is b n = -1 means no match to b n ; After traversing all the tracks a m Afterwards, if there is a remaining track b n , then according to b n Generate corresponding independent fused tracks, that is, the fused track only contains b n The corresponding track data, the track number of the fused track is C q =[c q ,-1,b n ], where -1 is a m = -1 means no a m ;

[0049] After matching and fusion at time t, the fused track is updated according to the set A and set B at time t+1: the track number of each fused track obtained at time t is extracted respectively, and the track number of the fused track is updated according to the track number of a m and b n The corresponding track is searched in set A and set B at time t+1, and the track data in the fused track is replaced to obtain a new fused track; the track data here includes radar track data and optoelectronic track data.

[0050] For an independent fused track, if the track number is C q =[c q ,a m ,-1], then traverse the b in set B that has not been matched and fused n With track number a m The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, a munchanged; if the match fails, the track number at time t+1 is a m The track generates a new independent fusion track, and the track number is C q =[c q ,a m ,-1] unchanged;

[0051] If the track number is C q =[c q ,-1,b n ], then traverse the a in set A that has not been matched and fused m With track number b n The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, b n unchanged; if the match fails, the track number at time t+1 is b n The track generates a new independent fusion track, and the track number is C q =[c q ,-1,b n ]constant;

[0052] After the fused track is updated, the remaining tracks in set A and tracks in set B that have not been matched and fused are traversed, and matching and fusion are performed according to the above matching and fusion process to generate a new fused track;

[0053] The process of updating the fused track is repeated to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

[0054] Among them, when traversing a m In the process, for each track a m , if there are multiple tracks b n If the matching condition is met, select b n The track coordinates and a m The track b with the smallest absolute distance of track coordinates n , match and fuse, and generate the fused track with the track number C p =[c p ,a m ,b n [, where c p It is the track label of the fused track.

[0055] The matching judgment condition is: the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than the matching threshold set by the system.

[0056] The above matching fusion and updating process records the matching fusion process and ideas of the present invention. In the description process, no specific set is looped or searched for matching, but the description of the radar track set and the photoelectric track set is replaced by the form of set A and set B, in order to show that in the loop process, the radar track can be looped to search for the photoelectric track to match it, or the photoelectric track can be looped to search for the radar track to match it. At the same time, before the actual matching fusion process, it is necessary to time-align the radar track and the photoelectric track, and convert the spatial coordinates of the time-aligned radar track and the photoelectric track to the same coordinate system. It can also be said that the tracks in set A and set B are time-aligned, and the track coordinates of the tracks in set A and the track coordinates of the tracks in set B are converted to the same coordinate system. Specifically, the track coordinates in which set are converted to be consistent with the track coordinates of the tracks in the other set, or the track coordinates of the tracks in the two sets are simultaneously converted to the same coordinate system. There is no restriction on the conversion method of the coordinates.

[0057] To avoid confusion, when the radar track set and the optoelectronic track set are summarized as set A and set B, the track numbers of the tracks in the two sets and the track numbers of the fused tracks after matching and fusion are also described differently. Specifically, set A contains M tracks [a1, a2, …, a M ], if set A represents a radar track set, then its track set is Track R , which contains multiple radar tracks R , the corresponding track number is Id R ,Id R is a positive integer, and the track number in set A is a m If set A represents an optoelectronic track set, then its track set is Track V , which contains multiple radar tracks V , the corresponding track number is Id V ,Id V is a positive integer, and the track number in set A is a m Similarly, if set B represents a radar track set, then its track set is Track R , which contains multiple radar tracks R , the corresponding track number is Id R ,Id R is a positive integer, and the track number in set B is b n If set B represents an optoelectronic track set, then its track set is Track V , which contains multiple radar tracks V, the corresponding track number is Id V ,Id V is a positive integer, and the track number in set B is b n The track number C of the fused track generated after matching and fusing the tracks in the two sets q =[c q ,a m ,b n ], C q is a positive integer, indicating the track number of the fused track, c q is a positive integer, indicating the track label of the fused track, a m , b n It is the track number corresponding to the tracks in set A and the tracks in set B contained in the fused track. If one of the tracks in set A and set B does not exist, it is represented by -1. For example, C q =[c q ,a m , -1] or C q =[c q ,-1,b n ].

[0058] Embodiment 1

[0059] The following is an example description of the scheme of the present invention in the order of steps in the actual operation process, and uses specific set names, namely radar track set and optoelectronic track set, and uses corresponding track numbers to facilitate technicians in this field to better understand the technical scheme of the present invention.

[0060] Step 1: Time-align the radar track and the optoelectronic track, and convert the spatial coordinates of the time-aligned radar track and the optoelectronic track into the same coordinate system.

[0061] First, the radar track and the optoelectronic track are time-aligned. There are two ways to extract data: one is to extract two adjacent frames of radar track data and one frame of optoelectronic track data, and interpolate the two frames of radar track data to obtain radar track data that is time-aligned with the extracted optoelectronic track data; the time corresponding to the extracted radar track data is Time R1 、Time R2 The time corresponding to the extracted photoelectric track data is Time V ,Time R1 ≤Time V ≤Time R2 ;

[0062] The other is to extract two adjacent frames of photoelectric track data and one frame of radar track data, and interpolate the two frames of photoelectric track data to obtain photoelectric track data that is time-aligned with the extracted radar track data; the time corresponding to the extracted photoelectric track data is Time V1 、Time V2 , the time corresponding to the extracted radar track data is Time R ,Time V1 ≤Time R ≤Time V2 .

[0063] In the embodiment of the present invention, the time alignment is performed in the first data extraction mode, that is, extracting two adjacent frames of radar track data and one frame of photoelectric track data. The processing flow of the time alignment is as follows: Figure 1 As shown, the following steps are included:

[0064] Step 11: Extract a frame of electro-optical track V and its time V , extract the radar tracks of two adjacent frames R1 、track R2 and its time R1 、Time R2 ; The photoelectric track is in the pixel UV coordinate system, and the track number is represented by Id V ; The radar track is the radar rectangular coordinate XYZ coordinate system, and the track number is represented by Id R .

[0065] Step 12, determine the time V 、Time R1 With Time R2 Time relationship:

[0066] If Time V <Time R1 <Time R2 , execute step 13;

[0067] If Time R1 <Time R2 <Time V , execute step 14;

[0068] If Time R1 ≤Time V ≤Time R2 , execute step 15;

[0069] Step 13: Extract the next frame of optoelectronic track set and its time and assign it to track V and Time V, continue to step 12 to determine the time relationship;

[0070] Step 14: Track the track R2 Assign to track R1 、Time R2 Assign to Time R1 , and extract the next frame radar track set and its time and assign it to track R2 、Time R2 , continue to step 12 to determine the time relationship;

[0071] Step 15, Time R1 、Time R2 The state of all radar tracks in the rectangular coordinate system in the radar track set at the time (x R1 ,y R1 ,z R1 )、(x R2 ,y R2 ,z R2 ) interpolation, obtain the track in Time V Radar track set at the moment R , the state is (x R ,y R ,z R ), and optoelectronic track V Aligned in time, such as Figure 2 shown.

[0072] In the actual implementation process, linear interpolation, quadratic interpolation, Sinc interpolation and other methods may be used. In this embodiment, linear interpolation is taken as an example and the interpolation process is as follows:

[0073] x R =x R1 +Rate·(x R2 -x R1 )

[0074] y R =y R1 +Rate·(y R2 -y R1 )

[0075] z R =z R1 +Rate·(z R2 -z R1 )

[0076] In the formula, x R ,y R and z R Time after interpolation V The track status at the moment, the Rate is calculated as follows:

[0077]

[0078] After time alignment, the spatial coordinates of the time-aligned radar tracks and optoelectronic tracks are transformed into the same coordinate system.

[0079] When converted to the same coordinate system, the radar track coordinates can be converted from the radar rectangular coordinate system to the pixel coordinate system, and the pixel coordinate system of the photoelectric track coordinates can be converted to the radar rectangular coordinate system. In the actual implementation process, there is no restriction on what coordinate form the two coordinates are converted into, as long as they are converted to the same coordinate system.

[0080] In this embodiment, the radar track coordinates are converted from the radar rectangular coordinate system to the pixel coordinate system, and the conversion process is as follows:

[0081] Radar coordinates to camera coordinates (x c ,y c ,z c ), camera coordinates to image coordinates (x, y), image coordinates to pixel coordinates (u, v).

[0082] like Figure 3 As shown, the radar coordinates are obtained by translation and rotation to obtain the camera coordinates:

[0083]

[0084] Among them, α, β, and γ are the radar coordinates around Z R , Y R and X R The rotation angle of the axis, b x , b y and b z For Z R , Y R and X R The amount of translation on the axis.

[0085] like Figure 4 As shown, the camera coordinates are obtained by the pinhole imaging principle to obtain the image coordinates:

[0086]

[0087] Where f is the focal length of the camera.

[0088] like Figure 5 As shown, the image coordinates are translated and scaled to obtain the pixel coordinates:

[0089]

[0090] Among them, dx and dy are camera parameters, indicating the image size represented by each pixel unit, and (u0, v0) is the origin of the image coordinate system, that is, the pixel coordinates at the center of the image.

[0091] Step 2: traverse the radar tracks and optoelectronic tracks in the same coordinate system, match and fuse the radar tracks and optoelectronic tracks that meet the matching judgment conditions, that is, take the union of the radar track data and the optoelectronic track data to obtain the fused track; the matching judgment condition is: the projection distance between the radar track coordinates at the current moment in the radar track and the optoelectronic track coordinates at the current moment in the optoelectronic track is less than the matching threshold set by the system.

[0092] There are two ways to traverse the loop. One is to traverse each radar track, find the photoelectric track that meets the matching judgment conditions with the radar track, and perform matching fusion; if there are multiple photoelectric tracks that meet the matching judgment conditions, select the photoelectric track with the smallest absolute distance between the photoelectric track coordinates at the current moment and the radar track coordinates at the current moment for matching fusion;

[0093] The other is to traverse each optoelectronic track, find the radar track that meets the matching judgment conditions with the optoelectronic track, and perform matching fusion; if multiple radar tracks meet the matching judgment conditions, select the radar track whose absolute distance between the radar track coordinates at the current moment and the optoelectronic track coordinates at the current moment is the smallest for matching fusion.

[0094] In this embodiment, the radar track set Track is traversed. R Each radar track in the optoelectronic track set Track V Find the optoelectronic track that meets the matching judgment conditions with the radar track, and match and fuse them. It should be noted that after the optoelectronic track in the optoelectronic track set is searched out and fused with the radar track, when the next radar track searches for the optoelectronic track to match and fuse with it, the optoelectronic track that has been matched and fused will not be searched out again and fused with the current radar track, that is, for an optoelectronic track, it can only be matched and fused with one radar track, and cannot be matched and fused with multiple radar tracks. Or if this optoelectronic track is not matched and fused, an independent optoelectronic track is generated. Similarly, if the optoelectronic track is traversed to search for a radar track that matches it for matching and fusion, the same is true for the radar track being searched, that is, in the scheme of the present invention, a radar track can only be matched and fused with at most one optoelectronic track.

[0095] The matching judgment condition established is: the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than the matching threshold set by the system. It can be expressed as:

[0096]

[0097] Among them, |·| is the modulo operation, (u V ,v V ) is the photoelectric track state in the pixel coordinate system, (u R ,v R ) is the radar track state in the pixel coordinate system, Thr U With Thr V They are the matching thresholds in the pixel coordinate system respectively. The two matching thresholds are set based on the actual situation, and the specific values ​​can be equal or unequal.

[0098] If no optoelectronic track meets the matching conditions, a radar independent track is established in the fusion track set. F,R , assign a new track number [Id RV ,Id R ,-1]; the radar independent track only includes the track data of the radar track;

[0099] If only one optoelectronic track matches the radar track, the fused track is directly generated by the two optoelectronic and radar tracks as track F,RV , assign a new track number [Id RV ,Id R ,Id V ];

[0100] If multiple optoelectronic tracks match the radar track, calculate the absolute distance between the radar track and the multiple optoelectronic tracks Select the optoelectronic track with the smallest absolute distance Dis to generate the fused track track F,RV , assign a new track number [Id RV ,Id R ,Id V ];

[0101] After the cycle is completed, the remaining unmatched optoelectronic tracks are processed to establish an optoelectronic independent track in the fusion track set. F,V , assign a new track number [Id RV ,-1,Id V In this embodiment, the optoelectronic tracks that are not matched and fused after the traversal cycle will generate optoelectronic independent tracks, which can ensure that the fused track completely contains all radar track data and optoelectronic track data, thereby ensuring the consistency and accuracy of the fused track.

[0102] According to the above process, there are three types of fusion tracks: optoelectronic radar fusion track F,RV , radar independent track F,R Heguang independent track F,V ;

[0103] Photoelectric radar fusion track F,RV It is formed by matching and fusion of radar tracks and optoelectronic tracks that meet the matching judgment conditions; optoelectronic radar fusion track F,RV The track number is [Id RV ,Id R ,Id V ], where Id RV is the track label of the fused track, Id R is the track number of the radar track, Id R is the track number of the optoelectronic track;

[0104] Radar independent track F,R A fused track established based on the radar track that has not been matched and fused with the optoelectronic track after the traversal cycle; radar independent track track F,R The track number is [Id RV ,Id R ,-1], where Id RV is the track label of the fused track, Id R is the track number of the radar track, -1 means there is no optoelectronic track;

[0105] Optical-electronic independent track F,V A fused track established based on the optoelectronic track that has not been matched and fused with the radar track after the traversal cycle; optoelectronic independent track track F,V The track number is [Id RV ,-1,Id V ], where Id RV is the track label of the fused track, -1 means there is no radar track, Id V is the track number of the optoelectronic track;

[0106] Among them, the track label Id of the fused track RV It is set continuously in the order of matching and fusion. The track number Id of the radar track R The track number of the electro-optical track V Always consistent with the target number in the bird target, ID RV 、Id R 、Id V That is, in this embodiment, the radar track is traversed to find the photoelectric track that meets the matching judgment condition for matching fusion.

[0107] For the radar track with track number 1, if no matching optoelectronic track is found, an independent radar track is generated in the fusion track, and its track number is [1,1,-1]. Here, ID RV The value of Id is 1. RThe value of Id is 1. V For the radar track with track number 2, if a matching optoelectronic track is found and the track number of the optoelectronic track is 5, then the optoelectronic radar fusion track in the fusion track is generated, and its track number is [2,2,5]. Here, Id RV The value of Id is 2. R The value of Id is 2. V For the remaining optoelectronic tracks that have not been matched and fused, an optoelectronic independent track is established in the fused track, and its track number is [10, -1, 45]. Here, Id RV The value of 10 means that the 10th matching fusion is the fusion track generated by the remaining optoelectronic tracks. R The value of -1 indicates that it does not exist. V 45 means that the optoelectronic track with track number 45 is not selected, and it is the remaining optoelectronic track that needs to be separately established as a fused track. Through this example, it can be clearly seen that the track label in the track number of the fused track is generated according to the order of loop traversal matching fusion, while the track number of the optoelectronic track and the track number of the radar track in the track number of the fused track correspond to the actual track number. This can further show that the fused track is the union of the optoelectronic track and the radar track, retaining all the information of the original optoelectronic track and radar track. At the same time, in order to distinguish the fused track itself, the track label of the fused track is set in sequence as an identification signal, and the value of the track label is only related to the order of matching fusion. The track label is 1 for the first fusion and 2 for the second fusion. The track label has nothing to do with the type of fused track obtained after fusion.

[0108] Step 3: Update the fused track according to the radar track and optoelectronic track at the next moment to obtain the real-time bird track; repeat this step to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

[0109] Extract the track number of each fused track at the current moment respectively, search for the corresponding radar track and optoelectronic track at the next moment according to the track number, match and fuse them, and obtain a new fused track;

[0110] If the extracted track number is [Id RV ,Id R ,Id V ], the radar track and the photoelectric track at the next moment are matched and fused to obtain a new fused track as the photoelectric radar fused track, with track number [Id RV ,Id R ,Id V ]constant;

[0111] If the extracted track number is [Id RV,Id R ,-1], then traverse each photoelectric track and track number Id R The radar track is matched and fused. If the match is successful, the new fused track is the photoelectric radar fusion track, and the track number is [Id RV ,Id R ,Id V ], where Id RV No change, ID R unchanged; if the match fails, the new fusion track is obtained as a radar independent track, and the track number is [Id RV ,Id R ,-1] unchanged;

[0112] If the extracted track number is [Id RV ,-1,Id V ], then traverse each radar track and track number Id V The photoelectric track is matched and fused. If the match is successful, the new fusion track is the photoelectric radar fusion track, and the track number is [Id RV ,Id R ,Id V ], where Id RV No change, ID V unchanged; if the match fails, the new fusion track is obtained as an optoelectronic independent track, and the track number is [Id RV ,Id R ,-1] unchanged;

[0113] After the fused track is updated, the remaining radar tracks and optoelectronic tracks that have not been matched and fused are traversed and fused to generate a new fused track.

[0114] Specifically, the execution process can be divided into the following steps:

[0115] Step 321, extract the track number of each fused track, including the fused track number Id RV 、Photoelectric track number ID V , Radar track number Id R , respectively according to the track number Id V With Id R In the optoelectronic track set Track V Track with radar track set R Search for the track in the optical track set Track V Including multiple optoelectronic tracks V , radar track set Track R Including multiple radar tracks R .

[0116] If Id V With Id R If the track exists, that is, it is not equal to -1, execute step 322;

[0117] If Id V Track existence, ID R If the track does not exist, it is equal to -1, and step 323 is executed;

[0118] If Id V If the track does not exist, it is equal to -1, Id R If the track exists, execute step 324;

[0119] Step 322, use the searched radar track and optoelectronic track to update the fused track, and the track label Id of the fused track RV constant;

[0120] Step 323, for the track number Id V The optoelectronic track is searched for a match in the radar track set. If the match is successful, the fused track is obtained. F,RV , assign the track number [Id RV ,Id R ,Id V ], track label Id of the fused track RV No change; if the match fails, obtain the optoelectronic independent track in the fusion track set F,V , assign the track number [Id RV ,-1,Id V ], track label Id of the fused track RV constant;

[0121] Step 324, for the track number Id R The radar track is searched for a match in the optoelectronic track set. If a match is successful, the fused track is updated to track F,RV , assign the track number [Id RV ,Id R ,Id V ], track label Id of the fused track RV No change; if the match fails, obtain the radar independent track track in the fusion track set F,R , assign the track number [Id RV ,Id R ,-1], track label Id of the fused track RV constant;

[0122] Step 33, matching the remaining unprocessed radar and optoelectronic tracks, is the same as step 2.

[0123] If the match is successful, a new fusion track is created F,RV, assign a new track number [Id RV ,Id R ,Id V ];

[0124] If the match fails and it is a radar track, create a radar independent track in the fused track set F,R , assign a new track number [Id RV ,Id R ,-1];

[0125] If the match fails and it is an optoelectronic track, establish an optoelectronic independent track in the fusion track set F,V , assign a new track number [Id RV ,-1,Id V ].

[0126] In this embodiment, after the fused track is updated, there will still be radar tracks and optoelectronic tracks that have not been selected for matching and fusion. The remaining radar tracks and optoelectronic tracks that have not been matched and fused are traversed again, matched and fused, and new fused tracks are generated, ensuring that the fused track obtained by updating over time can still contain all the updated radar track data and optoelectronic track data, while ensuring real-time performance, further ensuring continuity and accuracy.

[0127] Fusion track with track F Indicates that multiple fused tracks are track F Composition of fused track set Track F , multiple radar independent tracks F,R Composed radar independent track set Track F,R , multiple optoelectronic independent tracks F,V Form an optoelectronic independent track set Track F,V , multiple optoelectronic radar fusion tracks F,RV Composition of optoelectronic radar fusion track set Track F,RV .

[0128] According to a low-altitude bird detection method based on radar optoelectronic information fusion, the present invention also provides a low-altitude bird detection system based on radar optoelectronic information fusion, the low-altitude bird detection system includes a radar module, an optoelectronic module and a fusion module, and the radar module and the optoelectronic module are used to track low-altitude bird targets in real time. In the fusion module, the radar track and the optoelectronic track are matched and fused to obtain a fused track, that is, a bird track.

[0129] The radar module is used to store radar tracks obtained by radar equipment tracking bird targets and form a radar track set. The optoelectronic module is used to store optoelectronic tracks obtained by optoelectronic equipment tracking bird targets and form an optoelectronic track set. The fusion module is used to store fused tracks formed by matching and fusion of radar tracks and optoelectronic tracks and form a fused track set. The fusion module includes a time alignment unit, a coordinate conversion unit, a matching fusion unit and an update unit.

[0130] The time alignment unit is used to align the time of the radar track and the optoelectronic track.

[0131] Before the matching fusion unit performs matching fusion, the time alignment unit needs to time align the radar track and the optoelectronic track, and the coordinate conversion unit converts the spatial coordinates of the time-aligned radar track and the optoelectronic track into the same coordinate system; after the matching fusion unit performs matching fusion, the fused track is updated by the updating unit.

[0132] The time alignment method is: extract two adjacent frames of radar track data and one frame of photoelectric track data, interpolate the two frames of radar track data to obtain radar track data that is time-aligned with the extracted photoelectric track data; the time corresponding to the extracted radar track data is Time R1 、Time R2 The time corresponding to the extracted photoelectric track data is Time V ,Time R1 ≤Time V ≤Time R2 ;

[0133] Alternatively, two adjacent frames of photoelectric track data and one frame of radar track data are extracted, and the two frames of photoelectric track data are interpolated to obtain photoelectric track data that is time-aligned with the extracted radar track data; wherein the time corresponding to the extracted photoelectric track data is Time V1 、Time V2 , the time corresponding to the extracted radar track data is Time R ,Time V1 ≤Time R ≤Time V2 .

[0134] The specific time alignment steps and interpolation process are the same as those in the above-mentioned bird target detection method and will not be repeated here.

[0135] The coordinate conversion unit is used to convert the spatial coordinates of the radar track and the photoelectric track into the same coordinate system. The specific coordinate conversion process is the same as the steps in the above-mentioned bird target detection method, and will not be repeated here.

[0136] The matching fusion unit is used to match and fuse the radar track and the optoelectronic track that meet the matching judgment conditions, that is, to take the union of the radar track data and the optoelectronic track data, obtain the fused track and store it.

[0137] The matching fusion unit needs to traverse each radar track in the circulating radar module, find the photoelectric track that meets the matching judgment condition with the radar track, and perform matching fusion. Or traverse each photoelectric track in the circulating photoelectric module, find the radar track that meets the matching judgment condition with the photoelectric track, and perform matching fusion. The matching judgment condition is: the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than the matching threshold set by the system. The formula expression is the same as the formula in the above-mentioned bird target detection method, and will not be repeated here. The type and characteristics of the fused track and the track number content are the same as the steps in the above-mentioned bird target detection method, and will not be repeated here.

[0138] The updating unit is used to update the fused track according to the radar track and the photoelectric track at the next moment to obtain the real-time bird track. The specific updating steps are the same as those in the above-mentioned bird target detection method, and will not be repeated here.

[0139] In the low-altitude bird detection system provided by the present invention, when its fusion module performs matching fusion, it needs to be fused through the matching fusion unit first, and then updated through the update unit. However, only when there is no fused track, it needs to be fused through the matching fusion unit first, and when there is a fused track, it needs to be updated through the update unit. Therefore, in the fusion module, it is necessary to first determine whether the fused track set is empty. If it is empty, it is first fused through the matching fusion unit; if it is not empty, the track number is directly extracted through the update unit for update. Its judgment and loop process is as follows: Figure 6 As shown, Figure 6 The "optical / radar track number search" means searching according to the track numbers of the optoelectronic track and the radar track, including three situations and corresponding search methods: both track numbers exist, the optoelectronic track number exists but the radar track number does not exist, and the radar track number exists but the optoelectronic track number does not exist. These have been recorded in the above-mentioned bird target detection method and will not be repeated here. Figure 6 The "remaining optoelectronic / radar track processing" means that the remaining radar tracks and optoelectronic tracks that have not been matched and fused are traversed again, matched and fused, and new fused tracks are generated. The specific processing process has been recorded in the above-mentioned bird target detection method and will not be repeated here.

[0140] After determining whether the fused track set is empty, the specific steps are the same as the matching fusion and track updating process of the above-mentioned bird target detection method, and will not be repeated here.

[0141] The present invention provides a method and system for low-altitude bird detection based on radar optoelectronic information fusion, which synchronizes the radar track with the optoelectronic track through time interpolation alignment, realizes the unification of radar track and optoelectronic track coordinates by converting the radar track or optoelectronic track coordinates, realizes the matching of radar track and optoelectronic track in the same coordinate system, such as pixel coordinate system, and fuses the successfully matched tracks. When the number of targets indicated by the radar track is inaccurate, the accurate number can be identified by optoelectronics, and when the optoelectronic target is blocked and the track is interrupted, continuous tracking can be maintained through the radar track. Thereby, the problems of mistracking and loss of a single sensor (radar or optoelectronic) can be solved, and multiple sensors have richer target information. The information fusion processing method of radar optoelectronic collaborative detection has good tracking effect, simple implementation, and application advantages.

[0142] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A low-altitude bird detection method based on radar optoelectronic information fusion, characterized in that: At the same time, radar equipment and optoelectronic equipment are used to track low-altitude bird targets in real time, and the radar track and optoelectronic track are matched and fused to obtain the fused track, that is, the bird track; The matching fusion process is as follows: one of the radar track set and the optoelectronic track set is recorded as set A, and the other is recorded as set B; set A contains M tracks [a1, a2,, a M ], a m ∈[a1,a2,,a M ], m∈[1,2,,M], set B contains N tracks [b1,b2,,b N ], b n ∈[b1,b2,,b N ], n∈[1,2,,N]; at time t, traverse the M tracks in set A, for each track numbered a m Search the N tracks in set B for the track number a. m The track number of the track that meets the matching judgment condition is b n Matching and fusion of the tracks is to take the union of the track data contained in the track and obtain the track number C q The fused track number is b n The track number is a m After the track number is matched and fused, it is no longer matched with the track number a m+1 The track matching fusion of Q fused tracks constitutes the fused track set [C1,C2,…,C Q ], C q ∈[C1,C2,…,C Q ], q∈[1,2,…,Q], where Q is the maximum value of M and N, M, N, Q are all positive integers, a m , b n , C q Indicates the track number; the track number is C q The track number of the fused track is C q =[c q ,a m ,b n ], where c q It is the track label of the fused track; If there is no b n with a m If the matching judgment condition is met, then according to a m Generate a corresponding independent fusion track, the track number is C q =[c q ,a m ,-1],-1 is b n = -1 means no match to b n ; After traversing all a m After that, if there are b left n , then according to b n Generate a corresponding independent fusion track, the track number is C q =[c q ,-1,b n ], where -1 is a m = -1 means no a m ; After matching and fusion at time t, the fused track is updated according to the set A and set B at time t+1: the track number of each fused track obtained at time t is extracted respectively, and the track number of the fused track is updated according to the track number of a m and b n The value of , searches for the corresponding track in set A and set B at time t+1, replaces the track data in the fused track, and obtains a new fused track; For an independent fused track, if the track number is C q =[c q ,a m ,-1], then traverse the b in set B that has not been matched and fused n With track number a m The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, a m unchanged; if the match fails, the track number at time t+1 is a m The track generates a new independent fusion track, and the track number is C q =[c q ,a m ,-1] unchanged; If the track number is C q =[c q ,-1,b n ], then traverse the a in set A that has not been matched and fused m With track number b n The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, b n unchanged; if the match fails, the track number at time t+1 is b n The track generates a new independent fusion track, and the track number is C q =[c q ,-1,b n ]constant; After the fused track is updated, the remaining tracks in set A and tracks in set B that have not been matched and fused are traversed, and matching and fusion are performed according to the matching and fusion process to generate a new fused track; The process of updating the fused track is repeated to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

2. The low-altitude bird detection method according to claim 1, characterized in that: In traversing a m In the process, for each track number a m If there are multiple tracks with the number b n If the track meets the matching judgment condition, then select b n The track coordinates and a m The absolute distance between the track coordinates and the minimum b n , match and fuse, and generate the fused track with the track number C p =[c p ,a m ,b n ], where c p It is the track label of the fused track.

3. The low-altitude bird detection method according to claim 1, characterized in that: The matching judgment condition is that the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than the matching threshold set by the system.

4. The low-altitude bird detection method according to claim 1, characterized in that: Before the matching and fusion is performed, it is necessary to time-align the radar track and the optoelectronic track, and transform the spatial coordinates of the time-aligned radar track and the optoelectronic track into the same coordinate system.

5. The low-altitude bird detection method according to claim 4, characterized in that: The time-aligned radar track and optoelectronic track are converted into the same coordinate system by converting the radar track coordinates from the radar rectangular coordinate system to the pixel coordinate system or converting the pixel coordinate system of the optoelectronic track coordinates to the radar rectangular coordinate system.

6. A low-altitude bird detection system based on radar optoelectronic information fusion, characterized in that: It includes a radar module, an optoelectronic module and a fusion module; the radar module and the optoelectronic module simultaneously track low-altitude bird targets in real time to obtain radar tracks and optoelectronic tracks; the fusion module matches and fuses the radar tracks and the optoelectronic tracks to obtain a fused track, i.e., the bird track; The fusion module includes a time alignment unit, a coordinate conversion unit, a matching fusion unit and an updating unit; The time alignment unit is used to align the time of the radar track and the photoelectric track; the coordinate conversion unit is used to convert the spatial coordinates of the radar track and the photoelectric track to the same coordinate system; the matching fusion unit is used to match and fuse the radar track and the photoelectric track that meet the matching judgment conditions, that is, to take the union of the radar track data and the photoelectric track data, obtain the fused track and store it; the updating unit is used to update the fused track according to the radar track and the photoelectric track at the next moment, and obtain the real-time bird track; The matching fusion unit performs matching fusion in the following manner: one of the radar track set and the optoelectronic track set is recorded as set A, and the other is recorded as set B; set A contains M tracks [a1, a2, ..., a M ], a m ∈[a1,a2,…,a M ], m∈[1,2,…,M], set B contains N tracks [b1,b2,…,b N ], b n ∈[b1,b2,,b N ], n∈[1,2,,N]; at time t, traverse the M tracks in set A, for each track numbered a m Search the N tracks in set B for the track number a. m The track number of the track that meets the matching judgment condition is b n Matching and fusion of the tracks is to take the union of the track data contained in the track and obtain the track number C q The fused track number is b n The track number is a m After the track number is matched and fused, it is no longer matched with the track number a m+1 The track matching fusion of Q fused tracks constitutes the fused track set [C1,C2,…,C Q ], C q ∈[C1,C2,…,C Q ], q∈[1,2,…,Q], where Q is the maximum value of M and N, M, N, Q are all positive integers, a m , b n , C q Indicates the track number; the track number is C q The track number of the fused track is C q =[c q ,a m ,b n ], where c q It is the track label of the fused track; If there is no b n with a m If the matching judgment condition is met, then according to a m Generate a corresponding independent fusion track, the track number is C q =[c q ,a m ,-1],-1 is b n = -1 means no b n ; After traversing all a m After that, if there are b left n , then according to b n Generate a corresponding independent fusion track, the track number is C q =[c q ,-1,b n ], where -1 is a m = -1 means no a m ; After matching and fusion at time t, according to set A and set B at time t+1, the fused track is updated in the updating unit: the track number of each fused track obtained at time t is extracted respectively, and the track number of the fused track is updated according to a m and b n , searching for the corresponding track in set A and set B at time t+1, replacing the track data in the fused track, obtaining a new fused track, and storing it in the matching fusion unit; For an independent fused track, if the track number is C q =[c q ,a m ,-1], then traverse the b in set B that has not been matched and fused n With track number a m The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, a m unchanged; if the match fails, the track number at time t+1 is a m The track generates a new independent fusion track, and the track number is C q =[c q ,a m ,-1] unchanged; If the track number is C q =[c q ,-1,b n ], then traverse the a in set A that has not been matched and fused m With track number b n The tracks are matched and fused. If the match is successful, the fused track is obtained, and the track number is C q =[c q ,a m ,b n ], where c q unchanged, b n unchanged; if the match fails, the track number at time t+1 is b n The track generates a new independent fusion track, and the track number is C q =[c q ,-1,b n ]constant; After the fused track is updated, the remaining tracks in set A and tracks in set B that have not been matched and fused are traversed, and matching and fusion are performed in the matching and fusion unit according to the matching and fusion process to generate a new fused track; The process of updating the fused track is repeated to complete the real-time tracking of low-altitude bird targets using radar equipment and optoelectronic equipment at the same time.

7. The low-altitude bird detection system according to claim 6, characterized in that: Before the matching fusion unit performs matching fusion, the time alignment unit needs to time align the radar track and the optoelectronic track, and the coordinate conversion unit converts the spatial coordinates of the time-aligned radar track and the optoelectronic track into the same coordinate system; after the matching fusion unit performs matching fusion, the fused track is updated by the updating unit.

8. The low-altitude bird detection system according to claim 6, characterized in that: In traversing a m In the process, for each track number a m If there are multiple tracks with the number b n If the track meets the matching judgment condition, then select b n The track coordinates and a m The absolute distance between the track coordinates and the minimum b n , match and fuse, and generate the fused track with the track number C p =[c p ,a m ,b n ], where c p It is the track label of the fused track.

9. The low-altitude bird detection system according to claim 6, characterized in that: The matching judgment condition is that the projection distance between the radar track coordinates at the current moment in the radar track and the photoelectric track coordinates at the current moment in the photoelectric track is less than the matching threshold set by the system.

Images