A millimeter-wave radar fusion method, system, device and medium

Abstract

The present invention belongs to the field of in-vehicle radar fusion technology, and specifically relates to a millimeter-wave radar fusion method, system, device, and medium. The method includes: obtaining the fusion relationship table X of the previous frame of radar A and radar B n‑1 , as well as the track list A of the current frame of radar A n , and the track list B of the current frame of radar B n ; according to the fusion relationship table X n‑1 , as well as the relative relationship between each track point in the track list A n and each track point in the track list B n , fuse the track list A n and the track list B n to obtain the fused track list C of the current frame n and the fusion relationship table X n ; obtain the association relationship table T of the previous frame of radar A and radar B n‑1 and the track list C n‑1 ; construct the association relationship table T of each track point after fusion of the current frame n . By fusing multiple millimeter-wave radars through the above fusion method, the present invention can improve the accuracy and precision of target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-mounted radar fusion, and particularly relates to a millimeter-wave radar fusion method, system, device and medium. Background Art

[0002] When a millimeter-wave radar performs environmental perception, it often uses multiple radars instead of a single radar, and thus the fusion of multiple radars is required.

[0003] Millimeter-wave radar fusion is a process of information processing that uses computer technology to automatically analyze and synthesize the information and data from multiple millimeter-wave radars under certain criteria to complete the required decision-making and estimation. The fusion process mainly includes: Hardware synchronization, hard synchronization: Using the same hardware to simultaneously issue trigger acquisition commands to achieve time synchronization of the acquisition and measurement of each radar, so as to collect the same information at the same moment. Software synchronization: It includes time synchronization and space synchronization. Among them, time synchronization, timestamp synchronization, and soft synchronization provide a reference time for each radar through a unified host. Each radar adds timestamp information to the independently collected data according to its own calibrated time, and all radar timestamps can be synchronized. However, since the respective acquisition cycles of each radar are independent of each other, it is impossible to ensure that the same information is collected at the same moment. And space synchronization is to convert the measurement values in different radar coordinate systems into the same coordinate system.

[0004] The basic principle of millimeter-wave radar information fusion technology is like the process of the human brain comprehensively processing information. It performs multi-level and multi-space information complementation and optimization combination processing on each radar, and finally generates a consistent interpretation of the observed environment. In this process, the data of multiple millimeter-wave radars should be fully utilized for reasonable allocation and use, and the ultimate goal of fusion is to derive more useful information through multi-level and multi-faceted combination of information based on the separated observation information obtained by each radar. This not only utilizes the advantages of multiple radars operating in cooperation with each other, but also comprehensively processes the data of multiple millimeter-wave radars to improve the intelligence of the entire radar system.

[0005] There is no method for the fusion of multiple millimeter-wave radars in the prior art. Therefore, the detection accuracy of millimeter-wave radars is limited and it is difficult to meet the growing usage needs of users. Summary of the Invention

[0006] In view of the above-mentioned disadvantages of the prior art, the purpose of the present invention is to provide a millimeter-wave radar fusion method, system, device and medium to solve the problems of low fusion rate and insufficient detection accuracy of millimeter-wave radars in the prior art.

[0007] To achieve the above object and other related objects, the present invention provides a millimeter-wave radar fusion method, including the following steps:

[0008] Obtain the fusion relationship table X of the previous frame of radar A and radar B n-1 , and the track list A of the current frame of radar A n , and the track list B of the current frame of radar B n ; n represents the current frame number, n≥2;

[0009] According to the fusion relationship table X n-1 , and the track list A n The relative relationship between each track point in and the track list B n Fuse the track list A n And the track list B n To obtain the fused track list C of the current frame n And the fusion relationship table X n ;

[0010] Obtain the association relationship table T of the previous frame of radar A and radar B n-1 And the track list C n-1 ;

[0011] According to the association relationship table T n-1 , the track list A n , the fusion relationship table X n-1 , the fusion relationship table X n , the track list C n-1 , the track list C n , construct the association relationship table T of each track point after fusion of the current frame n .

[0012] In an optional embodiment of the present invention, the according to the fusion relationship table X n-1 , and the track list A n The relative relationship between each track point in and the track list B n Fuse the track list A n And the track list B n To obtain the track list C of the current frame n And the fusion relationship table X n , including the following steps:

[0013] Obtain the track numbers i, j of the track points of radar A and radar B that were fused with each other in the previous frame from the fusion relationship table X n-1 ;

[0014] Find the track points A with track numbers i, j from the track list A of the current frame n And the track list B n respectively ni、 Bnj ;

[0015] When the track list A n There is the track point A ni , and the track list B n The track point B exists in nj When the track point A ni and the track point B nj Fusion, and put the fused track points into the track list C n , set track point A ni With track point B nj The fusion relationship is put into the fusion relationship table X n ;

[0016] The track list A n and the track list B n The remaining track points in the track list C are fused according to the position relationship, and the fused track points are put into the track list C n , put the fusion relationship of the two fused track points into the fusion relationship table X n .

[0017] In an optional embodiment of the present invention, the track list A n and the track list B n The remaining track points in the track list C are fused according to the position relationship, and the fused track points are put into the track list C n , put the fusion relationship of the two fused track points into the fusion relationship table X n , including the following steps:

[0018] Determine track list A n The remaining track points A nk Is there a track list B within the preset distance? n The remaining track points in ;

[0019] When track list A n The remaining track points A nk There is a track list B within the preset distance around n When there are remaining track points in the list, select the closest track point B. nq With the track point A nk Fusion is performed, otherwise no fusion is performed; track point A nk With track point B nq The fusion relationship is put into the fusion relationship table X n , put the fused track points into track list C n ;

[0020] The track list An and the un-fused track points in the track list B n are directly put into the track list C n .

[0021] In an alternative embodiment of the present invention, the correlation relationship table T n-1 , the track list A n , the track list B n , the fusion relationship table X n-1 , the fusion relationship table X n , the track list C n-1 , the track list C n are used to construct the correlation relationship table T of each track point after fusion in the current frame n , including the following steps:

[0022] Obtain the track number m of the track point C n in the track list C nm ;

[0023] Obtain the correlation relationship of the track point C n-1 with the track number m from the correlation relationship table T (n-1)m ;

[0024] When the track point C (n-1)m is associated with the track point A of radar A (n-1)i , obtain the track number i of the track point A (n-1)i ;

[0025] Judge whether there is a track point A n with the track number i in the track list A ni ;

[0026] If there is a track point A n in the track list A ni , then associate C nm with A ni and put it into the correlation relationship table T n ;

[0027] If there is no track point A n in the track list A ni , judge whether there is a track point A (n-1) in the fusion relationship list X (n-1)i ;

[0028] If there is a track point A (n-1) in the fusion relationship list X (n-1)i , then obtain the track point B (n-1)i fused with the track point A (n-1)j and its track number j;

[0029] Judge the track list B n The track point B with track number j in the middle nj Whether it exists in the fusion relationship list X n ;

[0030] If the fusion relationship list X n There is a track point B in it nj , then obtain the track point A fused with the track point B nj , and associate the track point C np with the track point A nm and put it into the association relationship table T np ; n ;

[0031] If the fusion relationship list X n There is no track point B in it nj , then associate the track point C nm with the track point B nj and put it into the association relationship table T n ;

[0032] Repeat the above steps to traverse all track points in the track list C n ;

[0033] In an optional embodiment of the present invention, when traversing all track points in the track list C n , when there is still a track point Cns that has not established an association relationship, perform the following steps:

[0034] Judge whether there is an unused track point C n-1 in the track list C (n-1)q , the unused track point C (n-1)q refers to the track point C that has established an association relationship nm and has not been used during the establishment of the association relationship.

[0035] When there is such a track point C (n-1)q , associate the track point C ns with the track point C (n-1)q according to the position relationship, and put it into the association relationship table T n ;

[0036] In an optional embodiment of the present invention, the step of associating the track point C ns with the track point C (n-1)q according to the position relationship, and putting it into the association relationship table T n includes the following steps:

[0037] Calculate the track point C(n-1)q The position difference from the track point C ns , if the position difference is less than the preset value, then the track point C (n-1)q is associated with the track point C ns and put into the track association relationship table T n , and use the said position difference as the new preset value;

[0038] Traverse the track list C n , if there is a track point C n in the track list C nt whose position difference from the track C (n-1)q is smaller than the preset value, then use the track point C nt to replace the track point C ns and associate it with C (n-1)q , and update the preset value to the new position difference;

[0039] Update the track association relationship in the track association relationship table T n to be the association between C nt and C (n-1)q .

[0040] Repeat the above steps until all track points in the track list C n are traversed;

[0041] Extrapolate the remaining track points in the track list C (n-1) to the track list C n , and put the remaining track points in the track list C n without established association relationships as new items into the said track association relationship table T n .

[0042] In an optional embodiment of the present invention, when the current frame is the first frame, the fusion relationship table X1, the track list C1 and the association relationship table T1 of the current frame are obtained by the following method:

[0043] Obtain the first-frame track list A1 of the radar A and the first-frame track list B1 of the radar B;

[0044] Fuse the track points in the track list A1 and the track list B1 that meet the preset position relationship respectively, and generate the fusion relationship table X1, the track list C1 and the association relationship table T1 according to the fusion results.

[0045] In an optional embodiment of the present invention, the step of fusing the track points in the track list A1 and the track list B1 that meet the preset position relationship respectively, and generating the fusion relationship table X1, the track list C1 and the association relationship table T1 according to the fusion results includes the following steps:

[0046] Select a track point A from the track list A1 1i , where i is the track number of the track point,

[0047] Determine whether there is a track point in the track list B1 that falls within the preset range of the track point A 1i ;

[0048] When there is a track point in the track list B1 that falls within the preset range of the track point A 1i , select the track point B closest to the track point A 1i from it, 1j fuse the track point B 1j with the track point A 1i , put the fusion relationship into the fusion relationship table X1, and put the fused track point C 1m into the track list C1;

[0049] When there is no track point in the track list B1 that falls within the preset range of the track point A 1i , use the track point A 1i as the fused track point C 1m and put it into the track list C1;

[0050] Associate the track point C 1m with the track point A 1i and put them into the association relationship table T1;

[0051] Repeat the above process until all track points in the track list A1 are traversed;

[0052] If there are remaining track points in the track list B1 at this time, put these track points into the track list C1 and update the association relationship list T1.

[0053] To achieve the above and other related purposes, the present invention also provides a millimeter-wave radar fusion method, including the following steps:

[0054] Fuse the first radar and the second radar using the above millimeter-wave radar fusion method to obtain a first fusion result;

[0055] Fuse the first fusion result and the third radar using the above millimeter-wave radar fusion method.

[0056] To achieve the above and other related purposes, the present invention also provides a millimeter-wave radar fusion method, including the following steps:

[0057] Fuse the first radar and the second radar using the above millimeter-wave radar fusion method to obtain a first fusion result;

[0058] Fuse the third radar and the fourth radar using the above millimeter-wave radar fusion method to obtain a second fusion result;

[0059] Fuse the first fusion result and the second fusion result using the above millimeter-wave radar fusion method.

[0060] To achieve the above object and other related objects, the present invention also provides a millimeter-wave radar fusion system, including:

[0061] A data acquisition module for acquiring the fusion relationship table X of the previous frame of radar A and radar B n-1 , as well as the track list A of the current frame of radar A n , and the track list B of the current frame of radar B n ; n represents the current frame number, n≥2;

[0062] A fusion module for, according to the fusion relationship table X n-1 , and the relative relationship between each track point in the track list A n and each track point in the track list B n , fuse the track list A n and the track list B n to obtain the fused track list C of the current frame n and the fusion relationship table X n ;

[0063] An execution module for acquiring the association relationship table T of the previous frame of radar A and radar B n-1 and the track list C n-1 ; and according to the association relationship table T n-1 , the track list A n , the fusion relationship table X n-1 , the fusion relationship table X n , the track list C n-1 , the track list C n , construct the association relationship table T of each track point after fusion of the current frame n .

[0064] To achieve the above object and other related objects, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and the steps of the method are implemented when the processor executes the computer program.

[0065] To achieve the above object and other related objects, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and characterized in that the steps of the method are implemented when the computer program is executed by a processor.

[0066] The technical effects of the present invention are as follows: When the present invention performs fusion, if the two points fused in the previous frame still exist in the current frame, then the current frame will also fuse these two points; When the present invention performs association, if the track number remains unchanged, then the target with the same track number will be preferentially associated, that is, the tracking result of a single radar is trusted; When the track number of radar B changes and disappears, and the track number of radar A remains unchanged, since the tracking is mainly based on the track number of radar A, no special treatment is performed; When the track number of radar A changes and disappears, first determine whether it is a fusion point. If it is a fusion point and the track number of radar B remains unchanged, during subsequent tracking, the original track will be maintained and the track of radar B will be preferentially associated; When continuously tracking the track of radar B and the track of radar A can be fused with this track at a certain moment, then the original track will be maintained and the track of radar A will be preferentially associated; By fusing multiple millimeter-wave radars through the above fusion method, the present invention can improve the accuracy and precision of target detection. Description of the Drawings

[0067] Figure 1 is a schematic diagram of the distribution of vehicle-mounted radars provided by the embodiment of the present invention;

[0068] Figure 2 is a flowchart of the front-fusion algorithm of the radar;

[0069] Figure 3 is a flowchart of the back-fusion algorithm of the radar;

[0070] Figure 4 is a flowchart of the millimeter-wave radar fusion method provided by the embodiment of the present invention;

[0071] Figure 5 is a schematic diagram of the principle of the millimeter-wave radar fusion process provided by the embodiment of the present invention;

[0072] Figure 6 is a flowchart of the track point fusion process provided by the embodiment of the present invention;

[0073] Figure 7 is a flowchart of fusing track points according to the position relationship provided by the embodiment of the present invention;

[0074] Figure 8 is a flowchart of the method for generating an association relationship table provided by the embodiment of the present invention;

[0075] Figure 9 is a flowchart of establishing an association relationship according to the position relationship provided by the embodiment of the present invention;

[0076] Figure 10 is a specific flowchart of establishing an association relationship according to the position relationship provided by the embodiment of the present invention;

[0077] Figure 11It is the fusion flowchart of the first-frame radar data provided by the embodiments of the present invention

[0078] Figure 12 It is the specific fusion flowchart of the first-frame radar data provided by the embodiments of the present invention;

[0079] Figure 13 It is the multi-radar fusion flowchart provided by the embodiments of the present invention;

[0080] Figure 14 It is the multi-radar fusion flowchart provided by another embodiment of the present invention;

[0081] Figure 15 The structural block diagram of the millimeter-wave radar fusion system provided by the embodiments of the present invention;

[0082] Figure 16 It is the mechanism block diagram of the electronic device provided by the embodiments of the present invention;

[0083] Figure 17 It is the position distribution diagram of one of the frames of track points before and after fusion provided by the embodiments of the present invention;

[0084] Figure 18 It is the position distribution diagram of another frame of track points before and after fusion provided by the embodiments of the present invention;

[0085] Figure 19 It is the fusion principle diagram of the first frame of track points provided by the embodiments of the present invention;

[0086] Figure 20 It is the fusion principle diagram of the second frame of track points provided by the embodiments of the present invention;

[0087] Figures 21 - 26 It is the schematic diagram of the on-vehicle radar distribution provided by different embodiments of the present invention. Specific embodiments

[0088] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0089] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present invention. Therefore, only the components related to the present invention are shown in the illustrations, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0090] Please refer to Figures 1 - 3 As shown, there are mainly two ways of track fusion for millimeter-wave radars. One is the pre-fusion algorithm. As Figure 2 shown, the point track information of each single radar is centralized to the multi-millimeter-wave radar processing center for spatio-temporal alignment, track correlation, data interconnection, track filtering, prediction, and integrated tracking. The biggest advantage of this structure is small information loss, but data correlation is relatively difficult, and it requires the system to have a large-capacity ability, heavy computational burden, and poor survivability of the system. The other is the post-fusion algorithm. As Figure 3 shown, first, the track of each single radar data processor is tracked to generate a single radar track, and then it is sent to the fusion center; the center completes association and fusion to form a multi-millimeter-wave radar fusion track. It not only has the ability of local independent tracking, but also has the ability of global monitoring and evaluation; the cost of the system is relatively low; and it has strong survivability. Therefore, the present invention preferably uses the post-fusion algorithm to fuse the tracks.

[0091] In the process of fusing the data of two radars, if a certain track point of any one radar does not change in two adjacent frames of data, the track point can be fused. The specific fusion situation analysis is listed as follows:

[0092] Radar 1 Radar 2 Fusion situation Unchanged Unchanged Fusable Unchanged Disappeared Fusable Unchanged Appeared Fusable Unchanged Transformed Fusable Disappeared Unchanged Fusable Disappeared Disappeared Non - fusable Disappeared Appeared Non - fusable Disappeared Transformed Non - fusable Appeared Unchanged Fusable Appeared Disappeared Non - fusable Appeared Appeared Non - fusable Appeared Transformed Non - fusable Transformed Unchanged Fusable Transformed Disappeared Non - fusable Transformed Appeared Non - fusable Transformed Transformed Non - fusable

[0093] Please refer to Figure 4 、 5 As shown, a millimeter-wave radar fusion method includes the following steps:

[0094] S1: Obtain the fusion relationship table X of the previous frame of radar A and radar B n-1 , and the track list A of the current frame of the radar A n , and the track list B of the current frame of the radar B n ; n represents the current frame number, n≥2. It should be noted that when the current frame is the first frame, since there is no previous frame data, the fusion process is slightly different, and the fusion process of the first frame data will be described in detail in the subsequent embodiments.

[0095] S2: According to the fusion relationship table X n-1 , and each track point in the track list A n and the track list B nThe relative relationships of the track points in , for the track list A n and the track list B n are fused to obtain the fused track list C n for the current frame and the fusion relationship table X n .

[0096] In this embodiment, the relative relationship may be the track number relationship between track points, and / or the position relationship between track points.

[0097] Please refer to Figure 6 as shown. In an alternative embodiment of the present invention, the step S2 includes the following steps:

[0098] S21: Obtain the track numbers i, j of the track points of the radar A and the radar B that are fused with each other in the previous frame from the fusion relationship table X n-1 ;

[0099] S22: Search for the track points A n and B n with track numbers i, j respectively from the track list A ni and the track list B nj for the current frame;

[0100] S23: When the track point A n exists in the track list A ni , and the track point B n exists in the track list B nj , fuse the track point A ni and the track point B nj , and put the fused track point into the track list C n , and put the fusion relationship between the track point A ni and the track point B nj into the fusion relationship table X n ;

[0101] S24: Fuse the remaining track points in the track list A n and the track list B n according to the position relationship, and put the fused track points into the track list C n , and put the fusion relationship between the two mutually fused track points into the fusion relationship table X n .

[0102] Please refer to Figure 7 as shown. In an alternative embodiment of the present invention, the step S24 specifically includes the following steps:

[0103] S241: Judge the track list A nThe remaining waypoint A in nk Whether there is a waypoint list B within a preset distance around n The remaining waypoints in

[0104] S242: When the waypoint list A n The remaining waypoint A in nk There is a waypoint list B within a preset distance around n When there are remaining waypoints in, select the closest waypoint B nq And the waypoint A nk Perform fusion, otherwise do not perform fusion; put the waypoint A nk And the waypoint B nq The fusion relationship into the fusion relationship table X n Put the fused waypoints into the waypoint list C n ;

[0105] S243: Directly put the unfused waypoints in the waypoint list A n And the waypoint list B n Into the waypoint list C n .

[0106] In a specific embodiment, for example, each time the fusion relationship table X is called n-1 , delete the fusion relationship of this call from the fusion relationship table X n-1 To avoid repeated calls. When the fusion relationship table X n-1 Is empty, it means that all fusion relationships have been called. At this time, the waypoints will be fused according to the positional relationship.

[0107] S3: Obtain the association relationship table T of the previous frame of the radar A and the radar B n-1 And the waypoint list C n-1 ;

[0108] S4: According to the association relationship table T n-1 , the waypoint list A n , the fusion relationship table X n-1 , the fusion relationship table X n , the waypoint list C n-1 , the waypoint list C n , construct the association relationship table T of the fused waypoints in the current frame n .

[0109] Please refer to Figure 8 As shown, in an alternative embodiment of the present invention, the step S4 includes the following steps:

[0110] S41: Obtain the waypoint C in the waypoint list C n innm The track number m;

[0111] S42: From the association relationship table T n-1 Get the track point C with track number m (n-1)m 1. The relationship between

[0112] S43: When track point C (n-1)m Track point A with radar A (n-1)i When associated, obtain track point A (n-1)i The track number i;

[0113] S44: Determine the track list A n Is there a track point A with track number i in ni ;

[0114] S45: If track list A n There is a track point A in ni , then C nm With A ni After association, put it into the association table T n middle;

[0115] S46: If track list A n There is no track point A in ni , determine the fusion relationship list X (n-1) Is there a track point A in ( n -1)i;

[0116] S47: If the fusion relationship list X (n-1) There is a track point A in (n-1)i , then get the value corresponding to track point A (n-1)i Fusion track point B (n-1)j The track number j;

[0117] S48: Determine track list B n Track point B with track number j nj Whether it exists in the fusion relationship list X n middle;

[0118] S49: If the fusion relationship list X n There is a track point B in nj , then get the value corresponding to track point B nj The merged track point A np , and the track point C nm With the track point A np After association, put it into the association table T n middle;

[0119] S410: If the fusion relationship list X nThere is no waypoint B nj , then the waypoint C nm is associated with the waypoint B nj and put into the association relation table T n ;

[0120] S411: Repeat the above steps to traverse all the waypoints in the waypoint list C n .

[0121] It should be noted that in a specific embodiment, for example, every time an association relation is successfully established, the C nm , C (n-1)m , A (n-1)m called in this operation are deleted from their respective tables to avoid repeated calls. After traversing all the waypoints, the remaining data in each table is associated through the following positional relationship.

[0122] Please refer to Figure 9 shown. In an alternative embodiment of the present invention, the step S4 further includes: when there are still unassociated waypoints Cns after traversing all the waypoints in the waypoint list C n , the following steps are executed:

[0123] S412: Determine whether there is an unused waypoint C n-1 in the waypoint list C (n-1)q , where the unused waypoint C (n-1)q refers to the waypoint C nm that has not been used during the establishment of the association relation.

[0124] S413: When there is such a waypoint C (n-1)q , according to the positional relationship, the waypoint C ns is associated with the waypoint C (n-1)q and put into the association relation table T n .

[0125] Please refer to Figure 10 shown. In an alternative embodiment of the present invention, the step S413 includes the following steps:

[0126] S4131: Calculate the positional difference between the waypoint C (n-1)q and the waypoint C ns . If the positional difference is less than the preset value, then the waypoint C (n-1)q is associated with the waypoint C ns and put into the waypoint association relation table T n , and the positional difference is used as the new preset value;

[0127] S4132: Traverse the track list C n , if there is a track point C n in the track list C nt whose position difference from track C (n-1)q is smaller than the preset value, then replace track point C nt with track point C ns associated with C (n-1)q , and re-update the preset value to the new position difference;

[0128] S4133: Update the track association relationship table T n in the track association relationship to be C nt associated with C (n-1)q .

[0129] S4134: Repeat the above steps until all track points in the track list C n are traversed;

[0130] S4135: Extrapolate the remaining track points in the track list C (n-1) into the track list C n , and use the remaining track points in the track list C n that have not established an association relationship as new items and put them into the track association relationship table T n .

[0131] As Figure 17 , 18 shown, in a specific embodiment, it is the track distribution diagrams of radar A and radar B for two consecutive frames of data and the fused track distribution diagram; on the left are the results after data processing of radar A and radar B respectively, the plus sign is the result of radar A, the square is the result of radar B, and the circle on the right is the fused result. The numbers next to the track points in the left figure represent the track numbers. In the right figure, the first group of numbers next to the track points represents the track numbers, the second group of numbers represents the number of times the track point has appeared before, and the third group of numbers represents the number of times the track point has disappeared before. It can be seen that even if the single-radar target track number changes, the fused target track number remains stable and does not change.

[0132] Please refer to Figure 11 shown. In an alternative embodiment of the present invention, when the current frame is the first frame, the fusion relationship table X1, the track list C1, and the association relationship table T1 of the current frame are obtained by the following method:

[0133] S11: Obtain the first-frame track list A1 of the radar A and the first-frame track list B1 of the radar B;

[0134] S12: Fuse the track points in the track list A1 and the track list B1 that meet the preset positional relationship respectively, and generate the fusion relationship table X1, the track list C1 and the association relationship table T1 according to the fusion result.

[0135] Please refer to Figure 12 As shown, in an alternative embodiment of the present invention, the step S12 includes the following steps:

[0136] S121: Select a track point A from the track list A1 1i , where i is the track number of the track point,

[0137] S122: Determine whether there is a track point in the track list B1 that falls within the preset range of the track point A 1i ;

[0138] S123: When there is a track point in the track list B1 that falls within the preset range of the track point A 1i , select the track point B 1i nearest to the track point A 1j , fuse the track point B 1j with the track point A 1i , put the fusion relationship into the fusion relationship table X1, and put the fused track point C 1m into the track list C1;

[0139] S124: When there is no track point in the track list B1 that falls within the preset range of the track point A 1i , use the track point A 1i as the fused track point C 1m and put it into the track list C1;

[0140] S125: Associate the track point C 1m with the track point A 1i and put it into the association relationship table T1;

[0141] S126: Repeat the above process until all track points in the track list A1 are traversed;

[0142] Similarly, in a specific embodiment, each time the above operation is repeated, the track points called in this operation are deleted from the track list to avoid repeated calls.

[0143] S127: If there are remaining track points in the track list B1 at this time, put these track points into the track list C1 and update the association relationship list T1.

[0144] Please refer to Figure 19 ,20 As shown below, the solution of the present invention will be described in conjunction with the fusion process of the second frame of data in a specific embodiment:

[0145] First, obtain the first frame of data as follows:

[0146] Table 1. First Frame Data Table

[0147] Serial number Track list A1 Track list B1 Fusion relationship X1 Fused track list C1 Association relationship table T1 1 A11 B11 1,1 A11 + B11 A11 + B11 2 A12 B12 2,2 A12 + B12 A12 + B12 3 A13 B13 3,3 A13 + B13 A13 + B13 4 A14 B14 4,4 A14 + B14 A14 + B14 5 A15 B15 5,5 A15 + B15 A15 + B15 6 A16 B16 A16 A16 7 A17 B17 A17 A17 8 B16 B16 9 B17 B17

[0148] At time 1, track lists A1 and B1. A11 and B11, A12 and B12, A13 and B13, A14 and B14, A15 and B15 are fused through the position relationship. A16 and A17 have no tracks to fuse with them, and B16 and B17 have no tracks to fuse with them. The fusion relationship is put into X1, the fused result is put into C1, and the unfused tracks are directly put into C1. The association relationship is put into T1.

[0149] The serial numbers in Table 1 and Table 2 can represent the track numbers of each track point.

[0150] The second frame of data obtained by using the above fusion method is as follows:

[0151] Table 2. Second Frame Data Table

[0152] Serial number Track list A2 Track list B2 Fusion relationship X2 Fused track list C2 Association relationship table T2 1 A21 B21 1,1 A21 + B21 A21 + B21 2 A22 B24 2,8 A22 + B28 A22 + B28 3 A23 B25 8,4 A23 A23 4 A26 B26 9,9 A28 + B24 A28 + B24 5 A28 B28 1,1 B25 B25 6 A29 B29 A26 A26 7 A2a B2a A2a A2a 8 B26 B26 9 A29 + B29 A29 + B29

[0153] At time 2, track lists A2 and B2. A21 and B21 are fused through the fusion relationship X1. A22 and B28, A28 and B24, A29 and B29 are fused through the position relationship. A23, A26, and A2a have no tracks to fuse with them, and B26 and B2a have no tracks to fuse with them. The fusion relationship is put into X2, the fused result is put into C2, and the unfused tracks are directly put into C2. The association relationships A21 + B21, A22 + B28 (the serial numbers of A12 and A22 are the same), A23 (the serial numbers of A13 and A23 are the same), A28 + B24 (the serial numbers of B14 and B24 are the same), B25 (the serial numbers of B15 and B25 are the same), A26, and B26 are put into the association relationship T2 through the association relationship T1. A2a is associated with A17 through the position relationship and put into the association relationship T2. B17 has no association and is directly deleted. A29 + B29 and B2a have new association relationships put into T2.

[0154] Please refer to Figure 13 、 14 As shown, based on the above fusion method, the present invention also provides a multi-radar fusion method, including the following steps:

[0155] S100: Use the above millimeter-wave radar fusion method to fuse the first radar and the second radar to obtain the first fusion result;

[0156] S200: The first fusion result and the third radar are fused using the above-mentioned millimeter-wave radar fusion method.

[0157] Or:

[0158] S300: The first radar and the second radar are fused using the above-mentioned millimeter-wave radar fusion method to obtain a first fusion result;

[0159] S400: The third radar and the fourth radar are fused using the above-mentioned millimeter-wave radar fusion method to obtain a second fusion result;

[0160] S500: The first fusion result and the second fusion result are fused using the above-mentioned millimeter-wave radar fusion method.

[0161] The above two multi-radar fusion methods can be used in combination. For example:

[0162] As Figure 21 shown, in this embodiment, Radar 1 and Radar 2 are fused into Result 1.

[0163] As Figure 22 shown, in this embodiment, Radar 1 and Radar 2 are fused into Result 1, Radar 3 and Radar 4 are fused into Result 2; Result 1 and Result 2 are fused into Result 3.

[0164] As Figure 23 shown, in this embodiment, Radar 2 and Radar 3 are fused into Result 1; Radar 1 and Result 1 are fused into Result 2.

[0165] As Figure 24 shown, in this embodiment, Radar 2 and Radar 3 are fused into Result 1, Radar 4 and Radar 5 are fused into Result 2; Result 1 and Result 2 are fused into Result 3; Radar 1 and Result 3 are fused into Result 4.

[0166] As Figure 25 shown, in this embodiment, Radar 1 and Radar 2 are fused into Result 1, Radar 3 and Radar 4 are fused into Result 2; Result 1 and Result 2 are fused into Result 3.

[0167] As Figure 26 shown, in this embodiment, Radar 1 and Radar 2 are fused into Result 1, Radar 3 and Radar 4 are fused into Result 2, Radar 5 and Radar 6 are fused into Result 3; Result 2 and Result 3 are fused into Result 4; Result 1 and Result 4 are fused into Result 5.

[0168] Please refer to Figure 15 shown, the present invention also provides a millimeter-wave radar fusion system, including a data acquisition module 10, a fusion module 20, and an execution module 30. The data acquisition module 10 is used to acquire the fusion relationship table X of the previous frame of Radar A and Radar B n-1, and the track list A of the current frame of the radar A n , and the track list B of the current frame of the radar B n ; n represents the current frame number, n ≥ 2; the fusion module 20 is used to according to the fusion relation table X n-1 , and the track list A n in each track point and the track list B n in the relative relationship of each track point, for the track list A n and the track list B n are fused to obtain the fused track list C of the current frame n and the fusion relation table X n ; the execution module 30 is used to obtain the association relation table T of the previous frame of the radar A and the radar B n-1 and the track list C n-1 ; and according to the association relation table T n-1 , the track list A n , the fusion relation table X n-1 , the fusion relation table X n , the track list C n-1 , the track list C n , construct the association relation table T of each track point after fusion of the current frame n .

[0169] Please refer to Figure 16 shown, the present invention also provides an electronic device, including a memory 200, a processor 100 and a computer program stored on the memory 200 and executable on the processor 100. When the processor 100 executes the computer program, the steps of the method are implemented

[0170] The present invention also provides a computer-readable storage medium, on which a computer program is stored. It is characterized in that when the computer program is executed by a processor, the steps of the method are implemented

[0171] In summary, during the fusion process of the present invention, if the two points fused in the previous frame still exist in the current frame, the current frame will also fuse these two points; the present invention only fuses points between different radars and does not fuse points from the same radar; when fusing the first-frame data, the track number of Radar A is used as the main one, that is, the points of Radar B are fused into Radar A, and the track number after fusion is the track number of Radar A; during the association process of the present invention, if the track number remains unchanged, the targets with the same track number are preferentially associated, that is, the tracking results of a single radar are trusted; when the track number of Radar B changes or disappears and the track number of Radar A remains unchanged, no special treatment is performed because the tracking is mainly based on the track number of Radar A; when the track number of Radar A changes or disappears, first determine whether it is a fused point. If it is a fused point and the track number of Radar B remains unchanged, during subsequent tracking, the original track will be maintained and the track of Radar B will be preferentially associated; when continuously tracking the track of Radar B and at a certain moment the track of Radar A can be fused with this track, then the original track will be maintained and the track of Radar A will be preferentially associated; by fusing multiple millimeter-wave radars through the above fusion method, the present invention can improve the accuracy and precision of target detection.

[0172] The above embodiments are only illustrative of the principles and effects of the present invention and are not used to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

[0173] In the description herein, numerous specific details are provided, such as examples of components and / or methods, to provide a complete understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the embodiments of the present invention can be practiced without one or more of the specific details or by using other devices, systems, components, methods, parts, materials, parts, etc. In other cases, well-known structures, materials, or operations are not specifically shown or described in detail to avoid obscuring aspects of the embodiments of the present invention.

[0174] Throughout the specification, reference to "an embodiment", "embodiment" or "specific embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention and not necessarily in all embodiments. Thus, the appearances of the phrases "in an embodiment", "in an embodiment" or "in a specific embodiment" in various places throughout the specification are not necessarily referring to the same embodiment. In addition, the specific features, structures, or characteristics of any specific embodiment of the present invention can be combined with one or more other embodiments in any suitable manner. It should be understood that other variations and modifications of the embodiments of the invention described and shown herein may be made in accordance with the teachings herein and will be considered part of the spirit and scope of the present invention.

[0175] It should also be understood that one or more of the elements shown in the figures can also be implemented in a more separated or more integrated manner, or even removed because they cannot be operated in some cases or provided because they can be useful according to a specific application.

[0176] In addition, unless otherwise expressly specified, any reference signs in the figures should be regarded only as exemplary and not as limiting. Further, unless otherwise indicated, the term "or" as used herein is generally intended to mean "and / or". In cases where the terms are anticipated to be unclear due to the ability to provide separation or combination, the combination of components or steps will also be regarded as having been specified.

[0177] As used in the description herein and throughout the claims below, unless otherwise indicated, the articles "a" and "the" include plural referents. Also, as used in the description herein and throughout the claims below, unless otherwise indicated, the phrase "in" means "in" and "on".

[0178] The foregoing description of the embodiments of the invention (including what is described in the abstract) is not intended to be exhaustive or to limit the invention to the precise forms disclosed herein. Although specific embodiments of the invention and examples of the invention have been described herein for illustrative purposes only, various equivalent modifications will be apparent to and can be made by those skilled in the art within the spirit and scope of the invention. As noted, these modifications can be made to the invention as described in the foregoing description of the embodiments of the invention, and such modifications will be within the spirit and scope of the invention.

[0179] The systems and methods have been described generally herein to assist in understanding the details of the invention. In addition, various specific details have been given to provide a general understanding of embodiments of the invention. However, those skilled in the relevant art will recognize that embodiments of the invention can be practiced without one or more of the specific details, or with other devices, systems, accessories, methods, components, materials, parts, etc. In other instances, well-known structures, materials, and / or operations have not been shown or described in detail to avoid obscuring aspects of the embodiments of the invention.

[0180] Accordingly, while the present invention has been described herein with reference to its specific embodiments, modifications, various changes and substitutions are also within the above disclosure, and it should be understood that in some cases, some features of the present invention will be employed without corresponding use of other features without departing from the scope and spirit of the claimed invention. Accordingly, many modifications may be made to adapt a particular situation or material to the essential scope and spirit of the present invention. The present invention is not intended to be limited to the specific terms used in the following claims and / or to the specific embodiments disclosed as the best mode contemplated for carrying out the present invention, but the present invention will include any and all embodiments and equivalents falling within the scope of the appended claims. Accordingly, the scope of the present invention will be determined only by the appended claims.

Claims

1. A millimeter-wave radar fusion method, characterized in that, It includes the following steps: Obtain the pre - obtained fusion relationship table X of the previous frame of radar A and radar B n-1 , and the track list A of the current frame of radar A n , and the track list B of the current frame of radar B n ; The fusion relationship table X n-1 includes the track numbers of the track points of radar A and radar B that were fused with each other in the previous frame. n represents the current frame number, and n≥2; According to the fusion relationship table X n-1 , and the track list A n The relative relationship between each track point in and the track list B n in, for the track list A n and the track list B n are fused, and the fused track list C n for the current frame and the fusion relationship table X n are generated; Obtain the association relation table T of the previous frame of the radar A and the radar B n-1 and the fused track list C n-1 ; According to the association relation table T n-1 , the track list A n , the track list B n , the fusion relation table X n-1 , the fusion relation table X n , the track list C n-1 , the track list C n , construct the association relation table T of each track point after fusion in the current frame n ; the association relation table T n includes the association relation between the track points after fusion in the current frame and the track points of radar A or radar B; According to the fusion relation table X n-1 , and the track list A n The relative relationship between each track point in and the track list B n Fuse the track list A n and the track list B n to obtain the track list C of the current frame n and the fusion relation table X n , including the following steps: Obtain the track numbers i and j of the track points of the radar A and the radar B that are fused with each other in the previous frame from the fusion relationship table X n-1 ; From the track list A of the current frame n and the track list B n respectively find the track points A with track numbers i and j ni、 B nj ; When there is the track point A n in the track list A ni , and there is the track point B n in the track list B nj , fuse the track point A ni and the track point B nj , and put the fused track point into the track list C n , and put the fusion relationship between the track point A ni and the track point B nj into the fusion relationship table X n ; Fuse the remaining track points in the track list A n and the track list B n according to their positional relationships, and put the fused track points into the track list C n , and put the fusion relationships of the two mutually fused track points into the fusion relationship table X n ; The said according to the associated relationship table T n-1 、the said track list A n 、the said track list B n 、the said fusion relationship table X n-1 、the said fusion relationship table X n 、the said track list C n-1 、the said track list C n , construct the associated relationship table T of each track point after fusion in the current frame n , including the following steps: Obtain the track list C n The track point C in the middle nm Track number m; Retrieve the association relationship of the track point C with track number m from the association relationship table T n-1 ; (n-1)m The association relationship When the track point C (n-1)m is associated with the track point A (n-1)i of radar A, obtain the track number i (n-1)i of the track point A; Determine whether there is a track point A with track number i in the track list A n ni ;​ If there is a track point A in the track list A n then C ni will be associated with A nm and put into the association relation table T ni ; n ​ If there is no track point A in the track list A n , determine whether there is a track point A in the fusion relationship list X ni ; (n-1) ; (n-1)i ; If there is a track point A in the fusion relationship list X (n-1) , then obtain the track number j of the track point B (n-1)i that is fused with the track point A (n-1)i ; (n-1)j ​ Judge the track list B n The track point B with track number j in nj Whether it exists in the fusion relationship list X n ; If there is a track point B n in the fusion relationship list X nj , then obtain the track point A nj fused with the track point B np , and associate the track point C nm with the track point A np and put it into the association relationship table T n ; If the fusion relationship list X n does not contain the track point B nj , then the track point C nm is associated with the track point B nj and then put into the association relationship table T n ; Repeat the above steps to traverse all the track points in the track list C n ​ 2. The millimeter-wave radar fusion method according to claim 1, wherein Said track list A n and said track list B n The remaining track points in are fused according to their positional relationships, and the fused track points are placed in said track list C n The fusion relationships between the two mutually fused track points are placed in said fusion relationship table X n comprising the following steps: Judge the remaining track points A in track list A n whether there are remaining track points in track list B nk within a preset distance around it n ; When there are remaining track points A in the track list A n and there are remaining track points in the track list B within a preset distance around them nk , select the nearest track point B n and fuse it with the track point A nq , otherwise do not fuse; put the track point A nk and the track point B nk into the fusion relationship table X nq as their fusion relationship, and put the fused track points into the track list C n ; n ​ Put the un-fused track points in the track list A n and the track list B n directly into the track list C n .

3. The millimeter-wave radar fusion method according to claim 1, characterized in that After traversing all the track points in the track list C n if there are still track points C for which no association relationship has been established ns perform the following steps: Determine whether there are unused track points C in the track list C n-1 , where the unused track points C (n-1)q refer to the track points C for which an association relationship has been established (n-1)q and which were not used during the establishment of the association relationship; nm ​ When the track point C exists (n-1)q At this time, according to the positional relationship, the track point C ns is associated with the track point C (n-1)q and put into the association relationship table T n .

4. The millimeter-wave radar fusion method according to claim 3, wherein According to the positional relationship, the track point C ns is associated with the track point C (n-1)q and put into the association relationship table T n The process includes the following steps: Calculate the track point C (n-1)q The position difference from the track point C ns If the position difference is less than the preset value, then the track point C (n-1)q After associating with the track point C ns Put it into the track association relationship table T n , and use the position difference as the new preset value; Traverse the track list C n , if there is a track point C n in the track list C nt whose position difference from track C (n-1)q is smaller than the preset value, then use track point C nt to replace track point C ns associated with C (n-1)q and re-update the preset value to the new position difference; Update the track association relation table T n The track association relation in it is C nt is associated with C (n-1)q be associated; Repeat the above steps until all the track points of the track list C n are traversed; Extrapolate the remaining track points in the track list C to the track list C (n-1) In, take the remaining unassociated track points in the track list C n In the track list C n As new items and put them into the track association relationship table T n .

5. The millimeter-wave radar fusion method according to claim 1, characterized in that when the current frame is the first frame, the fusion relationship table X1, the track list C1 and the association relationship table T1 of the current frame are obtained by the following method: Obtain the first-frame track list A1 of radar A and the first-frame track list B1 of radar B; Fuse the track points in the track list A1 and the track list B1 that satisfy the preset position relationship respectively, and generate the fusion relationship table X1, the track list C1 and the association relationship table T1 according to the fusion result.

6. The millimeter-wave radar fusion method according to claim 5, wherein The step of fusing the track points in the track list A1 and the track list B1 that satisfy the preset position relationship respectively, and generating the fusion relationship table X1, the track list C1 and the association relationship table T1 according to the fusion result includes the following steps: Select the track point A from the track list A1 1i , where i is the track number of the track point Determine whether there is a track point in the track list B1 that falls within the preset range of the track point A 1i ; When there is a track point in the track list B1 that falls within the preset range of the track point A 1i , select the track point B 1i that is closest to the track point A 1j , fuse the track point B 1j with the track point A 1i , put the fusion relationship into the fusion relationship table X1, and put the fused track point C 1m into the track list C1; When there are no track points in the track list B1 that fall within the preset range of the track point A 1i , the track point A 1i is used as the fused track point C 1m and placed in the track list C1; Associate the track point C 1m with the track point A 1i and then put them into the association relationship table T1; Repeat the above process until all track points in the track list A1 are traversed; If there are remaining track points in the track list B1 at this time, put these track points into the track list C1 and update the association relationship list T1.

7. A millimeter-wave radar fusion method, characterized in that, It includes the following steps: Fuse the first radar and the second radar by using the millimeter-wave radar fusion method according to any one of claims 1 to 6 to obtain a first fusion result; Fuse the first fusion result and the third radar by using the millimeter-wave radar fusion method according to any one of claims 1 to 6.

8. A millimeter-wave radar fusion method, characterized in that, It includes the following steps: Fuse the first radar and the second radar by using the millimeter-wave radar fusion method according to any one of claims 1 to 6 to obtain a first fusion result; Fuse the third radar and the fourth radar by using the millimeter-wave radar fusion method according to any one of claims 1 to 6 to obtain a second fusion result; Fuse the first fusion result and the second fusion result by using the millimeter-wave radar fusion method according to any one of claims 1 to 6.

9. A millimeter-wave radar fusion system, characterized in that, It includes: A data acquisition module, configured to obtain a pre-generated fusion relationship table X of the previous frame of radar A and radar B n-1 , as well as the track list A of the current frame of radar A n , and the track list B of the current frame of radar B n ; the fusion relationship table X n-1 includes the track numbers of the track points of radar A and radar B that were fused with each other in the previous frame, where n represents the current frame number and n≥2; A fusion module, configured to, according to the fusion relationship table X n-1 , and the track list A n the relative relationship between each track point in and the track list B n in, fuse the track list A n and the track list B n to obtain the fused track list C of the current frame n and the fusion relationship table X n ; According to the fusion relationship table X n-1 , and the track list A n The relative relationship between each track point in and the track list B n Perform fusion on the track list A n And the track list B n To obtain the track list C of the current frame n And the fusion relationship table X n , including the following steps: Obtain the track numbers i and j of the track points of the radar A and the radar B that are fused with each other in the previous frame from the fusion relationship table X n-1 ; From the track list A of the current frame n and the track list B n respectively find the track points A with track numbers i and j ni、 B nj ; When there is the track point A n in the track list A ni , and there is the track point B n in the track list B nj , fuse the track point A ni and the track point B nj , and put the fused track point into the track list C n , and put the fusion relationship between the track point A ni and the track point B nj into the fusion relationship table X n ; The remaining track points in the track list A n and the track list B n are fused according to the positional relationship, and the fused track points are placed in the track list C n , and the fusion relationship between the two mutually fused track points is placed in the fusion relationship table X n ; An execution module, configured to obtain the association relationship table T of the previous frame of the radar A and the radar B n-1 and the track list C n-1 ; and according to the association relationship table T n-1 , the track list A n , the track list B n , the fusion relationship table X n-1 , the fusion relationship table X n , the track list C n-1 , the track list C n , construct the association relationship table T of each fused track point in the current frame n ; the association relationship table T n includes the association relationship between the fused track points in the current frame and the track points of the radar A or the radar B; According to the associated relationship table T n-1 , the track list A n , the track list B n , the fusion relationship table X n-1 , the fusion relationship table X n , the track list C n-1 , the track list C n , construct the associated relationship table T of each track point after fusion in the current frame n , including the following steps: Obtain the track list C n The middle track point C nm Track number m; Retrieve the association relationship of the track point C with track number m from the association relationship table T n-1 ; (n-1)m Retrieve the association relationship of the track point C with track number m from the association relationship table T When the track point C (n-1)m is associated with the track point A (n-1)i of radar A, obtain the track number i (n-1)i of the track point A; Determine whether there is a track point A with track number i in the track list A n ; ni ; If there is a track point A in the track list A n , then C ni is associated with A nm and then put into the association relationship table T ni ; n ​ If there is no track point A in the track list A n , determine whether there is a track point A in the fusion relationship list X ni ; (n-1) ; (n-1)i ; If there is a track point A in the fusion relationship list X (n-1) , then obtain the track number j of the track point B (n-1)i that is fused with the track point A (n-1)i ; (n-1)j ​ Judge the track list B n The track point B with track number j in the middle nj Whether it exists in the fusion relationship list X n ; If there is a track point B n in the fusion relationship list X nj , then obtain the track point A nj fused with the track point B np , and associate the track point C nm with the track point A np and put it into the association relationship table T n ; If the list of fusion relationships X n does not contain the track point B nj , then the track point C nm is associated with the track point B nj and then placed in the association relationship table T n ; Repeat the above steps to traverse all the track points in the track list C n .

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

Images