Radar and vision target robust association method for missing measurement data

Abstract

The application discloses a radar and visual target robust association method for measurement data loss, first, radar and visual data are preprocessed to obtain a radar and visual measurement set which is spatially aligned and time-synchronized, then a radar and visual extended measurement set is constructed, a multi-hypothesis extended association result is obtained by using a Hungarian algorithm and statistical analysis is performed, a high-stability association target area and a secondary-stability candidate area under the multi-hypothesis extended association result are divided, finally, a high-stability and high-consistency association result is screened out based on association frequency and variance consistency, and a complete and robust association result is constructed. The method of the application considers multiple scenarios where measurement loss exists by extending the measurement, optimizes the radar and visual target similarity disorder and association jump caused by the measurement loss phenomenon, significantly improves the reliability of radar and visual fusion perception in traffic target perception detection and other scenes, and is suitable for automatic driving and traffic supervision and other scenes which require radar and visual fusion.

Description

Technical Field

[0001] This invention belongs to the field of multi-sensor fusion technology, specifically relating to a robust method for correlating radar and visual targets in the event of loss of surface-based measurement data. Background Technology

[0002] Millimeter-wave radar (radar for short) and vision sensors are the most common perception sensors in intelligent transportation and autonomous driving fields due to their low cost and mature technology. However, millimeter-wave radar lacks the ability to detect target texture and morphological information, making it difficult to achieve reliable target detection and classification. A single vision sensor cannot robustly extract motion state information (speed and distance), which significantly limits its reliability in perceiving the adjacent environment. Research on the fusion of radar and vision, such as the fusion of radar and vision sensors, is an important way to break down the barriers of single sensors and achieve highly reliable, multi-level, and multi-attribute environmental perception.

[0003] Radar and vision fusion can complement and integrate multi-source detected targets, and also enrich and expand target attribute information to construct a comprehensive adjacent environment perception domain with detailed target counts and complete target attribute information. Achieving radar and vision target fusion relies on target association processing, i.e., determining whether measurements from different sensors originate from the same target. This allows for the fusion and attribute expansion of measurements from the same target, while supplementing and improving measurements from different targets. Due to the different environmental perception methods of radar and vision, their measurement errors and stability vary, greatly complicating the difficulty of target association. Measurement loss is one of the main factors affecting target association and is easily coupled with factors such as measurement offset and false alarm points, leading to unstable association results. The paper "Radar-Vision Fusion for Vehicle Detection and Tracking. 2023 International Applied Computational Electromagnetics Society Symposium (ACES), 1-2" projects radar measurements onto images to perform target association. This method can effectively reduce the impact of inaccurate depth information from monocular vision. However, the information in the image plane cannot be directly used for downstream control and decision-making, and lacks practical application value. The paper "Robust Multiobject Tracking Using Mmwave Radar-Camera Sensor Fusion. IEEE Sensors Letters, 6(10), 1-4" is based on world coordinates and uses the L2 norm as the cost to measure the similarity between radar and visual targets. It uses the Hungarian algorithm to complete the association between radar and vision. However, it ignores the phenomenon of sensor measurement loss and is difficult to achieve robust association between radar and visual targets in complex multi-target scenarios. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a robust radar-visual target association method for areas with measurement data loss. This method solves the problem of measurement loss causing abrupt changes in radar-visual target association, which is neglected by existing methods, thereby achieving reliable radar-visual target association fusion.

[0005] The technical solution adopted in this invention is: a robust association method between radar and visual targets in the event of surface vector measurement data loss, the specific steps of which are as follows:

[0006] S1. Preprocess radar and visual data to obtain spatially aligned and time-synchronized radar and visual measurement sets;

[0007] Install radar and vision sensors, and use Zhang Zhengyou's calibration method to calibrate the sensors to obtain visual intrinsic and extrinsic parameters and radar extrinsic parameter data.

[0008] The system acquires raw images from the camera, uses a neural network to perform target detection on the visual images, and obtains visual measurements of pixel coordinates. It also acquires radar echo data, removes background clutter and zero-Doppler targets based on frequency domain information, and analyzes the data to obtain radar measurements.

[0009] The acquired intrinsic and extrinsic parameters are used to transform the radar and visual measurements into a unified coordinate system. Then, the measurements are synchronized to the same moment according to the sensor time, resulting in a spatially aligned and time-synchronized set of radar and visual measurements.

[0010] Then, the radar and visual measurement sets are filtered, and the filtered and saved radar and visual measurement points are correlated, as shown in the following expression:

[0011]

[0012] in, Represents the radar measurement set, r i express The i-th radar measurement point, N r Indicates the number of radar measurement points; Represents the visual measurement set, v i express The i-th visual measurement point, N v This indicates the number of visual measurement points.

[0013] S2. Construct extended radar and visual measurement sets;

[0014] N is selected from all elements of the radar measurement set. r -1 elements are used to construct a measurement subset, with m = 1, 2, ..., N in sequence. r The subsets of radar measurements under the given conditions collectively constitute the extended radar measurement set; similarly, all subsets of visual measurements collectively constitute the extended visual measurement set, as expressed below:

[0015]

[0016] in, and These represent the measurement subsets formed by the absence of the m-th measurement in the radar and visual measurement sets, respectively.

[0017] S3. Perform Hungarian target association based on the extended measurement set obtained in step S2, and adaptively generate multi-hypothesis extended association results;

[0018] Based on the extended radar and visual measurement sets, Hungarian target association is performed, and the association results of a single radar measurement subset and a visual measurement subset are obtained sequentially, as shown in the following expression:

[0019]

[0020] in, Indicates based on radar measurement set With visual measurement set The Hungarian association results, i a represents the i-th radar-visual association pair, ( i r, i v) represents the corresponding association pair. i r represents the corresponding radar measurement point in the associated pair. i v represents the corresponding visual measurement point in the association pair, and N represents the number of association pairs, with a maximum value of N = min(N). r N v ), which represents the minimum number of elements in the radar and visual measurement sets; H represents the Hungarian association algorithm; and These represent the association results generated based on the corresponding measurement sets.

[0021] Based on the difference in the number of measurement points between radar and visual measurement sets, multiple hypotheses for potential measurement loss are established. These hypotheses are then coupled with an expanded measurement set to adaptively generate extended association results between radar and vision under multiple hypotheses. The extended association results are a set of elements that can be repeated, expressed as follows:

[0022]

[0023] in, This represents the extended correlation results between radar and visual measurement sets for a specific number of samples. j a represents the j-th association pair in the expanded association results, ( j r, j v) represents the radar and visual measurement points of the corresponding correlation pair, N m ∪ represents the number of all association pairs (including duplicate elements) in the expanded association result; ∪ represents the cumulative union operation.

[0024] S4. Statistically analyze the results of multi-hypothesis extended association and divide the highly stable association target region and the sub-stable candidate region under the results of multi-hypothesis extended association.

[0025] The multi-hypothesis extended association results of statistical radar and visual measurement sets are used to extract all unique association pairs in the extended association results and construct mutually exclusive association results, denoted as... The expression is as follows:

[0026]

[0027] in, k 'a' represents the k-th association pair in the mutually exclusive association results, and its corresponding radar and visual measurements are ( k r, k v), N′ m This indicates the number of association pairs in the result of a mutually exclusive association.

[0028] Mutually exclusive association results Based on this, statistics are compiled on its extended association results. The frequency of occurrence in is expressed as follows:

[0029]

[0030] Where G(·) represents the frequency statistics function; n k Indicates related pairs k a in Frequency in; express The frequency set of all elements in the set.

[0031] Sort the frequency set of mutually exclusive association results to obtain association results in descending order of frequency, as shown in the following expression:

[0032]

[0033] in, This represents mutually exclusive association results sorted in descending order of frequency. l 'a' represents the association pair ranked at the l-th position in frequency order. l r, l v) represents the corresponding radar and visual measurement points; S des (·) denotes a frequency descending function. Indicates based on Attributes right Sort in descending order.

[0034] The maximum number of correctly associated pairs can be calculated based on the number of radar and visual measurement points, denoted as N = min(N r N v Couple the results of the maximum association pairs with the association results in descending order of frequency, and divide the mutually exclusive association results into target regions with high stability and candidate regions under multiple hypotheses, as shown in the following expression:

[0035]

[0036]

[0037] in, N represents the set of association pairs belonging to the target region;t This represents the largest index that is divided into target regions in the mutually exclusive association results in descending frequency order, i.e. Middle front N t A number of correlation pairs are included in the target region, and N t Satisfies equation (9); G(·) represents the frequency statistics function; This represents the set of association pairs belonging to the candidate region.

[0038] S5. Based on the association frequency and variance consistency, select association results with high stability and strong consistency, and construct complete and robust association results;

[0039] The variance of stable association pairs is calculated based on target region association pairs, and the expression is as follows:

[0040]

[0041] in, This represents the variance of the distance between target region association pairs; D( represents the mean distance between target region association pairs; l a) Indicates a related pair l The distance calculation function for a.

[0042] Next, select association pairs that satisfy variance consistency from the candidate region association pairs. Iterate through all association pairs, filter out association pairs whose association costs exceed the threshold, and construct a joint association result based on the maximum number of association pairs, as shown in the following expression:

[0043]

[0044]

[0045] in, This indicates the supplementary association pair selected from the candidate region, whose generation condition satisfies equation (11); n a represents the nth additional association pair; S asc (·) denotes a descending order function to achieve a descending order of the consistency variance of candidate region associations. Indicates based on Attribute consistency variance Sort the data; topK(·) represents the set of the top K associative pairs after sorting, where K = N. t ; This indicates the final, complete association result.

[0046] The beneficial effects of this invention are as follows: The method of this invention first preprocesses radar and visual data to obtain spatially aligned and time-synchronized radar and visual measurement sets. Then, it constructs extended radar and visual measurement sets, uses the Hungarian algorithm to obtain multi-hypothesis extended association results, and performs statistical analysis. It then divides the multi-hypothesis extended association results into highly stable association target regions and sub-stable candidate regions. Finally, based on association frequency and variance consistency, it selects association results with high stability and strong consistency, constructing complete and robust association results. This invention, by extending the measurements, considers various scenarios where measurements are lost, optimizing the problems of radar and visual target similarity disorder and association jumps caused by measurement loss. It significantly improves the reliability of radar and visual fusion perception in scenarios such as traffic target perception and detection, and is suitable for scenarios requiring radar and visual fusion, such as autonomous driving and traffic monitoring. Attached Figure Description

[0047] Figure 1 This is a flowchart of a robust method for correlating radar and visual targets in the event of surface vector measurement data loss, according to the present invention.

[0048] Figure 2 This is a simulation of a lane-changing and overtaking scenario and its corresponding attribute information diagram in an embodiment of the present invention.

[0049] Figure 3 This is a statistical chart of correlation indicators for the simulation scenario in this embodiment of the invention.

[0050] Figure 4 This is a graph showing the change in overall accuracy as a function of measurement loss probability in an embodiment of the present invention.

[0051] Figure 5 This is a diagram showing the simulation scene association results in an embodiment of the present invention.

[0052] Figure 6 This is a diagram showing the extended correlation results of the actual test scenario in an embodiment of the present invention.

[0053] Figure 7 This is a comparison chart of the final association results of the actual test scenarios in the embodiments of the present invention. Detailed Implementation

[0054] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0055] This embodiment is verified based on simulation and measured data, and all steps and conclusions have been verified to be correct in Python.

[0056] like Figure 1 The flowchart shown is a robust association method between radar and visual targets for surface vector measurement data loss according to the present invention. The specific steps are as follows:

[0057] S1. Preprocess radar and visual data to obtain spatially aligned and time-synchronized radar and visual measurement sets;

[0058] Install radar and vision sensors, and use Zhang Zhengyou's calibration method to calibrate the sensors to obtain visual intrinsic and extrinsic parameters and radar extrinsic parameter data.

[0059] The system acquires raw images from the camera, uses a neural network to perform target detection on the visual images, and obtains visual measurements of pixel coordinates. It also acquires radar echo data, removes background clutter and zero-Doppler targets based on frequency domain information, and analyzes the data to obtain radar measurements.

[0060] The acquired intrinsic and extrinsic parameters are used to transform the radar and visual measurements into a unified coordinate system. Then, the measurements are synchronized to the same moment according to the sensor time, resulting in a spatially aligned and time-synchronized set of radar and visual measurements.

[0061] Then, the radar and visual measurement sets are filtered separately to satisfy a certain neighborhood range (D). t Isolated measurements without another sensor measurement point within the same area are considered as single sensor capture points and are not involved in correlation, as detailed below:

[0062]

[0063] Where, r i N represents the i-th radar measurement point. r0 Indicates the number of unfiltered radar measurement points; v i N represents the j-th visual measurement point. v0 D represents the number of unfiltered visual measurement points; D(·) represents the distance calculation function between measurement points. t This represents the threshold range of the neighborhood.

[0064] The filtered and saved radar and visual measurement points are correlated using the following expression:

[0065]

[0066] in, r represents the filtered and stored radar measurement set. i express The i-th radar measurement point, N r This indicates the number of radar measurement points saved by the filter. v represents the set of visual measurements saved by filtering. i express The i-th visual measurement point, N v This indicates the number of visual measurement points saved by the filter.

[0067] S2. Construct extended radar and visual measurement sets;

[0068] N is selected from all elements of the radar measurement set. r -1 elements are used to construct a measurement subset, with m = 1, 2, ..., N in sequence. r The subsets of radar measurements under the given conditions collectively constitute the extended radar measurement set; similarly, all subsets of visual measurements collectively constitute the extended visual measurement set, as expressed below:

[0069]

[0070] in, and These represent the measurement subsets formed by the absence of the m-th measurement in the radar and visual measurement sets, respectively.

[0071] S3. Perform Hungarian target association based on the extended measurement set obtained in step S2, and adaptively generate multi-hypothesis extended association results;

[0072] Based on the extended radar and visual measurement sets, Hungarian target association is performed, and the association results of a single radar measurement subset and a visual measurement subset are obtained sequentially, as shown in the following expression:

[0073]

[0074] in, Indicates based on radar measurement set With visual measurement set The Hungarian association results, i a represents the i-th radar-visual association pair, ( i r, i v) represents the corresponding association pair. i r represents the corresponding radar measurement point in the associated pair. i v represents the corresponding visual measurement point in the association pair, and N represents the number of association pairs, with a maximum value of N = min(N). r N v ), which represents the minimum number of elements in the radar and visual measurement sets; H represents the Hungarian association algorithm; and These represent the association results generated based on the corresponding measurement sets.

[0075] Based on the difference in the number of measurement points between radar and visual measurement sets, multiple hypotheses for potential measurement loss are established. These hypotheses are then coupled with an expanded measurement set to adaptively generate extended association results between radar and vision under multiple hypotheses. The extended association results are a set of elements that can be repeated, expressed as follows:

[0076]

[0077] in, This represents the extended correlation results between radar and visual measurement sets for a specific number of samples. ja represents the j-th association pair in the expanded association results, ( j r, j v) represents the radar and visual measurement points of the corresponding correlation pair, N m ∪ represents the number of all association pairs (including duplicate elements) in the expanded association result; ∪ represents the cumulative union operation.

[0078] S4. Statistically analyze the results of multi-hypothesis extended association and divide the highly stable association target region and the sub-stable candidate region under the results of multi-hypothesis extended association.

[0079] The multi-hypothesis extended association results of statistical radar and visual measurement sets are used to extract all unique association pairs in the extended association results and construct mutually exclusive association results, denoted as... The expression is as follows:

[0080]

[0081] in, k 'a' represents the k-th association pair in the mutually exclusive association results, and its corresponding radar and visual measurements are ( k r, k v), N′ m This indicates the number of association pairs in the result of a mutually exclusive association.

[0082] Mutually exclusive association results Based on this, statistics are compiled on its extended association results. The frequency of occurrence in is expressed as follows:

[0083]

[0084] Where G(·) represents the frequency statistics function; n k Indicates related pairs k a in Frequency in; express The frequency set of all elements in the set.

[0085] Sort the frequency set of mutually exclusive association results to obtain association results in descending order of frequency, as shown in the following expression:

[0086]

[0087] in, This represents mutually exclusive association results sorted in descending order of frequency. l 'a' represents the association pair ranked at the l-th position in frequency order. l r, l v) represents the corresponding radar and visual measurement points; S des (·) denotes a frequency descending function. Indicates based on Attributes right Sort in descending order.

[0088] The maximum number of correctly associated pairs can be calculated based on the number of radar and visual measurement points, denoted as N = min(N r N v Couple the results of the maximum association pairs with the association results in descending order of frequency, and divide the mutually exclusive association results into target regions with high stability and candidate regions under multiple hypotheses, as shown in the following expression:

[0089]

[0090]

[0091] in, N represents the set of association pairs belonging to the target region; t This represents the largest index that is divided into target regions in the mutually exclusive association results in descending frequency order, i.e. Middle front N t A number of correlation pairs are included in the target region, and N t Satisfies equation (10); G(·) represents the frequency statistics function; This represents the set of association pairs belonging to the candidate region.

[0092] S5. Based on the association frequency and variance consistency, select association results with high stability and strong consistency, and construct complete and robust association results;

[0093] The variance of stable association pairs is calculated based on target region association pairs, and the expression is as follows:

[0094]

[0095] in, This represents the variance of the distance between target region association pairs; D( represents the mean distance between target region association pairs; l a) Indicates a related pair l The distance calculation function for a.

[0096] Next, select association pairs that satisfy variance consistency from the candidate region association pairs. Iterate through all association pairs, filter out association pairs whose association costs exceed the threshold, and construct a joint association result based on the maximum number of association pairs, as shown in the following expression:

[0097]

[0098]

[0099] in, This indicates that the supplementary association pair selected from the candidate region is generated under the condition that equation (12) is satisfied; n a represents the nth additional association pair; S asc (·) denotes a descending order function to achieve a descending order of the consistency variance of candidate region associations. Indicates based on Attribute consistency variance Sort the data; topK(·) represents the set of the top K associative pairs after sorting, where K = N. t ; This indicates the final, complete association result.

[0100] like Figure 2 As shown, this embodiment illustrates a simulation of a lane-changing and overtaking scenario and its corresponding attribute information. The scenario involves three lanes (lane width W is 3.5 meters), three targets (T1, T2, and T3) maintaining constant speed while traveling in a straight line, and one target (T4) changing lanes to overtake target T2 and returning to its original lane. In the attribute table, " / " indicates that the target's motion state does not change.

[0101] like Figure 3 As shown in the figure, the correlation index of the simulation scene is statistically analyzed in this embodiment. It can be seen that compared with the existing method (Hungarian algorithm), the method of the present invention greatly improves the correlation accuracy of negative samples, with the accuracy increasing by about 19%. At the same time, the correlation accuracy of positive samples also increases slightly. This indicates that the method of the present invention can effectively improve the correlation accuracy of radar and visual targets in the simulation scene and promote the subsequent fusion of radar and visual targets.

[0102] like Figure 4 As shown, the overall accuracy changes with the measurement loss probability in this embodiment. The corresponding dashed line represents the trend of the overall accuracy with the measurement loss probability. It can be seen that even when the measurement loss probability increases, the method of the present invention can still maintain a high overall association accuracy, while the association accuracy of existing methods decreases rapidly with the increase of the measurement loss probability. This indicates that the method of the present invention can overcome the problems of association jump caused by measurement loss to a large extent.

[0103] like Figure 5 As shown in this embodiment, a comparison is made between the simulation scene association results of the method of the present invention and the existing methods. Figure 5 (a), (b), (c), and (d) represent the correlation results of the method of the present invention in four consecutive frames, respectively. Figure 5 (e), (f), (g) and (h) represent the association results of the existing methods, respectively. It can be seen that the existing methods are prone to association jumps when measurements are lost, while the method of the present invention can achieve robust and continuous target association and reduce jump phenomena.

[0104] like Figure 6 As shown, in this embodiment, the extended correlation results of the method of the present invention and the existing method in the actual test scenario are compared. Figure 6 (a), (b), and (c) represent the extended association results of the method of the present invention in three consecutive frames, where the thickness of the association line indicates the association stability. Figure 6 (d) represents the corresponding reference image. It can be seen that the invention method can find all subsequent association results through the extended association process under multiple assumptions, reducing the problem of multiple possibilities of measurement loss that are difficult to take into account by existing methods. The association frequency can initially indicate the association stability, that is, the higher the association frequency, the more robust the association result. The consistency of the variance of the coupled target area can further improve the reliability of the association.

[0105] like Figure 7 As shown, this embodiment compares the final correlation results of the method of the present invention with those of existing methods in a real-world scenario. Figure 7 (a), (b), and (c) respectively represent Figure 6 The corresponding association results of the method of the present invention for three consecutive frames. Figure 7 (d), (e) and (f) represent the association results of the existing methods, respectively. It can be seen that the existing methods are also prone to association jump problems in the actual test scenario, while the method of the present invention can reduce the jump phenomenon and achieve robust and continuous target association.

[0106] In summary, the method of this invention fully considers the problem of measurement loss in radar and vision systems. It constructs an extended measurement set for both radar and vision, and designs a corresponding extended Hungarian association method. Adaptively performing extended Hungarian association based on measurement characteristics yields multi-hypothesis extended association results. Highly stable association pairs and consistency-considered association pairs are extracted from these results to form a complete and robust association outcome. The multi-hypothesis extension judgment significantly reduces association jumps caused by measurement loss during radar and vision target association, improving the accuracy of the association. This invention, by extending the measurements, considers various scenarios where measurement loss occurs, optimizing the problems of radar and vision target similarity disorder and association jumps caused by measurement loss. It significantly improves the reliability of radar and vision fusion perception in scenarios such as traffic target perception and detection, and is applicable to scenarios requiring radar and vision fusion, such as autonomous driving and traffic monitoring.

[0107] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A robust method for associating radar and visual targets despite the loss of surface-level measurement data, comprising the following steps: S1. Preprocess radar and visual data to obtain spatially aligned and time-synchronized radar and visual measurement sets; Install radar and vision sensors, and use Zhang Zhengyou's calibration method to calibrate the sensors to obtain vision intrinsic and extrinsic parameters and radar extrinsic parameter data; The system acquires raw images from the camera, uses a neural network to perform target detection on the visual images, and obtains visual measurements of pixel coordinates. Collect radar echo data, remove background clutter and zero-Doppler targets based on frequency domain information, and analyze and obtain radar measurements; The acquired intrinsic and extrinsic parameters are used to transform the radar and visual measurements into a unified coordinate system. Then, the measurements are synchronized to the same moment according to the sensor time to obtain a spatially aligned and time-synchronized set of radar and visual measurements. Then, the radar and visual measurement sets are filtered, and the filtered and saved radar and visual measurement points are correlated, as shown in the following expression: （1）； in, Represents radar measurement set, express The Middle One radar measurement point, Indicates the number of radar measurement points; Represents the visual measurement set. express The Middle One visual measurement point Indicates the number of visual measurement points; S2. Construct extended radar and visual measurement sets; Select from all elements of the radar measurement set Construct a measurement subset from each element, and take them sequentially. The subsets of radar measurements under the given conditions collectively constitute the extended radar measurement set; similarly, all subsets of visual measurements collectively constitute the extended visual measurement set, as expressed below: （2）； in, and These respectively indicate the missing number in the radar and visual measurement sets. A subset of measurements; S3. Perform Hungarian target association based on the extended measurement set obtained in step S2, and adaptively generate multi-hypothesis extended association results; Based on the extended radar and visual measurement sets, Hungarian target association is performed, and the association results of a single radar measurement subset and a visual measurement subset are obtained sequentially, as shown in the following expression: （3）； in, , indicating that it is based on radar measurement sets With visual measurement set The Hungarian association results, Indicates the first A radar-visual correlation pair Indicates the corresponding association pair, The corresponding radar measurement points in the associated pair are indicated. This indicates the corresponding visual measurement points in the associated pair. This represents the number of associated pairs, with a maximum value of 0. That is, the minimum number of elements in the radar and visual measurement sets; Represents the Hungarian association algorithm; and These represent the association results generated based on the corresponding measurement sets; Based on the difference in the number of measurement points between radar and visual measurement sets, multiple hypotheses for potential measurement loss are established. These hypotheses are then coupled with an expanded measurement set to adaptively generate extended association results between radar and vision under multiple hypotheses. The extended association results are a set of elements that can be repeated, expressed as follows: （4）； in, This represents the extended correlation results between radar and visual measurement sets for a specific number of samples. Indicating the first in the extended association results One association pair, This indicates the radar and visual measurement points of the corresponding correlation pairs. This indicates the number of all association pairs in the expanded association result, where each association pair contains duplicate elements; This represents the cumulative union operation; S4. Statistically analyze the results of multi-hypothesis extended association and divide the highly stable association target region and the sub-stable candidate region under the results of multi-hypothesis extended association. The multi-hypothesis extended association results of statistical radar and visual measurement sets are used to extract all unique association pairs in the extended association results and construct mutually exclusive association results, denoted as... The expression is as follows: （5）； in, Indicating the first element in the mutually exclusive association results Each pair of correlations corresponds to a radar and visual measurement. , This indicates the number of association pairs in the result of a mutually exclusive association; With mutually exclusive association results Based on this, statistics were compiled on its extended association results. The frequency of occurrence in is expressed as follows: （6）； in, Represents a frequency statistics function; Indicates related pairs exist Frequency in; express The frequency set of all elements in the set; Sort the frequency set of mutually exclusive association results to obtain association results in descending order of frequency, as shown in the following expression: （7）； in, , This represents mutually exclusive association results sorted in descending order of frequency. Indicates the frequency ranking in the order of the numbers. Bit-related pairs, Indicates the corresponding radar and visual measurement points; This represents a frequency descending order function. Indicates based on Attributes right Sort in descending order; The maximum number of correctly associated pairs is calculated based on the number of radar and visual measurement points, denoted as . Couple the maximum number of association pairs with the frequency-decreasing association results, and divide the mutually exclusive association results into highly stable target regions and sub-stable candidate regions under multiple hypotheses, as shown in the following expression: （8）； （9）； in, This represents the set of association pairs belonging to the target region; This represents the largest index that is divided into target regions in the mutually exclusive association results in descending frequency order, i.e. Center front Each associated pair was assigned to the target region, and Satisfying equation (9); Represents a frequency statistics function; This represents the set of association pairs belonging to the candidate region; S5. Based on the association frequency and variance consistency, select association results with high stability and strong consistency, and construct complete and robust association results; The variance of stable association pairs is calculated based on target region association pairs, and the expression is as follows: （10）； in, This represents the variance of the distance between target region association pairs; This represents the mean distance between target region association pairs; Indicates related pairs Distance calculation function; Next, select association pairs that satisfy variance consistency from the candidate region association pairs. Iterate through all association pairs, filter out association pairs whose association costs exceed the threshold, and construct a joint association result based on the maximum number of association pairs, as shown in the following expression: （11）； （12）； in, This indicates that the supplementary association pair selected from the candidate region is generated under the condition that equation (11) is satisfied. The supplementary number One association pair; This represents a descending order function, used to sort candidate regions in descending order of their consistency variance. Indicates based on Attribute consistency variance Sort; This indicates taking the first part after sorting. A set consisting of 10 association pairs ; This indicates the final, complete association result.

Images