Binocular vision-millimeter wave radar multi-element joint detection method

Abstract

The invention discloses a binocular vision-millimeter wave radar multi-element joint detection method, which comprises the following steps: constructing a vision monitoring component, carrying out target detection in a region by utilizing a learning network, and carrying out vision target tracking based on a multi-target tracking algorithm to obtain a vision target trajectory; the target echo signal is processed by a radar signal processing algorithm to generate a target plot, and based on a joint probability data association algorithm and a Kalman filtering algorithm, tracking and prediction of a radar target are realized to obtain a radar target trajectory; and according to the visual target trajectory and the radar target trajectory, establishing a multi-level constraint association algorithm, and constructing a feature-level deep fusion framework. According to the method, deep fusion of radar and binocular vision target data is realized, and the sensing coverage range and the information stability of a radar-binocular vision fusion system are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic information perception, and in particular to a binocular vision-millimeter wave radar multi-factor joint detection method. Background Art

[0002] In the field of traffic perception, in order to cope with the increasingly complex road environment and safety challenges, radar-vision fusion technology has gradually attracted attention. Traditional single sensor systems are easily limited in bad weather, light changes or complex scenes, affecting the accuracy and reliability of traffic information. Radar, with its strong penetration and anti-interference characteristics, can stably detect target distance and speed under various conditions, while visual sensors can capture rich environmental details and color information. The fusion of the two can not only significantly improve the accuracy of target recognition and tracking, but also provide real-time and comprehensive traffic data under all-weather conditions, supporting the efficient operation of the intelligent transportation system. Radar-vision fusion combines the complementary advantages of the two and has become a key technology to improve the safety of autonomous driving, optimize traffic management, and prevent accidents, achieving more intelligent and robust traffic environment perception.

[0003] However, existing radar-vision fusion perception solutions are mostly based on radar-monocular and radar-binocular vision fusion, and are matched and fused with target position and category information. However, there are problems such as unstable perception of targets in long-distance areas and large visual blind spots. Summary of the invention

[0004] The purpose of the present invention is to provide a binocular vision-millimeter wave radar multi-factor joint detection method. It aims to solve the problem that the existing radar-vision fusion perception scheme is mostly based on radar-monocular and radar-binocular vision fusion, and is matched and fused in combination with target position and category information, and has the problems of unstable perception of targets in long-distance areas and large visual blind spots.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a binocular vision-millimeter wave radar multi-factor joint detection method, comprising:

[0007] Step S1: construct a visual monitoring component, use a learning network to detect targets within the area, and perform visual target tracking based on a multi-target tracking algorithm to obtain visual target trajectories;

[0008] Step S2: For the radar sensor, the target echo signal is processed by the radar signal processing algorithm to generate a target point trace, and based on the joint probability data association algorithm and the Kalman filter algorithm, the radar target is tracked and predicted to obtain the radar target trajectory;

[0009] Step S3: According to the visual target trajectory, the radar target trajectory, and in combination with the traffic scene characteristics and sensor perception characteristics, a multi-level constraint association algorithm is established, and the target position, category, and speed information are used to filter out target multi-scattering points and false alarm points, and select the optimal target matching relationship;

[0010] Step S4: In the data fusion stage, a feature-level deep fusion framework is constructed according to the target optimal matching relationship.

[0011] Preferably, the visual monitoring component is constructed in step S1, which specifically includes:

[0012] The corresponding camera focal length is calculated according to the application scenario, and a long-focus monocular camera and a short-focus monocular camera are selected according to the camera focal length to construct the visual monitoring component with full vertical area coverage.

[0013] Preferably, in step S2, the target echo signal is processed by a radar signal processing algorithm to generate a target point trace, which specifically includes: in the target detection and data extraction stage, the target echo signal is received after the signal emitted by the radar illuminates the target, and the target echo signal is processed by a radar signal processing algorithm to obtain the target information and generate the target point trace.

[0014] Preferably, the radar signal processing algorithm includes: any one of pulse compression, moving target detection and Doppler processing or any combination thereof.

[0015] Preferably, in step S2, the tracking and prediction of the radar target is achieved based on the joint probability data association algorithm and the Kalman filter algorithm to obtain the radar target trajectory, which specifically includes: matching the measurement points of the same target in multiple consecutive frames based on the joint probability data association algorithm and the Kalman filter to obtain the radar target trajectory.

[0016] Preferably, in step S3, a multi-level constraint association algorithm is established according to the visual target trajectory, the radar target trajectory, and in combination with traffic scene characteristics and sensor perception characteristics, which specifically includes:

[0017] Step S3.1: performing temporal interpolation and spatial synchronization on the visual target trajectory and the radar target trajectory, converting the visual target trajectory and the radar target trajectory to the same spatiotemporal coordinate system, interpolating the target distribution after spatiotemporal synchronization using several spline interpolation methods, and calibrating using three-dimensional coordinate conversion to obtain a fixed relationship between the target positions of the two sensors;

[0018] Step S3.2: Select an associated time window, receive the radar target trajectory i and the visual target trajectory j at time k, and the trajectory states of the radar target trajectory i and the visual target trajectory j are respectively represented as s(A,i) 、s (B,j) , the current historical fusion trajectory F k , ε1~ε5 are the matching weights of the corresponding target features, and the trajectory overlap time T is obtained according to the radar target trajectory i and the visual target trajectory j. cross , target spatial position Target pixel position Similarity, get appearance similarity based on historical trajectory and visual trajectory Get motion similarity based on historical fusion trajectory and radar target trajectory Get the target matching degree φ ij The formula is:

[0019]

[0020] The maximum target matching degree threshold is set, and the matching relationship whose target matching degree is greater than the maximum threshold is determined to be invalid, and the target matching matrix is ​​obtained.

[0021] Preferably, in step S3, the information of the target position, category and speed is used to filter out the target multi-scattering points and false alarm points, and screen to obtain the target optimal matching relationship, which specifically includes:

[0022] Step S3.3: Use the bipartite graph maximum weight matching algorithm to screen the target matching matrix, with the vertices being the radar and visual detection targets and the edges being the target matching degrees, to achieve minimum weight matching, determine the preservation and elimination of many-to-many matching relationships, and release the matching pairs of confirmed matching trajectories and other trajectories to ensure the uniqueness of the matching relationship and obtain the global optimal matching pair.

[0023] Preferably, in step S4, a feature-level deep fusion framework is constructed according to the target optimal matching relationship, which specifically includes:

[0024] Step S4.1: Information fusion is performed on the sensor targets that are matched with each other according to the target matching pair ψ. Based on the detection capability of each sensor, the fusion coefficient G is introduced as a weighted weight, which is represented by the positioning accuracy of the sensor at each point in the working area. The expression of the fusion coefficient G is:

[0025]

[0026] In the formula, σ x , σ y Represents the positioning error variance in the x and y directions respectively, and calculates the single sensor G value with the measured value X m Take the multi-sensor fusion result as an example:

[0027]

[0028] In the formula, Gm is the positioning accuracy value of each sensor, X m is the measured value, is the fusion value, M is the number of sensors;

[0029] Step 4.2: After weighted fusion, perform target-level fusion on the single sensor information corresponding to the system trajectory;

[0030] Step 4.3: Dynamically adjust the weights according to the sensor environment adaptability.

[0031] Preferably, in step S4.2, the fusion content of the target-level fusion includes: a semantic description of the target output by the video structure and information output by the radar detection.

[0032] Preferably, the target semantic description of the structured video output specifically includes: any one of the type, color, and model style of the sensor or any combination thereof.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The binocular vision-millimeter wave radar multi-factor joint detection method provided by the present invention constructs a longitudinal long- and short-focus binocular vision-radar fusion perception solution to achieve longitudinal full-area coverage visual perception and solve the problem of large target perception blind spots;

[0035] (2) Design a multi-level constraint association algorithm that integrates target motion characteristics and detection characteristics to obtain the optimal association matching relationship, solving the problem of confusing matching relationships in dense traffic scenes;

[0036] (3) Construct a deep feature-level fusion framework and introduce fusion weights based on detection confidence, so that the target information is more complete and the long-distance motion trajectory and status of the vehicle are more reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for the description. Obviously, the drawings described below are an embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work:

[0038] Figure 1 A schematic flow chart of a binocular vision-millimeter wave radar multi-factor joint detection method provided by one embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the spatiotemporal synchronization results of radar and visual targets provided by an embodiment of the present invention;

[0040] Figure 3A schematic diagram of a radar-visual target matching relationship provided by an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of radar-visual target fusion processing output provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following is combined with Figure 1-4 The binocular vision-millimeter wave radar multi-factor joint detection method proposed in the present invention is further described in detail with specific implementation methods. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the implementation methods of the present invention. In order to make the purposes, features and advantages of the present invention more obvious and easy to understand, please refer to the drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention, so they have no technical substantive significance. Any structural modification, change in proportional relationship or adjustment of size, without affecting the effects that can be produced by the present invention and the purposes that can be achieved, should still fall within the scope of the technical content disclosed by the present invention.

[0043] The method provided by the present invention achieves full coverage of the area through a combination of monocular cameras with different focal lengths, and matches and fuses them with millimeter-wave radars. A multi-level constraint association trajectory fusion algorithm is designed in combination with traffic scene characteristics to achieve high-precision target perception, effectively solve the problems of unstable target perception in long-distance areas and large visual blind spots, and has stronger scene adaptability, thereby improving the stability and reliability of the fusion perception system.

[0044] refer to Figure 1 As shown, the present invention provides a binocular vision-millimeter wave radar multi-factor joint detection method, comprising:

[0045] Step S1: construct a visual monitoring component, use a learning network to detect targets in the area, and perform visual target tracking based on a multi-target tracking algorithm to obtain visual target trajectories, which specifically include:

[0046] The corresponding camera focal length is calculated according to the application scenario, and a long-focus monocular camera and a short-focus monocular camera are selected according to the camera focal length to construct the visual monitoring component with full vertical area coverage.

[0047] refer to Figure 1As shown in the figure, the visual monitoring component is composed of a long-focus monocular camera and a short-focus monocular camera. In the 500m monitoring section, long-focus and short-focus monocular cameras are selected to form a longitudinal full-area coverage visual monitoring, and the YOLOv5 learning network is used to detect targets in the area, and visual target tracking is performed based on DeepSort to obtain the visual target trajectory.

[0048] Step 1-1: For 25-500m road monitoring scenarios, set the equipment installation height H dev , select a 12mm short-focus camera with a horizontal field of view angle β, a 50mm long-focus camera with a horizontal field of view angle δ, and set the minimum distance for a single camera, that is,

[0049]

[0050] The short-focus camera pitch angle α can be obtained, and the camera's maximum coverage distance can be calculated as:

[0051]

[0052] According to the requirement that the telephoto camera and the short-focus camera partially overlap, the minimum telephoto distance d1′ is set, and the maximum coverage distance d2′ is calculated accordingly, and finally the longitudinal joint coverage range is d1-d2′, and the overlap area is d1′-d2.

[0053] Step 1.2: Use the YOLOv5 deep learning network to input long and short focal length visual images. Through deep learning and video structuring technology, effectively extract multi-dimensional feature information such as target color, type, bounding box, license plate, etc., and perform visual target tracking based on DeepSort to achieve target recognition and assign a specified ID, thereby generating a visual target: trajectory.

[0054] Step S2: For radar sensors, target echo signals are processed by radar signal processing algorithms to generate target points, and radar target tracking and prediction are achieved based on joint probability data association algorithms and Kalman filter algorithms to obtain radar target trajectories. For radar sensors, target echo signals are processed by radar signal processing algorithms to generate target points, and radar target tracking and prediction are achieved based on joint probability data association algorithms (JPDA) and Kalman filter algorithms to obtain radar target trajectories. m means that m targets are currently detected. Specifically, they include:

[0055] Step S2.1: In the target detection and data extraction stage, the radar transmits a signal to illuminate the target and then receives the target echo signal. The target echo signal is processed by a radar signal processing algorithm to obtain target information and generate the target point trace. The radar signal processing algorithm includes: any one or any combination of pulse compression, moving target detection and Doppler processing. Useful information such as the distance, speed, angle, etc. of the target is separated from the received complex signal. This process will filter out clutter and interference to ensure that the echo characteristics of the real target are extracted.

[0056] Step S2.2: The radar target trajectory is obtained by matching the measurement points of the same target in multiple consecutive frames based on the joint probability data association algorithm and Kalman filtering. At the same time, the situation of temporary occlusion of the target or loss of measurement is handled to ensure that the generated trajectory is smooth and coherent, accurately reflecting the actual movement of the target.

[0057] Step S3: According to the visual target trajectory, the radar target trajectory, and in combination with the traffic scene characteristics and sensor perception characteristics, a multi-level constraint association algorithm is established, and the target position, category, and speed information are used to filter out target multi-scattering points and false alarm points, and screen out the optimal target matching relationship.

[0058] Step S3.1: performing time interpolation and spatial synchronization on the visual target trajectory and the radar target trajectory, and converting the visual target trajectory and the radar target trajectory to the same time-space coordinate system, Figure 2 The target distribution after time-space synchronization. In the case where the output of each sensor is not absolutely uniform, the cubic spline interpolation method is used for interpolation. , and the fixed relationship between the target positions of the two sensors is obtained by 3D coordinate conversion calibration, realizing the two-way spatial synchronization of radar→vision and vision→radar.

[0059] Step S3.2: Select an associated time window, receive the radar target trajectory i and the visual target trajectory j at time k, and the trajectory states of the radar target trajectory i and the visual target trajectory j are respectively represented as s (A,i) 、s (B,j) , the current historical fusion trajectory F k , ε1~ε5 are the matching weights of the corresponding target features, and the trajectory overlap time T is obtained according to the radar target trajectory i and the visual target trajectory j. cross , target spatial position Target pixel position Similarity, get appearance similarity based on historical trajectory and visual trajectory Get motion similarity based on historical fusion trajectory and radar target trajectory Get the target matching degree φ ij The formula is:

[0060]

[0061] The maximum target matching degree threshold is set, and the matching relationship whose target matching degree is greater than the maximum threshold is determined to be invalid, and the target matching matrix is ​​obtained.

[0062] Step S3.3: Use the bipartite graph maximum weight matching algorithm to screen the target matching matrix, with the vertices being radar and visual detection targets and the edges being target matching degrees, to achieve minimum weight matching, prioritize confirming associations with smaller target matching degrees, determine the retention and elimination of many-to-many matching relationships, and remove the matching pairs of confirmed matching trajectories and other trajectories to ensure the uniqueness of the matching relationship and obtain the global optimal matching pair.

[0063] Step S4: In the data fusion stage, a feature-level deep fusion framework is constructed according to the target optimal matching relationship.

[0064] According to the target optimal matching relationship, a feature-level deep fusion framework is constructed, which specifically includes:

[0065] Step S4.1: Information fusion is performed on the sensor targets that are matched with each other according to the target matching pair ψ. Based on the detection capability of each sensor, the fusion coefficient G is introduced as a weighted weight, which is represented by the positioning accuracy of the sensor at each point in the working area. The expression of the fusion coefficient G is:

[0066]

[0067] In the formula, σ x , σ y Represents the positioning error variance in the x and y directions respectively, and calculates the single sensor G value with the measured value X m Take the multi-sensor fusion result as an example:

[0068]

[0069] In the formula, G m is the positioning accuracy value of each sensor, X m is the measured value, is the fusion value, and M is the number of sensors.

[0070] Step 4.2: After weighted fusion, target-level fusion is performed on the single sensor information corresponding to the system trajectory. Finally, complete and accurate target trajectory information is obtained. The fusion content of the target-level fusion includes: the semantic description of the target output by the video structure, the information output by the radar detection (such as precise speed, RCS, lane, etc.), where RCS stands for RadarCross Section. The semantic description of the target output by the video structure specifically includes: any one or any combination of the type, color, and model style of the sensor.

[0071] Step 4.3: Dynamically adjust the weights according to the adaptability of the sensor environment. For example, in good lighting conditions, the credibility of visual information is improved, and the weight of visual perception information is increased. In low light or bad weather, the millimeter wave radar performs more stably, and the weight of visual perception information is reduced to maintain the stability and accuracy of the fusion system.

[0072] The final radar-visual target matching relationship and fusion processing output are as follows: Figure 3 , Figure 4 As shown in the figure, the perception coverage range reaches 500m, the trajectory tracking accuracy reaches 97.3%, the target false alarm rate and tracking loss rate are greatly reduced, the tracking accuracy and stability of the fusion system for the target are greatly improved, and the sensing system's perception ability of the environment is greatly improved.

[0073] In summary, the binocular vision-millimeter wave radar multi-factor joint detection method provided in this embodiment constructs a longitudinal long-short focal length binocular vision-radar fusion perception scheme, realizes longitudinal full-area coverage visual perception, and solves the problem of large target perception blind spots; designs a multi-level constraint association algorithm, integrates target motion characteristics and detection characteristics, obtains the optimal association matching relationship, and solves the problem of chaotic matching relationships in dense traffic scenes; constructs a feature-level deep fusion framework, introduces fusion weights based on detection confidence, and makes target information more complete, and the long-distance motion trajectory and state of the vehicle more reasonable. The method realizes deep fusion of radar and binocular vision target data, enhances the perception coverage and information stability of the radar-binocular vision fusion system, and has multiple advantages.

[0074] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0075] It should be noted that the devices and methods disclosed in the embodiments of this article can also be implemented in other ways. The device implementation described above is only schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of this article. In this regard, each box in the flowchart or block diagram can represent a part of a module, program or code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system for performing a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0076] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A binocular vision-millimeter wave radar multi-factor joint detection method, characterized in that: include: Step S1: construct a visual monitoring component, use a learning network to detect targets within the area, and perform visual target tracking based on a multi-target tracking algorithm to obtain visual target trajectories; Step S2: For the radar sensor, the target echo signal is processed by the radar signal processing algorithm to generate a target point trace, and based on the joint probability data association algorithm and the Kalman filter algorithm, the radar target is tracked and predicted to obtain the radar target trajectory; Step S3: According to the visual target trajectory, the radar target trajectory, and in combination with the traffic scene characteristics and sensor perception characteristics, a multi-level constraint association algorithm is established, and the target position, category, and speed information are used to filter out target multi-scattering points and false alarm points, and select the optimal target matching relationship; Step S4: In the data fusion stage, a feature-level deep fusion framework is constructed according to the target optimal matching relationship.

2. The binocular vision-millimeter wave radar multi-factor joint detection method according to claim 1, characterized in that: The visual monitoring component is constructed in step S1, which specifically includes: The corresponding camera focal length is calculated according to the application scenario, and a long-focus monocular camera and a short-focus monocular camera are selected according to the camera focal length to construct the visual monitoring component with full vertical area coverage.

3. The binocular vision-millimeter wave radar multi-factor joint detection method according to claim 1, characterized in that: In the step S2, the target echo signal is processed by a radar signal processing algorithm to generate a target point trace, which specifically includes: in the target detection and data extraction stage, the target echo signal is received after the signal emitted by the radar illuminates the target, and the target echo signal is processed by a radar signal processing algorithm to obtain the target information and generate the target point trace.

4. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 3, characterized in that: The radar signal processing algorithm includes: any one of pulse compression, moving target detection and Doppler processing or any combination thereof.

5. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 3, characterized in that: In the step S2, the tracking and prediction of the radar target is realized based on the joint probability data association algorithm and the Kalman filter algorithm to obtain the radar target trajectory, which specifically includes: matching the measurement points of the same target in multiple consecutive frames based on the joint probability data association algorithm and the Kalman filter to obtain the radar target trajectory.

6. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 5, characterized in that: In step S3, a multi-level constraint association algorithm is established based on the visual target trajectory, the radar target trajectory, and in combination with traffic scene characteristics and sensor perception characteristics, which specifically includes: Step S3.1: performing temporal interpolation and spatial synchronization on the visual target trajectory and the radar target trajectory, converting the visual target trajectory and the radar target trajectory to the same spatiotemporal coordinate system, interpolating the target distribution after spatiotemporal synchronization using a spline interpolation method, and calibrating using a three-dimensional coordinate conversion to obtain a fixed relationship between the target positions of the two sensors; Step S3.2: Select an associated time window, receive the radar target trajectory i and the visual target trajectory j at time k, and the trajectory states of the radar target trajectory i and the visual target trajectory j are respectively represented as s (A,i) 、s (B,j) , the current historical fusion trajectory F k , ε1~ε5 are the matching weights of the corresponding target features, and the trajectory overlap time T is obtained according to the radar target trajectory i and the visual target trajectory j. cross , target space position Target pixel position Similarity, get appearance similarity based on historical trajectory and visual trajectory Get motion similarity based on historical fusion trajectory and radar target trajectory Get the target matching degree φ ij The formula is: The maximum target matching degree threshold is set, and the matching relationship whose target matching degree is greater than the maximum threshold is determined to be invalid, and the target matching matrix is ​​obtained.

7. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 6, characterized in that: In step S3, the information of the target position, type and speed is used to filter out the target multi-scattering points and false alarm points, and the optimal matching relationship of the target is obtained by screening, which specifically includes: Step S3.3: Use the bipartite graph maximum weight matching algorithm to screen the target matching matrix, with the vertices being the radar and visual detection targets and the edges being the target matching degrees, to achieve minimum weight matching, determine the preservation and elimination of many-to-many matching relationships, and release the matching pairs of confirmed matching trajectories and other trajectories to ensure the uniqueness of the matching relationship and obtain the global optimal matching pair.

8. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 7, characterized in that: In step S4, a feature-level deep fusion framework is constructed according to the target optimal matching relationship, which specifically includes: Step S4.1: Information fusion is performed on the sensor targets that are matched with each other according to the target matching pair ψ. Based on the detection capability of each sensor, the fusion coefficient G is introduced as a weighted weight, which is represented by the positioning accuracy of the sensor at each point in the working area. The expression of the fusion coefficient G is: In the formula, σ x , σ y Represents the positioning error variance in the x and y directions respectively, and calculates the single sensor G value with the measured value X m Take the multi-sensor fusion result as an example: In the formula, G m is the positioning accuracy value of each sensor, X m is the measured value, is the fusion value, M is the number of sensors; Step 4.2: After weighted fusion, perform target-level fusion on the single sensor information corresponding to the system trajectory; Step 4.3: Dynamically adjust the weights according to the sensor environment adaptability.

9. The binocular vision-millimeter wave radar multi-factor joint detection method as claimed in claim 8, characterized in that: In the step S4.2, the fusion content of the target level fusion includes: the semantic description of the target output by the video structure and the information output by the radar detection.

10. The multi-wave position test method of the near-field pattern of a phased array antenna according to claim 9, characterized in that: The target semantic description of the structured video output specifically includes: any one or any combination of the type, color, and model style of the sensor.

Images