Microwave perception sensing point visual mapping method and system based on visual fusion

Abstract

The application provides a microwave perception point visualization mapping method and system based on visual fusion, comprising: rigidly connecting a microwave perception device and a visual device; collecting echo of a target area and obtaining coordinate information of a corresponding measurement point in a range-azimuth two-dimensional spectrum diagram; solving height difference, pitch and / or angle information between the microwave perception device and the target measurement point, and expanding the measurement point coordinate in the range-azimuth two-dimensional spectrum diagram from two dimensions to three-dimensional space coordinates; transforming the radar and camera coordinate systems based on a specific reference coordinate system; mapping the three-dimensional space coordinates of the measurement point to the camera coordinate system and projecting them into an image obtained by the camera device based on the height difference, angle and geometric parameters between the microwave device and the measurement point obtained by pre-solution, and the converted coordinate system, so as to realize visualization mapping display of the microwave perception device measurement point selection result in the image.

Description

Technical Field

[0001] This invention relates to the field of sensor fusion technology, and more specifically, to a method and system for visual mapping of microwave sensing measurement points based on visual fusion. Background Technology

[0002] In the process of using microwave sensing technology for structural deformation and vibration monitoring, to obtain stable and reliable measurement results, it is usually necessary to select appropriate target measurement points and determine the spatial geometric relationship between the microwave device and the target measurement points before monitoring. However, in actual measurement, the selection of target measurement points often presents a high technical hurdle. On the one hand, the measurement output of microwave sensing technology usually only presents a two-dimensional spectrum containing only the line-of-sight distance and horizontal azimuth angle, while the vertical dimension information in three-dimensional space is compressed, making it difficult to intuitively map to the specific location in the real scene. On the other hand, due to the influence of factors such as installation height, pitch angle, and the geometric position of the measured target, it is difficult for non-professionals to quickly determine whether the signal points in the two-dimensional spectrum accurately correspond to the actual target measurement points. In actual operation, this not only requires extremely inefficient repeated comparison and trial and error but also affects the stability of subsequent monitoring data.

[0003] Currently, apart from manual point selection based on two-dimensional maps, there are no other methods. Common methods such as deploying reflectors or using external tools only enhance the signal quality or intensity of signal points in the two-dimensional map; confirming the correspondence between signal points and actual target measurement points still relies on manual judgment. The main reason for these drawbacks is that the technical principle of microwave radar sensing devices means they lack the ability to sense the elevation dimension or have very poor elevation angle resolution. This fundamentally limits the accurate positioning of target measurement points in the two-dimensional map and makes the point selection process largely dependent on experience and trial and error in the field.

[0004] To overcome the aforementioned problems, the field of autonomous driving typically adopts a technical approach that fuses microwave sensing and visual information. High-resolution pitch information provided by visual cameras compensates for the limitations of microwave sensing technology in pitch dimension perception. However, directly applying this approach to the selection of measurement points in microwave sensing devices has several limitations. First, the fusion algorithms in autonomous driving scenarios are mostly optimized for dynamic target detection and tracking, which differs significantly from the static feature scale used in selecting measurement points, making the methods unsuitable for direct transfer. Second, there are significant differences between visual and microwave data in terms of coordinate systems and resolution, making the precise conversion and fusion process complex. Furthermore, while the introduction of visual information can provide spatial reference to some extent, it remains difficult to achieve a high-precision correspondence between two-dimensional spectral signals and actual measurement points in the absence of accurate depth information or 3D reconstruction capabilities.

[0005] Therefore, to address the above problems, this invention establishes a reference coordinate system to map the coordinates of microwave sensing signal points into the visual image dimension, so as to achieve accurate mapping between signal points and real target measurement points in the two-dimensional map. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a visualization mapping method and system for microwave sensing measurement points based on visual fusion.

[0007] The microwave sensing measurement point visualization mapping method based on visual fusion provided by the present invention includes:

[0008] Step 1: Rigidly connect the microwave sensing device and the vision device to keep their relative pose constant; Step 2: Collect the echo of the area where the target measurement point is located using a microwave sensing device and detect the corresponding measurement point information in the range-azimuth two-dimensional spectrum; collect the visual image of the echo area using a vision device; Step 3: Using a fixed reference coordinate system as the reference coordinate system, transform the microwave sensing coordinate system and the visual coordinate system to establish the first transformation relationship from the microwave sensing coordinate system to the reference coordinate system and the second transformation relationship from the visual coordinate system to the reference coordinate system. Step 4: Obtain the height difference information between the microwave sensing device and the target measurement point by measurement, and expand the signal points in the range-azimuth two-dimensional spectrum into three-dimensional space accordingly; Step 5: Map and project the three-dimensional spatial coordinates of the target measurement point onto the visual image acquired by the vision device to realize the visualization and mapping display of the echo signal point of the microwave sensing device in the visual image.

[0009] Preferably, step 2 includes: determining one or more target points to be measured based on the stress characteristics, deformation sensitivity, or monitoring task requirements of the structure, and setting up reflectors at the target points or selecting natural reflection points with preset stable scattering characteristics as measurement points.

[0010] Preferably, step 3 includes: establishing a first transformation relationship from the microwave sensing coordinate system to the reference coordinate system based on the installation position and attitude parameters of the microwave sensing device. ;in, These are the coordinates of the target measurement point in the reference coordinate system. These are the coordinates of the target measurement point in the microwave sensing coordinate system. and This refers to the attitude and position of the microwave sensing device in the reference coordinate system. The second transformation relationship from the visual coordinate system to the reference coordinate system is established as follows: ;in These are the coordinates of the target measurement point in the visual coordinate system. and This refers to the orientation and position of the vision device in the reference coordinate system. Based on the first and second transformation relationships, the transformation relationship from the microwave sensing coordinate system to the vision coordinate system is obtained as follows: ;in It is the rotation matrix from the microwave sensing coordinate system to the visual coordinate system. It is the translation vector from the microwave sensing coordinate system to the visual coordinate system.

[0011] Preferably, step 4 includes: the microwave sensing device using an auxiliary device to move at a preset vertical displacement. Under the condition of acquiring the absolute distance between the microwave sensing device and the target measuring point before and after displacement, and or absolute distance before displacement and the difference in relative distance before and after displacement Based on the horizontal angle of the measured target in the two-dimensional range-azimuth spectrum. Establish a three-dimensional spatial coordinate system for the microwave sensing device and calculate the three-dimensional coordinate information of the target to be measured:

[0012]

[0013]

[0014] In this coordinate system, the center of the microwave sensing device is the origin. The Y-axis passes through the origin and is horizontally tangent to the transmitting plane of the microwave sensing device, with the left side of the microwave sensing device as the positive direction. The X-axis passes through the origin and is perpendicular to the transmitting plane of the microwave sensing device, with the line of sight of the microwave sensing device as the positive direction. The Z-axis passes through the origin and is perpendicular to the XY plane, with upward as the positive direction. The vertical angle between the X-axis and the target measuring point is calculated based on the three-dimensional coordinate information. The horizontal height difference between the microwave sensing device and the target measuring point This allows the two-dimensional distance-azimuth measurement points to be expanded and restored to three-dimensional spatial coordinates in the coordinate system of the microwave sensing device.

[0015] Preferably, step 5 includes: calibrating the intrinsic parameters of the vision device to obtain focal length, principal point, and distortion parameters; transforming the three-dimensional coordinates of the target measurement point in the microwave sensing coordinate system to the vision coordinate system, calculating the pixel coordinates of the measurement point based on the pinhole imaging model and performing distortion correction, and then superimposing them onto the world image acquired by the vision device to form a visual annotation of the target measurement point in the image. The distortion correction includes: denoting the calculated distortion-free pixel coordinates as... The pixel coordinates after distortion correction are Define the normalized radius as Then the distortion correction expression is:

[0016]

[0017] in, , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

[0018] Preferably, step 5 includes: mapping and projecting the signal points in the two-dimensional map of the microwave sensing device onto a visual image, and displaying the corresponding distance, azimuth, height difference and measurement point number information.

[0019] The microwave sensing measurement point visualization mapping system based on visual fusion provided by the present invention includes: The mounting bracket module is used to support the installation and fixation of the microwave sensing device and the vision device, so that their relative pose remains constant. The output end of the mounting bracket module is connected to the input end of the microwave sensing device and the vision device. A calibration module is used to assist the microwave sensing device in obtaining the height difference and angle information between the microwave sensing device and the target measuring point. The output terminal of the calibration module is connected to the input terminal of the microwave measuring instrument module. The coordinate system transformation module is used to establish the microwave sensing coordinate system, the vision device coordinate system, and the reference coordinate system, and to calculate and store the coordinate transformation relationship with the reference coordinate system as a reference. The input end of the coordinate system transformation module is connected to the output end of the mounting bracket module, and the output end is connected to the input end of the measurement point mapping and image projection module. The microwave measuring instrument module includes one and / or more microwave sensing devices for acquiring target area echoes and displaying target measurement point information in a range-azimuth two-dimensional spectral diagram. The input terminal of the microwave measuring instrument module is connected to the output terminal of the calibration module, and the output terminal is connected to the input terminal of the measurement point mapping and image projection module. The measurement point mapping and image projection module is used to map and project the spatial position of the measurement point onto the visual image acquired by the vision device based on the coordinate transformation relationship and geometric parameters, so as to realize the visualization and superposition display of the signal points in the two-dimensional map of the microwave sensing device in the visual image. The input end of the measurement point mapping and image projection module is connected to the output end of the coordinate system transformation module and the microwave measuring instrument module respectively, and the output end is connected to the input end of the information recording module. The information recording module is used to record and store the measurement point number, pixel coordinates, spatial coordinates of the measurement point, calibration parameters and measurement configuration parameters, and output the measurement point configuration results and operation records.

[0020] Preferably, the mounting bracket module is further used to support the installation and fixation of one and / or microwave sensing device and one and / or vision device on the same platform at the monitoring site, and has a structural design to prevent loosening and vibration to ensure that the relative posture remains constant over a long period of time; the mounting bracket module is also used to adjust the beam orientation or installation pitch angle of the microwave sensing device so that the detection range of the microwave sensing device can cover the area where the target measurement point is located, while ensuring that the field of view of the vision device includes the target measurement point area.

[0021] Preferably, the calibration module further includes a displacement slide with a fixed stroke, used to assist the microwave sensing device in obtaining three-dimensional information of the height difference and vertical angle of the measured target; Preferably, the coordinate system transformation module is used to perform intrinsic parameter calibration on the vision device to obtain focal length, principal point and distortion parameters, and to perform distortion correction on the projection results before superimposing and displaying them; Preferably, the measurement point mapping and image projection module is used to transform the three-dimensional coordinates of the measurement point in the microwave sensing reference coordinate system to the visual coordinate system, calculate the pixel coordinates of the measurement point based on the imaging model, and superimpose the measurement point projection position, corresponding distance and azimuth information and measurement point number into the image for output. Preferably, the information recording module is used to write the target or target point number, the pixel coordinates of the measuring point, the spatial coordinates of the measuring point, the coordinate transformation parameters, the geometric calibration parameters, and the configuration parameters of the microwave sensing device into a database or local file, and output a configuration list, verification screenshots, or reports.

[0022] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention provides a method and system for visual selection and mapping of microwave sensing measurement points based on visual fusion, which does not require repeated testing and experience judgment by the operator, reduces the threshold for use, and facilitates rapid deployment and long-term stable operation in engineering sites; (2) By establishing the transformation relationship between the microwave sensing coordinate system, the visual coordinate system and the reference coordinate system, and combining the height difference and horizontal angle parameter information between the microwave sensing device and the target measurement point, the present invention realizes the reliable solution of the three-dimensional spatial coordinates of the signal point in the two-dimensional map and the image projection mapping, so that the signal point of the microwave sensing device and the real target measurement point form a one-to-one visual confirmation path, thereby improving the accuracy, repeatability and traceability of the selection point. (3) Under complex engineering site conditions, this invention combines the positioning of the distance-azimuth two-dimensional spectrum with the visualization constraints of the visual image side to achieve accurate differentiation and positioning between multiple target measurement points, reducing misselection, wrong selection and point drift. Attached Figure Description

[0023] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the microwave sensing measurement point visualization mapping method based on visual fusion according to the present invention. Figure 2 This is a schematic diagram of microwave sensing, vision, and reference coordinate system in the coordinate transformation step of this invention; Figure 3 This is a block diagram of the visualization mapping system for microwave sensing measurement points based on visual fusion, as described in this invention. Detailed Implementation

[0024] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0025] Example 1 Figure 1 This is a flowchart of one embodiment of the present invention. Figure 1 As shown, this invention provides a visualization mapping method for microwave sensing measurement points based on visual fusion, comprising the following steps: Step 1: Rigidly connect the microwave sensing device and the vision device to keep their relative pose constant; Step 2: Collect the echo of the area where the target measurement point is located using a microwave sensing device and detect the corresponding measurement point information in the range-azimuth two-dimensional spectrum; collect the visual image of the echo area using a vision device; Step 3: Using a fixed reference coordinate system as the reference coordinate system, transform the microwave sensing coordinate system and the visual coordinate system to establish the first transformation relationship from the microwave sensing coordinate system to the reference coordinate system and the second transformation relationship from the visual coordinate system to the reference coordinate system. Step 4: Obtain the height difference information between the microwave sensing device and the target measurement point by measurement, and expand the signal points in the range-azimuth two-dimensional spectrum into three-dimensional space accordingly; Step 5: Map and project the three-dimensional spatial coordinates of the target measurement point onto the visual image acquired by the vision device to realize the visualization and mapping display of the echo signal point of the microwave sensing device in the visual image.

[0026] In some embodiments, the microwave sensing device may be a millimeter-wave radar, and the vision device may be a monocular camera or a binocular camera; both are fixedly installed on the side or below the structure at a position that meets the field of view and measurement requirements by means of rigid connectors or integrated brackets, so as to ensure that their relative pose remains constant during operation.

[0027] In some embodiments, optionally, one or more target points to be measured are determined based on the stress characteristics, deformation sensitivity, or monitoring task requirements of the structure, and reflectors are placed at the target measurement points or natural reflection points with stable scattering characteristics are selected as measurement points to improve the stability and repeatability of microwave sensing measurements. The target measurement points can be set at locations where the structure is subjected to large stress, has obvious deformation response, or is representative of the monitoring results. For example, when the object under test is a beam structure or frame structure, the mid-span location, node connection, or stress concentration area can be determined as the target measurement points; when the monitoring task is deflection monitoring or settlement monitoring, locations with large theoretical deflection or locations sensitive to differential settlement can be determined as the target measurement points.

[0028] In some embodiments, optionally, a microwave sensing coordinate system, a visual coordinate system, and a reference coordinate system are established respectively, with the reference coordinate system serving as a unified reference coordinate system; by measuring the installation position and attitude parameters, the first coordinate transformation relationship from the microwave sensing coordinate system to the reference coordinate system is obtained. ,in These are the coordinates of the target measurement point in the reference coordinate system. These are the coordinates of the target measurement point in the microwave sensing coordinate system. and It refers to the attitude and position of the microwave sensing device in the reference coordinate system, and the second coordinate transformation relationship from the visual coordinate system to the reference coordinate system. ,in These are the coordinates of the target measurement point in the visual coordinate system. and This refers to the camera's attitude and position in the reference coordinate system, thereby realizing the positional representation of the three-dimensional measurement points of microwave sensing in the reference coordinate system, and further transforming them to the visual coordinate system. ,in It is the rotation matrix from the microwave sensing coordinate system to the visual coordinate system. It is a translation vector from the microwave sensing coordinate system to the visual coordinate system, used for projection display.

[0029] In some embodiments, optionally, the microwave sensing device utilizes an auxiliary device to achieve a preset vertical displacement. Under the condition of acquiring the absolute distance between the microwave sensing device and the target measuring point before and after displacement, and or absolute distance before displacement and the difference in relative distance before and after displacement Based on the horizontal angle of the measured target in the two-dimensional range-azimuth spectrum. Establish a three-dimensional spatial coordinate system for the microwave sensing device and calculate the three-dimensional coordinate information of the target to be measured:

[0030]

[0031]

[0032] In this coordinate system, the center of the microwave sensing device is the origin. The Y-axis passes through the origin and is horizontally tangent to the transmitting plane of the microwave sensing device, with the left side of the microwave sensing device as the positive direction. The X-axis passes through the origin and is perpendicular to the transmitting plane of the microwave sensing device, with the line of sight of the microwave sensing device as the positive direction. The Z-axis passes through the origin and is perpendicular to the XY plane, with upward as the positive direction. The vertical angle between the X-axis and the target measuring point is calculated based on the three-dimensional coordinate information. The horizontal height difference between the microwave sensing device and the target measuring point This allows the two-dimensional distance-azimuth measurement points to be expanded and restored to three-dimensional spatial coordinates in the coordinate system of the microwave sensing device.

[0033] In some embodiments, optionally, the vision device undergoes intrinsic parameter calibration to obtain focal length, principal point, and distortion parameters; after transforming the three-dimensional coordinates of the target measurement point in the microwave sensing coordinate system to the vision coordinate system, the pixel coordinates of the measurement point are calculated based on the pinhole imaging model and distortion correction is performed. These coordinates are then superimposed onto the world image acquired by the vision device to form a visual annotation of the target measurement point in the image. The distortion correction includes: denoting the calculated distortion-free pixel coordinates as... The pixel coordinates after distortion correction are Define the normalized radius as Then the distortion correction expression is:

[0034]

[0035] in, , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

[0036] In some embodiments, the signal points in the two-dimensional map of the microwave sensing device can be mapped and projected onto a visual image, and the corresponding distance, azimuth, height difference and measurement point number can be displayed.

[0037] This invention enables intuitive correspondence and reliable reproduction of signal points and real target measurement points in a two-dimensional microwave sensing spectrum in a visual image, reducing reliance on professional experience and repeated testing. It also improves the efficiency, consistency, and traceability of measurement point selection under complex field conditions such as installation height differences, pitch angles, and clutter, thereby meeting the needs of rapid deployment and long-term stable application in engineering monitoring.

[0038] Example 2 This invention also provides a visual fusion-based microwave sensing measurement point visualization mapping system, capable of executing the aforementioned visual fusion-based microwave sensing measurement point visualization mapping method, such as... Figure 3 As shown, it includes: The mounting bracket module is used to support the installation and fixation of the microwave sensing device and the vision device, so that their relative pose remains constant. The calibration module is used to assist the microwave sensing device in obtaining the horizontal height difference and horizontal angle information between the microwave sensing device and the target measuring point; The coordinate system transformation module is used to establish the microwave sensing coordinate system, the vision device coordinate system, and the reference coordinate system, and to calculate and store the coordinate transformation relationship with the reference coordinate system as a reference. The microwave measuring instrument module includes one and / or more microwave sensing devices for acquiring echoes from the target area and displaying target measurement point information in a range-azimuth two-dimensional spectral diagram; The measurement point mapping and image projection module is used to map and project the spatial position of the measurement point onto the visual image acquired by the vision device based on the coordinate transformation relationship and geometric parameters, so as to realize the visualization and overlay display of signal points in the two-dimensional map of the microwave sensing device in the visual image. The information recording module is used to record and store the measurement point number, pixel coordinates, spatial coordinates of the measurement point, calibration parameters and measurement configuration parameters, and output the measurement point configuration results and operation records.

[0039] In some embodiments, optionally, the mounting bracket module is further configured to support one and / or microwave sensing device and one and / or vision device to be installed and fixed on the same platform at the monitoring site, and has a structural design to prevent loosening and vibration to ensure that the relative posture remains constant over a long period of time.

[0040] In some embodiments, optionally, the mounting bracket module is further configured to adjust the beam orientation or mounting pitch angle of the microwave sensing device so that the detection range of the microwave sensing device can cover the area where the target measurement point is located, while ensuring that the field of view of the vision device includes the target measurement point area.

[0041] In some embodiments, optionally, the calibration module is further configured as a displacement slide with a fixed stroke to assist the microwave sensing device in obtaining three-dimensional information of the height difference and vertical angle of the measured target, thereby enabling the radar to obtain 3D target information.

[0042] In some embodiments, optionally, the coordinate system transformation module is further configured to acquire or solve a first transformation relationship from the microwave sensing device to the reference coordinate system and a second transformation relationship from the vision device to the reference coordinate system, and use the transformation relationship for the position expression and conversion of the measurement point in a unified reference coordinate system.

[0043] In some embodiments, optionally, the coordinate system transformation module is further configured to perform intrinsic parameter calibration on the vision device to obtain focal length, principal point and distortion parameters, and to overlay the projection results after distortion correction.

[0044] In some embodiments, optionally, the measurement point mapping and image projection module is further configured to transform the three-dimensional coordinates of the measurement point in the microwave sensing reference coordinate system to the visual coordinate system, calculate the pixel coordinates of the measurement point based on the imaging model, and output the projection position, corresponding distance and azimuth information of the measurement point and the measurement point number superimposed on the image.

[0045] In some embodiments, optionally, the information recording module is further configured to write the target or target point number, the measurement point pixel coordinates, the measurement point spatial coordinates, the coordinate transformation parameters, the geometric calibration parameters, and the microwave sensing device configuration parameters into a database or local file, and output a configuration list, verification screenshots, or reports to support consistency and traceability in repeated deployments.

[0046] On the other hand, the present invention also provides a microwave sensing measurement point visualization mapping device based on visual fusion, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor is configured to implement the steps of the above-described microwave sensing measurement point visualization mapping method based on visual fusion when executing the computer program.

[0047] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the above-described visualization mapping method for microwave sensing measurement points based on visual fusion.

[0048] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0049] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A visualization mapping method for microwave sensing measurement points based on visual fusion, characterized in that, include: Step 1: Rigidly connect the microwave sensing device and the vision device to keep their relative pose constant; Step 2: Collect the echo of the area where the target measurement point is located using a microwave sensing device and detect the corresponding measurement point information in the range-azimuth two-dimensional spectrum; collect the visual image of the echo area using a vision device; Step 3: Using a fixed reference coordinate system as the reference coordinate system, transform the microwave sensing coordinate system and the visual coordinate system to establish the first transformation relationship from the microwave sensing coordinate system to the reference coordinate system and the second transformation relationship from the visual coordinate system to the reference coordinate system. Step 4: Obtain the height difference information between the microwave sensing device and the target measurement point by measurement, and expand the signal points in the range-azimuth two-dimensional spectrum into three-dimensional space accordingly; Step 5: Map and project the three-dimensional spatial coordinates of the target measurement point onto the visual image acquired by the vision device to realize the visualization and mapping display of the echo signal point of the microwave sensing device in the visual image.

2. The method for visual mapping of microwave sensing measurement points based on visual fusion according to claim 1, characterized in that, Step 2 includes: determining one or more target points to be measured based on the stress characteristics, deformation sensitivity, or monitoring task requirements of the structure, and setting up reflectors or selecting natural reflection points with preset stable scattering characteristics as measurement points at the target points.

3. The method for visual mapping of microwave sensing measurement points based on visual fusion according to claim 1, characterized in that, Step 3 includes: establishing a first transformation relationship from the microwave sensing coordinate system to the reference coordinate system based on the installation position and attitude parameters of the microwave sensing device. ;in, These are the coordinates of the target measurement point in the reference coordinate system. These are the coordinates of the target measurement point in the microwave sensing coordinate system. and It is the attitude and position of the microwave sensing device in the reference coordinate system; the second transformation relationship from the visual coordinate system to the reference coordinate system is established as follows: ;in These are the coordinates of the target measurement point in the visual coordinate system. and This refers to the orientation and position of the vision device in the reference coordinate system; based on the first and second transformation relationships, the transformation relationship from the microwave sensing coordinate system to the vision coordinate system is obtained as follows: ;in It is the rotation matrix from the microwave sensing coordinate system to the visual coordinate system. It is the translation vector from the microwave sensing coordinate system to the visual coordinate system.

4. The method for visual mapping of microwave sensing measurement points based on visual fusion according to claim 1, characterized in that, Step 4 includes: the microwave sensing device using an auxiliary device to move at a preset vertical displacement. Under the condition of acquiring the absolute distance between the microwave sensing device and the target measuring point before and after displacement, and or absolute distance before displacement and the difference in relative distance before and after displacement Based on the horizontal angle of the target in the two-dimensional range-azimuth spectrum. Establish a three-dimensional spatial coordinate system for the microwave sensing device and calculate the three-dimensional coordinate information of the target to be measured: In this coordinate system, the center of the microwave sensing device is the origin. The Y-axis passes through the origin and is horizontally tangent to the transmitting plane of the microwave sensing device, with the left side of the microwave sensing device as the positive direction. The X-axis passes through the origin and is perpendicular to the transmitting plane of the microwave sensing device, with the line of sight of the microwave sensing device as the positive direction. The Z-axis passes through the origin and is perpendicular to the XY plane, with upward as the positive direction. The vertical angle between the X-axis and the target measuring point is calculated based on the three-dimensional coordinate information. The horizontal height difference between the microwave sensing device and the target measuring point This allows the two-dimensional distance-azimuth measurement points to be expanded and restored to three-dimensional spatial coordinates in the coordinate system of the microwave sensing device.

5. The method for visual mapping of microwave sensing measurement points based on visual fusion according to claim 1, characterized in that, Step 5 includes: calibrating the intrinsic parameters of the vision device to obtain focal length, principal point, and distortion parameters; transforming the three-dimensional coordinates of the target measurement point in the microwave sensing coordinate system to the vision coordinate system, calculating the pixel coordinates of the measurement point based on the pinhole imaging model and performing distortion correction, and then superimposing them onto the world image acquired by the vision device to form a visual annotation of the target measurement point in the image; the distortion correction includes: denoting the calculated distortion-free pixel coordinates as... The pixel coordinates after distortion correction are Define the normalized radius as Then the distortion correction expression is: in, , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

6. The method for visual mapping of microwave sensing measurement points based on visual fusion according to claim 1, characterized in that, Step 5 includes: mapping and projecting the signal points in the two-dimensional map of the microwave sensing device onto a visual image, and displaying the corresponding distance, azimuth, height difference and measurement point number information.

7. A visualization mapping system for microwave sensing measurement points based on visual fusion, characterized in that, For performing the method according to any one of claims 1 to 6, comprising: The mounting bracket module is used to support the installation and fixation of the microwave sensing device and the vision device, so that their relative pose remains constant. The output end of the mounting bracket module is connected to the input end of the microwave sensing device and the vision device. A calibration module is used to assist the microwave sensing device in obtaining the horizontal height difference and horizontal angle information between the microwave sensing device and the target measuring point. The output end of the calibration module is connected to the input end of the microwave measuring instrument module. The coordinate system transformation module is used to establish the microwave sensing coordinate system, the vision device coordinate system, and the reference coordinate system, and to calculate and store the coordinate transformation relationship with the reference coordinate system as a reference. The input end of the coordinate system transformation module is connected to the output end of the mounting bracket module, and the output end is connected to the input end of the measurement point mapping and image projection module. The microwave measuring instrument module includes one and / or more microwave sensing devices for acquiring target area echoes and displaying target measurement point information in a range-azimuth two-dimensional spectral diagram. The input terminal of the microwave measuring instrument module is connected to the output terminal of the calibration module, and the output terminal is connected to the input terminal of the measurement point mapping and image projection module. The measurement point mapping and image projection module is used to map and project the spatial position of the measurement point onto the visual image acquired by the vision device based on the coordinate transformation relationship and geometric parameters, so as to realize the visualization and superposition display of the signal points in the two-dimensional map of the microwave sensing device in the visual image. The input end of the measurement point mapping and image projection module is connected to the output end of the coordinate system transformation module and the microwave measuring instrument module respectively, and the output end is connected to the input end of the information recording module. The information recording module is used to record and store the measurement point number, pixel coordinates, spatial coordinates of the measurement point, calibration parameters and measurement configuration parameters, and output the measurement point configuration results and operation records.

8. The microwave sensing measurement point visualization mapping system based on visual fusion according to claim 7, characterized in that, The mounting bracket module is further used to support the installation and fixation of one and / or microwave sensing device and one and / or vision device on the same platform at the monitoring site, and has a structural design to prevent loosening and vibration to ensure that the relative posture remains constant over a long period of time; the mounting bracket module is also used to adjust the beam orientation or installation pitch angle of the microwave sensing device so that the detection range of the microwave sensing device can cover the area where the target measurement point is located, while ensuring that the field of view of the vision device includes the target measurement point area.

9. The microwave sensing measurement point visualization mapping system based on visual fusion according to claim 7, characterized in that, The calibration module further includes a displacement slide with a fixed stroke, used to assist the microwave sensing device in obtaining three-dimensional information of the height difference and vertical angle of the measured target.

10. The microwave sensing measurement point visualization mapping system based on visual fusion according to claim 7, characterized in that, The coordinate system transformation module is used to calibrate the intrinsic parameters of the vision device to obtain the focal length, principal point and distortion parameters, and to overlay the projection results after distortion correction.

11. The microwave sensing measurement point visualization mapping system based on visual fusion according to claim 7, characterized in that, The measurement point mapping and image projection module is used to transform the three-dimensional coordinates of the measurement points in the microwave sensing reference coordinate system to the visual coordinate system, calculate the pixel coordinates of the measurement points based on the imaging model, and superimpose the projection position, corresponding distance and azimuth information of the measurement points and the measurement point number into the image for output.

12. The microwave sensing measurement point visualization mapping system based on visual fusion according to claim 7, characterized in that, The information recording module is used to write the target or target point number, the pixel coordinates of the measuring point, the spatial coordinates of the measuring point, the coordinate transformation parameters, the geometric calibration parameters, and the configuration parameters of the microwave sensing device into a database or local file, and output a configuration list, verification screenshot or report.

Images