Stereoscopic vision and microwave fused microwave vibration camera system and visualization method thereof

Abstract

The invention provides a stereo vision and microwave fused microwave vibration camera system and a visualization method thereof, and the method comprises the steps: a microwave transceiver is fixedly connected with a stereo camera, and the stereo camera outputs camera data of a target to a camera data processing module to calculate the three-dimensional coordinates of each pixel point in a camera coordinate system; the coordinate conversion module converts the three-dimensional coordinates into microwave distance-angle heat map coordinates; the microwave data processing module processes a baseband signal obtained by the microwave transceiver, and performs distance-angle thermogram imaging on a target in a field of view; and the displacement visualization module maps the displacement extracted according to the microwave distance-angle heat map coordinates corresponding to each pixel point to a visual image in a mode including color coding to obtain a visual real-time vibration state in the field of view. According to the invention, a brand new measuring point identification and vibration source positioning method is provided for microwave vibration detection from the perspective of vibration visualization, and the application potential of a microwave full-field vibration measurement technology in a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of full-field vibration visualization of stereoscopic vision and microwave fusion, and specifically to a microwave vibration camera system and a visualization method thereof that integrates stereoscopic vision and microwave fusion. In particular, it relates to a full-field vibration visualization method and a microwave vibration camera system that integrates stereoscopic vision and microwave fusion. Background Art

[0002] By capturing the global vibration displacement, velocity or acceleration information of the target object, full-field vibration measurement technology can provide key information such as deformation, stress distribution and structural performance of the object being measured. Therefore, this technology is widely used in fields such as health monitoring of large structures, mechanical property analysis of materials, and fault diagnosis of mechanical equipment, and has become a key testing method in engineering and scientific research.

[0003] Microwave sensing technology, with its multi-scale, large detection range, high sensitivity, and all-weather characteristics, provides a new solution for full-field vibration monitoring of large structures and equipment. Based on microwave single-point vibration measurement technology and long-distance multi-point vibration measurement technology, microwave full-field vibration measurement technology has made great progress - using the distance-angle joint dimensional nonlinear demodulation algorithm, the microwave transceiver can perform distance-angle imaging of the global measurement points. Furthermore, using the interference phase evolution tracking algorithm, the vibration displacement information of each measurement point along the line of sight can be obtained.

[0004] However, due to the low resolution of distance-angle imaging of microwave transceivers, full-field group measurement point identification and vibration source location have become a thorny problem in actual measurement. Although some research works have attempted to align and calibrate visual images and microwave data, the microwave imaging in these studies is mostly three-dimensional point clouds, or it is assumed that the targets are all in the same plane, which is significantly different from the imaging method of distance-angle thermal images in microwave full-field vibration measurement, and it is difficult to meet the accuracy requirements of measurement point identification and vibration source location in microwave full-field vibration measurement, which restricts the application of microwave full-field vibration measurement technology in complex scenes.

[0005] Therefore, there is an urgent need for an intuitive and accurate microwave vibration measurement point identification and vibration source location technology and method. Summary of the invention

[0006] In view of the defects in the prior art, an object of the present invention is to provide a microwave vibration camera system integrating stereoscopic vision and microwave and a visualization method thereof.

[0007] A microwave vibration camera system integrating stereoscopic vision and microwave provided by the present invention comprises: a microwave transceiver, a stereo camera, a microwave data processing module, a camera data processing module, a coordinate conversion module and a displacement visualization module;

[0008] The microwave transceiver is fixedly connected to the stereo camera, and the stereo camera outputs the camera data of the target to the camera data processing module to calculate the three-dimensional coordinates of each pixel point in the camera coordinate system;

[0009] The coordinate conversion module converts the three-dimensional coordinates into microwave distance-angle heat map coordinates; the microwave data processing module processes the baseband signal obtained by the microwave transceiver to perform distance-angle heat map imaging on the target in the field of view;

[0010] The displacement visualization module maps the displacement extracted from the microwave distance-angle heat map coordinates corresponding to each pixel point onto the visual image in a manner including color coding to obtain a real-time vibration state visualized in the field of view.

[0011] Preferably, the stereo camera uses a stereo vision form including a binocular camera, a structured light camera, and a TOF camera to output camera data of the target;

[0012] The camera data includes disparity or depth information;

[0013] The camera data processing method includes triangulation method, time difference calculation method, structured light method and deep learning method.

[0014] Preferably, the microwave transceiver is used to transmit and receive linear frequency modulated continuous wave microwave signals, output a multi-channel baseband signal, and is used for subsequent distance-angle heat map positioning of targets in the field of view, and for vibration displacement measurement of targets in the field of view in the line of sight direction according to the distance-angle index of each target in the field of view.

[0015] Preferably, the coordinate conversion module is used to calculate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, and to convert and map each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate.

[0016] Preferably, the displacement visualization module is used to extract the sight line displacement indexed by the distance-angle index corresponding to each pixel point, and superimpose it on the original RGB image after color coding to obtain a real-time displacement amplitude heat map visualized in the field of view.

[0017] A visualization method of a microwave vibration camera system integrating stereoscopic vision and microwaves provided by the present invention comprises:

[0018] Step S1: calibrate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera;

[0019] Step S2: converting each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate;

[0020] Step S3: extract the displacement information in the microwave line of sight direction and output the displacement visualization.

[0021] Preferably, the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the calibration camera are calibrated using methods including the checkerboard calibration method, the Zhang Zhengyou calibration method, and an open source toolbox;

[0022] The coordinate transformation matrix of the calibration camera coordinate system and the antenna coordinate system includes recording the coordinates of each reference target in the camera coordinate system and the antenna coordinate system as follows:

[0023]

[0024] Among them, P cam is the three-dimensional coordinate of the current target in the camera coordinate system, P cam A column of is a target; radar is the three-dimensional coordinate of the current target in the antenna coordinate system, P radar A column is a target; n≥3;

[0025] Assume that the average coordinates of all current reference targets in the camera coordinate system are The average coordinate in the antenna coordinate system is Right now:

[0026]

[0027] Constructing the Matrix in Indicates P cam The i-th column of Indicates P radar The i-th column of . Perform singular value decomposition of the matrix M:

[0028] M=UΣV T

[0029] Then the rotation matrix from the camera coordinate system to the antenna coordinate system is:

[0030] R=VU T

[0031] The translation matrix is:

[0032]

[0033] Preferably, the step S2 comprises:

[0034] Step S2.1: Take a pixel point in the pixel plane of one of the RGB cameras of the stereo camera, and mark the pixel coordinate of the pixel point as (u i ,v i), calculate the three-dimensional coordinates (xc_i, yc_i, zc_i) of the pixel in the camera coordinate system;

[0035] Step S2.2: Then, the rotation matrix R and the translation matrix T obtained in step S1 are used to obtain the three-dimensional coordinates (x r_i ,y r_i ,z r_i ):

[0036]

[0037] Step S2.3: The three-dimensional coordinates (x r_i ,y r_i ,z r_i ) into a microwave distance angle heat map:

[0038]

[0039] Among them, R i is the distance value corresponding to the pixel in the microwave distance angle heat map; θ i is the angle value corresponding to the pixel point in the microwave distance angle heat map;

[0040] Repeat steps S2.1 to S2.3 until all pixels of the current RGB camera are traversed to obtain the distance and angle values ​​corresponding to all pixels in the microwave distance-angle heat map.

[0041] Preferably, the three-dimensional coordinates of the pixel in the camera coordinate system are calculated by methods including triangulation, time difference calculation, structured light method, and deep learning method.

[0042] Preferably, step S3 comprises:

[0043] Step S3.1: According to the microwave distance-angle heat map coordinates corresponding to each pixel point obtained in step S2, the distance index and angle index corresponding to the pixel point are reversely calculated;

[0044] Step S3.2: extracting the displacement value of each pixel at the current moment through a microwave full-field displacement measurement algorithm;

[0045] Step S3.3: At the current moment, all pixels are superimposed on the original RGB image according to the displacement values ​​by means of color coding, so as to realize the visualization of vibration displacement;

[0046] At subsequent moments, steps S3.1 to S3.2 are repeated until data collection is stopped and the vibration displacement visualization at all moments is completed.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention overcomes the problem that the distance-angle thermal imaging method used in the existing microwave full-field vibration measurement technology is difficult to complete measurement point identification and vibration source positioning in complex scenes, and proposes a novel concept of a microwave vibration camera, which integrates the vibration measurement information of the microwave transceiver and the spatial positioning information of the stereo camera, realizes the vibration visualization of the target in the field of view, expands the application scope of microwave full-field vibration measurement technology, and provides a feasible implementation plan for measurement point identification and vibration source positioning of microwave full-field vibration measurement in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0050] Figure 1 It is a schematic flow chart of the full-field vibration visualization method of the present invention integrating stereoscopic vision and microwaves;

[0051] Figure 2 A schematic diagram of the calibration scene of the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system in the present invention;

[0052] Figure 3 Schematic diagram of the coordinate mapping process of a pixel point from a pixel coordinate system to a microwave distance-angle heat map in the present invention;

[0053] Figure 4 This is a structural block diagram of the microwave vibration camera system in the present invention;

[0054] Figure 5 This is a schematic diagram of the experimental scene of Experiment 1 of the present invention;

[0055] Figure 6 This is a schematic diagram of the test results of Experiment 1 of the present invention;

[0056] Figure 7 This is a schematic diagram of the experimental scene and test results of Experiment 2 of the present invention;

[0057] Figure 8 This is a schematic diagram of the experimental scene and test results of Experiment 3 of the present invention. DETAILED DESCRIPTION

[0058] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0059] The present invention provides a new measurement point identification and vibration source positioning method for microwave vibration detection from the perspective of vibration visualization, thereby improving the application potential of microwave full-field vibration measurement technology in complex scenarios.

[0060] According to the present invention, a microwave vibration camera system integrating stereoscopic vision and microwave is provided. Figure 4 As shown, it includes: a microwave transceiver, a stereo camera, a microwave data processing module, a camera data processing module, a coordinate conversion module and a displacement visualization module. The microwave transceiver and the stereo camera are fixedly connected, and the stereo camera outputs the camera data of the target to the camera data processing module to calculate the three-dimensional coordinates of each pixel in the camera coordinate system. The coordinate conversion module converts the three-dimensional coordinates into microwave distance-angle heat map coordinates. The microwave data processing module processes the baseband signal obtained by the microwave transceiver and performs distance-angle heat map imaging of the target in the field of view. The displacement visualization module extracts the displacement obtained according to the microwave distance-angle heat map coordinates corresponding to each pixel, and superimposes it on the visual image after color coding to obtain a real-time displacement amplitude heat map visualized in the field of view.

[0061] The microwave transceiver is used to transmit and receive linear frequency modulated continuous wave microwave signals, and output a multi-channel baseband signal for subsequent distance-angle heat map positioning of the target in the field of view, and for measuring the vibration displacement of the target in the field of view in the line of sight direction according to the distance-angle index of each target in the field of view. The stereo camera uses stereo vision forms including binocular cameras, structured light cameras, and TOF cameras to output camera data of the target. The camera data includes parallax or depth information. The camera data processing method includes triangulation, time difference calculation method, structured light method, and deep learning method. The coordinate conversion module is used to calculate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, and to convert and map each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate. The displacement visualization module is used to extract the line of sight direction displacement corresponding to the distance-angle index of each pixel point, and superimpose it on the original RGB image after color coding to obtain a real-time displacement amplitude heat map visualized in the field of view.

[0062] According to a visualization method of a microwave vibration camera system integrating stereoscopic vision and microwave provided by the present invention, Figure 1 As shown, including:

[0063] First, calibrate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera. Then, traverse each pixel in the visual image and convert them one by one to obtain their corresponding distance-angle coordinates in the microwave thermal map. Finally, the microwave full-field vibration displacement measurement method is used to obtain the vibration displacement information of each pixel point in the microwave line of sight direction at the current moment, and color-code it and superimpose it on the original visual image to complete the visualization of the real-time amplitude at the current moment, and repeat the steps at the next moment. Specifically include the following:

[0064] Step S1: calibrate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera. Step S1 includes:

[0065] Step S1.1: Calibrate the camera's intrinsic matrix, extrinsic matrix, and distortion coefficients. Use methods including the checkerboard calibration method, Zhang Zhengyou calibration method, and open source toolbox to calibrate the stereo camera's intrinsic matrix, extrinsic matrix, and distortion coefficients.

[0066] Step S1.2: Calibrate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system. Figure 2 As shown, arrange the calibration scene and select three or more targets in the field of view that can be sensed by the microwave transceiver and the stereo camera as reference targets, for example Figure 2 The camera coordinate system and antenna coordinate system are as follows: Figure 3 As shown, the coordinates of each reference target in the camera coordinate system and the antenna coordinate system are recorded as:

[0067]

[0068] Among them, P cam is the three-dimensional coordinate of the current target in the camera coordinate system, P cam A column of is a target; radar is the three-dimensional coordinate of the current target in the antenna coordinate system, P radar Each column of is a target; n≥3, n represents the number of targets.

[0069] Assume that the average coordinates of all current reference targets in the camera coordinate system are The average coordinate in the antenna coordinate system is Right now:

[0070]

[0071] Constructing the Matrix in Indicates P cam The i-th column of Indicates Pradar The i-th column of . Perform singular value decomposition of the matrix M:

[0072] M=UΣV T

[0073] Then the rotation matrix from the camera coordinate system to the antenna coordinate system is:

[0074] R=VU T

[0075] The translation matrix is:

[0076]

[0077] Step S2: Convert each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate. Figure 3 As shown, step S2 includes:

[0078] Step S2.1: Take a pixel point in the pixel plane of one of the RGB cameras of the stereo camera and mark its pixel coordinates as (u i ,v i ), and calculate the three-dimensional coordinates (x c_i ,y c_i ,z c_i ):.

[0079] Step S2.2: Then, the rotation matrix R and the translation matrix T obtained in step S1.2 are used to obtain the three-dimensional coordinates (x r_i ,y r_i ,z r_i ):

[0080]

[0081] Step S2.3: The three-dimensional coordinates (x r_i ,y r_i ,z r_i ) into a microwave distance angle heat map:

[0082]

[0083] Among them, R i is the distance value corresponding to the pixel in the microwave distance angle heat map; θ i is the angle value corresponding to the pixel in the microwave distance angle heat map.

[0084] Repeat steps S2.1 to S2.3 until all pixels of the RGB camera are traversed to obtain the distance and angle values ​​corresponding to all pixels in the microwave distance-angle heat map.

[0085] Step S3: Extract the displacement information in the microwave line of sight direction and output the displacement visualization. According to the microwave distance-angle heat map coordinates corresponding to each pixel point obtained in step S2, the corresponding distance index and angle index are inversely calculated, and then the displacement value of each pixel point at the current moment is extracted through the microwave full-field displacement measurement algorithm. At the current moment, all pixels are color-coded according to the size of the displacement value and superimposed on the original RGB image to realize the visualization of vibration displacement.

[0086] The process of step S3 is repeated at subsequent moments until data collection is stopped and the vibration displacement visualization at all moments is completed.

[0087] Furthermore, by utilizing the microwave vibration camera system proposed in the present invention and executing the steps of the above-mentioned full-field vibration visualization method of stereoscopic vision and microwave fusion, the implementation results in three different experimental scenarios are specifically described as follows:

[0088] Embodiment 1

[0089] First, the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera are calibrated. The microwave vibration camera with the calibrated coordinate transformation matrix is ​​set up in front of the optical platform. Four corner reflectors are placed on the optical platform as targets. Among them, targets 1 and 2 are directly fixed on the optical platform and remain stationary throughout the experiment. Targets 3 and 4 are placed on two slides respectively and move along the Figure 5 In the direction of the black arrow, periodic reciprocating motions with peak-to-peak values ​​of 10 mm and 5 mm are performed respectively.

[0090] Then, the coordinates of each pixel are converted to the corresponding microwave distance-angle heat map coordinates. The displacement information of the microwave line of sight is extracted and the displacement is visualized and output. Figure 6 (a) shows the displacement time series of the four targets along the line of sight measured by the microwave vibration camera. It can be seen that due to the angle between the line of sight and the displacement direction of the two slides, Figure 6 The peak-to-peak displacements of targets 3 and 4 in (a) are slightly smaller than the set values ​​of the slide. Figure 6 (b) to (d) show the imaging results of the microwave vibration camera at three typical moments. The displacement amplitude measured by the microwave vibration camera is normalized, and each pixel is assigned a different color based on the ratio of the displacement value at the current moment to the maximum displacement amplitude. It can be seen that at the three typical moments, the normalized displacement values ​​corresponding to the pixel areas of targets 3 and 4 are consistent with the displacement values ​​at the corresponding moments.

[0091] Embodiment 2

[0092] like Figure 7 As shown in Figure 2, Experiment 2 uses a microwave vibration camera on the rooftop to visualize the vibration of an exhaust gas purification device. First, the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera are calibrated; then, each pixel coordinate is converted to obtain the corresponding microwave distance-angle heat map coordinate; finally, the displacement information of the microwave line of sight direction is extracted. Figure 7 As shown in (a), the microwave vibration camera is facing the exhaust duct of an exhaust gas purification device. Next to the exhaust duct, there is a centrifugal fan that keeps running to extract the purified exhaust gas. The fan generates a lot of noise and causes its casing and the exhaust duct to vibrate.

[0093] Then, the coordinates of each pixel are converted to the corresponding microwave distance-angle heat map coordinates. The microwave transceiver samples at a sampling frequency of 320 Hz. Figure 7 (b) shows the imaging result of the microwave vibration camera at the 76th frame, which intuitively shows the real-time vibration of each part of the device. At the same time, the accelerometer is fixed on the casing of the centrifugal fan and sampled at a sampling rate of 10000 Hz. Figure 7 (d) shows the frequency spectrum of the accelerometer sampling sequence.

[0094] Finally, the displacement information in the microwave line of sight direction is extracted and the displacement is visualized and output. Figure 7 (c) is the frequency spectrum of the displacement time series obtained by using a microwave vibration camera to monitor the displacement of the measuring point near the accelerometer on the casing of the centrifugal fan. Figure 7 (c) and Figure 7 (d) From these two spectrum diagrams, we can see that the vibration frequency measurement results of the microwave vibration camera are in good agreement with those of the accelerometer.

[0095] Embodiment 3

[0096] like Figure 8 As shown in Figure 3, Experiment 3 visualizes the vibration of the car engine startup and acceleration process using a microwave vibration camera. First, the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera are calibrated, and then the coordinates of each pixel are converted to obtain the corresponding microwave distance-angle heat map coordinates; finally, the displacement information of the microwave line of sight direction is extracted and the displacement is visualized and output. The experimental results are as follows:

[0097] like Figure 8As shown in (a), the microwave vibration camera is mounted on a tripod in front of the car engine, and an accelerometer is fixed on the engine protection cover. Their sampling frequencies are still maintained at 320Hz and 10000Hz respectively. During the simultaneous monitoring process of the microwave vibration camera and the accelerometer, the car engine is started, and after a while, the accelerator is pressed to increase the engine speed. Figure 8 (b) is the vibration visualization result at the moment of engine startup, which intuitively shows that the area with larger vibration displacement occurs near the engine casing. Figure 8 (c) and Figure 8 (d) shows the time-frequency diagram of the measurement results of the microwave vibration camera and the accelerometer (after downsampling), respectively. It can be seen that the frequency change trends of the two are basically the same, and both can be well matched to the behavior of engine startup and speed increase.

[0098] The present invention aims to solve the problem that the existing microwave full-field vibration measurement method cannot accurately and intuitively identify the measuring point and locate the vibration source. The present invention uses stereo vision technology to complete the identification of microwave distance-angle thermal imaging measuring points, and maps the microwave full-field vibration measurement results on the visual image through methods including color coding to obtain a visualized real-time vibration state in the field of view.

[0099] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0100] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A microwave vibration camera system integrating stereoscopic vision and microwave, characterized in that: include: Microwave transceiver, stereo camera, microwave data processing module, camera data processing module, coordinate conversion module and displacement visualization module; The microwave transceiver is used to transmit and receive linear frequency modulated continuous wave microwave signals and output multi-channel baseband signals; The microwave transceiver is fixedly connected to the stereo camera, and the stereo camera outputs the camera data of the target to the camera data processing module to calculate the three-dimensional coordinates of each pixel point in the camera coordinate system; The coordinate conversion module converts the three-dimensional coordinates into microwave distance-angle heat map coordinates; the microwave data processing module processes the baseband signal obtained by the microwave transceiver to perform distance-angle heat map imaging on the target in the field of view; The displacement visualization module maps the displacement extracted from the microwave distance-angle heat map coordinates corresponding to each pixel point onto the visual image in a manner including color coding to obtain a real-time vibration state visualized in the field of view.

2. The microwave vibration camera system integrating stereoscopic vision and microwave according to claim 1, characterized in that: The stereo camera uses stereo vision forms including binocular cameras, structured light cameras, and TOF cameras to output camera data of the target; The camera data includes disparity or depth information; The camera data processing method includes triangulation method, time difference calculation method, structured light method and deep learning method.

3. The microwave vibration camera system integrating stereoscopic vision and microwave according to claim 1, characterized in that: The microwave transceiver is used to transmit and receive linear frequency modulated continuous wave microwave signals, and output a multi-channel baseband signal for subsequent distance-angle heat map positioning of targets in the field of view, and for vibration displacement measurement of targets in the field of view in the line of sight direction according to the distance-angle index of each target in the field of view.

4. The microwave vibration camera system integrating stereoscopic vision and microwave according to claim 1, characterized in that: The coordinate conversion module is used to calculate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, and to convert and map each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate.

5. The microwave vibration camera system integrating stereoscopic vision and microwave according to claim 1, characterized in that: The displacement visualization module is used to extract the sight direction displacement of the distance-angle index corresponding to each pixel point, and map it on the visual image through a method including color coding to obtain the visual field. Target Visualize the real-time vibration status.

6. A visualization method for a microwave vibration camera system integrating stereoscopic vision and microwave, characterized in that: include: Step S1: calibrate the coordinate transformation matrix of the camera coordinate system and the antenna coordinate system, as well as the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the stereo camera; Step S2: converting each pixel coordinate to obtain the corresponding microwave distance-angle heat map coordinate; Step S3: extract the displacement information in the microwave line of sight direction and output the displacement visualization.

7. The visualization method of the microwave vibration camera system integrating stereoscopic vision and microwave according to claim 6, characterized in that: The intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the calibration camera are calibrated using methods including the checkerboard calibration method, the Zhang Zhengyou calibration method, and the open source toolbox; The coordinate transformation matrix of the calibration camera coordinate system and the antenna coordinate system includes recording the coordinates of each reference target in the camera coordinate system and the antenna coordinate system as follows: Among them, P cam is the three-dimensional coordinate of the current target in the camera coordinate system, P cam A column of is a target; radar is the three-dimensional coordinate of the current target in the antenna coordinate system, P radar A column is a target; n≥3; Assume that the average coordinates of all current reference targets in the camera coordinate system are The average coordinate in the antenna coordinate system is Right now: Constructing the Matrix in Indicates P cam The i-th column of Indicates P radar The i-th column of . Perform singular value decomposition of matrix M: M=UΣV T Then the rotation matrix from the camera coordinate system to the antenna coordinate system is: R=VU T The translation matrix is:

8. The visualization method of the microwave vibration camera system integrating stereoscopic vision and microwave according to claim 6, characterized in that: The step S2 comprises: Step S2.1: Take a pixel point in the pixel plane of one of the RGB cameras of the stereo camera, and mark the pixel coordinate of the pixel point as (u i ,v i ), calculate the three-dimensional coordinates (x c_i ,y c_i ,z c_i ); Step S2.2: Then, the rotation matrix R and the translation matrix T obtained in step S1 are used to obtain the three-dimensional coordinates (x r_i ,y r_i ,z r_i ): Step S2.3: The three-dimensional coordinates (x r_i ,y r_i ,z r_i ) into a microwave distance angle heat map: Among them, R i is the distance value corresponding to the pixel in the microwave distance angle heat map; θ i is the angle value corresponding to the pixel point in the microwave distance angle heat map; Repeat steps S2.1 to S2.3 until all pixels of the current RGB camera are traversed to obtain the distance and angle values ​​corresponding to all pixels in the microwave distance-angle heat map.

9. The visualization method of the microwave vibration camera system integrating stereoscopic vision and microwave according to claim 8, characterized in that: The three-dimensional coordinates of the pixel in the camera coordinate system are calculated by methods including triangulation, time difference calculation, structured light method, and deep learning method.

10. The visualization method of the microwave vibration camera system integrating stereoscopic vision and microwave according to claim 6, characterized in that: The step S3 comprises: Step S3.1: According to the microwave distance-angle heat map coordinates corresponding to each pixel point obtained in step S2, the distance index and angle index corresponding to the pixel point are reversely calculated; Step S3.2: extracting the displacement value of each pixel at the current moment through a microwave full-field displacement measurement algorithm; Step S3.3: At the current moment, all pixel points are mapped onto the original visual RGB image according to the displacement values ​​by means including color coding, so as to obtain the real-time vibration state of the target in the field of view; At subsequent moments, steps S3.1 to S3.2 are repeated until data collection is stopped and the vibration displacement visualization at all moments is completed.

Images