A global three-dimensional deformation remote sensing radar monitoring system and method

Abstract

This invention provides a full-domain three-dimensional deformation remote sensing radar monitoring system and method. The method includes: a first image acquisition unit acquiring multiple aperture radar images of the monitored scene generated sequentially by the radar in the current cycle to obtain a radar panoramic image; a second image acquisition unit acquiring multiple optical photographs of the monitored scene sequentially captured by an optical camera to construct a visual panoramic image; a radar monitoring quantity image construction unit, used to obtain a radar monitoring quantity image based on the change in monitoring quantity between the current cycle and the previous cycle's radar panoramic image; an image marking unit marking the positions of the radar monitoring quantity image and the visual panoramic image; and a fusion image construction unit cropping the radar monitoring quantity image and the visual panoramic image respectively, and superimposing the radar monitoring quantity image slices onto the corresponding visual image slices to obtain a fused image. This allows operators to directly understand the situation of potential hazard areas.

Description

Technical Field

[0001] This invention relates to the field of engineering monitoring technology, specifically to a full-domain three-dimensional deformation remote sensing radar monitoring system and method. Background Technology

[0002] In the field of engineering monitoring, construction monitoring and emergency monitoring of the structural stability of large foundation pits, tunnels, and buildings are crucial. Emergency monitoring can be long-term or short-term, depending on the specific event; for example, monitoring dilapidated buildings during rescue operations or monitoring the exterior walls of buildings in the event of roof collapse. Traditional monitoring methods have many limitations. For example, some monitoring equipment is greatly affected by environmental factors and cannot function properly under adverse conditions such as rain, snow, fog, smoke, dust, and fire. Some monitoring technologies cannot accurately obtain real-time deformation area information and cannot provide timely monitoring data such as the location, size, deformation amount, and deformation rate of the deformation area.

[0003] In addition, existing monitoring technologies are insufficient in terms of visualization, making it difficult for operators to intuitively understand the situation in potential hazard areas, which is not conducive to the timely analysis and handling of engineering disasters. Summary of the Invention

[0004] This invention provides a full-domain three-dimensional deformation remote sensing radar monitoring system and method, which can solve the above-mentioned technical problems in the prior art.

[0005] To achieve the above objectives, in one aspect, embodiments of the present invention provide a full-domain three-dimensional deformation remote sensing radar monitoring system, comprising:

[0006] The first image acquisition unit is used to acquire multiple aperture radar images of the scene to be monitored generated sequentially by the radar in the current period, and to construct a radar panoramic image of the scene to be monitored by the multiple aperture radar images in the current period.

[0007] The second image acquisition unit is used to acquire multiple optical photos of the scene to be monitored taken sequentially by an optical camera located in the scene to be monitored in the current period, and to construct a visual panoramic image of the scene to be monitored by the multiple optical photos in the current period. The visual panoramic image and the radar panoramic image involve the same range of the scene to be monitored.

[0008] The radar monitoring quantity image construction unit is used to obtain the radar monitoring quantity image of the scene to be monitored based on the change in monitoring quantity between the radar panoramic image of the current period and the radar panoramic image of the previous period.

[0009] An image labeling unit is used to match and label multiple locations between the radar monitoring image of the current period and the visual panoramic image.

[0010] The image fusion unit is used to acquire, through the AR device, the real-time pitch and azimuth angles of the operator's head movement, as well as the operator's field of view, when the operator is viewing an image of the scene to be monitored via the AR device; and

[0011] Based on the operator's real-time head movement, pitch angle, azimuth angle, and field of view, the radar monitoring image and visual panoramic image within the field of view are determined. The radar monitoring image and visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image, which is used to display to the operator in real time.

[0012] On the other hand, embodiments of the present invention provide a method for monitoring full-domain three-dimensional deformation remote sensing radar, including:

[0013] Acquire multiple aperture radar images of the scene to be monitored in the current period generated sequentially by the radar located in the scene to be monitored, and construct a radar panoramic image of the scene to be monitored by the multiple aperture radar images in the current period;

[0014] Multiple optical photos of the scene to be monitored are acquired sequentially by an optical camera located in the scene to be monitored in the current period. The multiple optical photos in the current period are used to construct a visual panoramic image of the scene to be monitored. The visual panoramic image covers the same range of the scene to be monitored as the radar panoramic image.

[0015] Based on the change in the monitoring quantity of the radar panoramic image in the current period and the radar panoramic image in the previous period, a radar monitoring quantity image of the scene to be monitored is obtained.

[0016] The radar monitoring image of the current period is matched and marked with multiple locations in the visual panoramic image;

[0017] When the operator views an image of the scene to be monitored through an AR device, the AR device acquires the operator's real-time pitch and azimuth angles of head movement, as well as the operator's field of view.

[0018] Based on the operator's real-time head movement, pitch angle, azimuth angle, and field of view, the radar monitoring image and visual panoramic image within the field of view are determined. The radar monitoring image and visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image, which is used to display to the operator in real time.

[0019] The above technical solution has the following beneficial effects: multiple aperture radar images of the scene to be monitored are obtained by a radar (multiple radars can be set at intervals according to specific circumstances) located in the scene to be monitored in the current cycle. The radar can obtain multiple aperture radar images of the scene to be monitored by transmitting signals and receiving reflected signals, and thus can also obtain accurate monitoring data.

[0020] By using AR technology and employing fused images, monitoring data can be displayed on a panoramic visual image, providing operators with a complete view of the scene to be monitored. Operators can clearly see the specific changes in monitoring data for specific parts of the scene, thus enabling them to directly understand the situation of potential hazard areas and facilitating timely analysis and handling of engineering disasters. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a structural diagram of a full-domain three-dimensional deformation remote sensing radar monitoring system according to an embodiment of the present invention;

[0023] Figure 2 This is a flowchart of a global three-dimensional deformation remote sensing radar monitoring method according to an embodiment of the present invention;

[0024] Figure 3 This is a flowchart illustrating the sequential steps of a full-domain three-dimensional deformation remote sensing radar monitoring method according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, in conjunction with embodiments of the present invention, a full-domain three-dimensional deformation remote sensing radar monitoring system is provided, comprising:

[0027] The first image acquisition unit 11 is used to acquire multiple aperture radar images of the scene to be monitored generated sequentially by the radar in the current period, and to construct a radar panoramic image of the scene to be monitored by the multiple aperture radar images in the current period.

[0028] The second image acquisition unit 12 is used to acquire multiple optical photos of the scene to be monitored taken sequentially by an optical camera located in the scene to be monitored in the current period, and to construct a visual panoramic image of the scene to be monitored by the multiple optical photos in the current period. The visual panoramic image and the radar panoramic image involve the same range of the scene to be monitored.

[0029] The radar monitoring quantity image construction unit 13 is used to obtain the radar monitoring quantity image of the scene to be monitored based on the change in the monitoring quantity of the radar panoramic image in the current period and the radar panoramic image in the previous period.

[0030] Image marking unit 14 is used to perform multiple location matching and marking between the radar monitoring image of the current period and the visual panoramic image;

[0031] The image fusion construction unit 15 is used to acquire, through the AR device, the real-time pitch and azimuth angles of the operator's head movement, as well as the operator's field of view, when the operator is viewing an image of the scene to be monitored via the AR device; and

[0032] Based on the operator's real-time head movement, pitch angle, azimuth angle, and field of view, the radar monitoring image and visual panoramic image within the field of view are determined. The radar monitoring image and visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image, which is used to display to the operator in real time.

[0033] Multiple aperture radar images of the scene to be monitored are obtained sequentially in the current cycle by radars (multiple radars can be set at intervals according to specific circumstances) located in the scene to be monitored. The radar can acquire multiple aperture radar images of the scene to be monitored by transmitting signals and receiving reflected signals. This can solve the technical problem that "the monitoring equipment is greatly affected by environmental factors and cannot work normally under adverse conditions such as rain, snow, fog, smoke, dust, and fire". Therefore, it can also obtain accurate monitoring data.

[0034] By using AR technology and employing fused images, monitoring data can be displayed on a panoramic visual image, providing operators with a complete view of the scene to be monitored. Operators can clearly see the specific changes in monitoring data for specific parts of the scene, thus enabling them to directly understand the situation of potential hazard areas and facilitating timely analysis and handling of engineering disasters.

[0035] Preferably, the second image acquisition unit 12 is specifically used for:

[0036] A two-dimensional turntable of photoelectric equipment, positioned at the scene to be monitored, rotates, causing an optical camera to rotate and tilt. The step angle of the photoelectric camera ensures a defined overlap between adjacent optical images. With each step of the photoelectric camera, an optical image of the scene to be monitored is captured, ultimately achieving 360°×90° coverage of the scene and obtaining multiple optical images. This shooting method and the overlapping area setting ensure the integrity and accuracy of the panoramic image of the scene to be monitored.

[0037] Preferably, the full-domain three-dimensional deformation remote monitoring system further includes:

[0038] An alignment unit is used at the beginning of each cycle to set the positions of the optical camera and the radar to be co-located and aligned at zero angle. The rotation step angle of the optical camera is set to be the same as the rotation step angle of the radar, ensuring that the position of the optical image captured each time corresponds to the monitored scene in the aperture radar image. Assuming that the radar and optical camera are co-located and aligned at zero angle, this simplifies the algorithm design and radar deployment within a certain error range, with minimal impact on the actual identification of potential hazard areas. This simplified design reduces system complexity and cost, while improving system practicality and operability.

[0039] Preferably, the radar monitoring image construction unit 13 includes:

[0040] The first monitoring quantity construction subunit is used to filter out stable points from the radar panoramic image of the current period and the radar panoramic image of the previous period when the monitoring quantity is deformation.

[0041] The interferometric synthetic aperture radar (PS-InSAR) method is used to perform phase interference between the current period image and the previous period image corresponding to each stationary point to obtain the interferometric phase difference. The interferometric phase difference is then inverted into the deformation along the line of sight, and the accuracy of the deformation is less than or equal to 0.1 mm. The interferometric phase difference is obtained by performing phase interference between the current period image and the previous period image at each stationary point using the PS-InSAR method, and the deformation of the current period image and the previous period image is obtained.

[0042] At the stable point of the radar panoramic image, the cumulative deformation in the line-of-sight direction corresponding to the stable point is used to construct the radar deformation field image in the direction of pitch angle and azimuth angle. The cumulative deformation refers to the sum of the cumulative deformation of all two adjacent cycles from the start of monitoring to the current cycle.

[0043] The second monitoring quantity construction subunit is used to obtain the deformation rate after obtaining the deformation quantity when the monitoring quantity is the deformation rate, and to obtain the deformation rate field image.

[0044] The monitoring parameters are determined based on the monitoring scenario and specific requirements, such as location, area size, deformation, deformation rate, etc.

[0045] Preferably, the first monitoring quantity construction subunit is specifically used for:

[0046] The upper and lower limits of the deformation of the scene to be monitored are set, and the cumulative deformation of the radar resolution unit in the line-of-sight direction is represented by color scheme on the radar panoramic image; wherein, the representation of the deformation of the radar resolution unit in the line-of-sight direction by color scheme on the radar panoramic image specifically includes:

[0047] On the radar panoramic image, stable points of intangible variables are represented by a first color, stable points of negative deformation variables are represented by a second color, with gradient changes in the second color representing changes in the magnitude of negative deformation variables. Stable points of positive deformation variables are represented by a third color, with gradient changes in the third color representing changes in the magnitude of positive deformation variables. Instable points are rendered completely transparent, thus constructing a radar deformation field image. Using color coding to represent the corresponding monitored quantities makes the image more intuitive for the operator.

[0048] Preferably, the image marking unit 14 is specifically used for:

[0049] The radar monitoring image is assigned radar resolution units and marked with corresponding numbers. The visual panoramic image is divided into multiple adjacent optical image grids according to the radar imaging resolution units, and each optical image grid is assigned a corresponding number. The number of each optical image grid is then mapped to the number of the corresponding radar resolution unit. The radar resolution is represented by a horizontal rotation angle. This process prepares for subsequent image fusion generation and improves the generation speed of the fused image.

[0050] Preferably, the fused image construction unit 15 is specifically used for:

[0051] The radar monitoring images within the field of view are cropped according to radar resolution units to form radar monitoring slices, and the visual panoramic images within the field of view are cropped according to optical image grids to form visual image slices.

[0052] Based on the correspondence between radar resolution unit numbers and optical image grid numbers, radar monitoring slices are superimposed on the corresponding optical image grids. The alpha channel in RGBA is set to adjust the transparency of the radar monitoring slice colors, forming a fused image.

[0053] Displaying monitoring quantities on top of a panoramic visual image is clear, intuitive, and convenient. Operators can visually see the location and deformation of potential hazard areas through the fused image, improving work efficiency and accuracy. When monitoring quantities include location, area size, deformation amount, and deformation rate, the location can be directly determined from the coordinates on the panoramic visual image, and the area size can also be estimated from the coordinates on the panoramic visual image. If it is necessary to display deformation amount and deformation rate simultaneously, they must be displayed alternately on the panoramic visual image.

[0054] like Figure 2 As shown, in conjunction with embodiments of the present invention, a method for monitoring full-domain three-dimensional deformation remote sensing radar is provided, comprising:

[0055] S201: Acquire multiple aperture radar images of the scene to be monitored generated sequentially by the radar in the current period, and construct a radar panoramic image of the scene to be monitored from the multiple aperture radar images in the current period.

[0056] S202: Acquire multiple optical photos of the scene to be monitored taken sequentially by an optical camera located in the scene to be monitored in the current period, and construct a visual panoramic image of the scene to be monitored from the multiple optical photos in the current period. The visual panoramic image and the radar panoramic image involve the same range of the scene to be monitored.

[0057] S203: Based on the change in the monitoring quantity of the radar panoramic image in the current period and the radar panoramic image in the previous period, obtain the radar monitoring quantity image of the scene to be monitored.

[0058] S204: Perform multiple location matching and marking between the radar monitoring image of the current period and the visual panoramic image;

[0059] S205: When the operator views an image of the scene to be monitored through an AR device, the AR device acquires the real-time pitch and azimuth angles of the operator's head movement, as well as the operator's field of view. Based on the real-time pitch angle, azimuth angle, and field of view, the radar monitoring image and visual panoramic image within the field of view are determined. The radar monitoring image and visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices, and the radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image. The fused image is used to display to the operator in real time.

[0060] Preferably, in S202, the process of sequentially capturing multiple optical images of the scene to be monitored by an optical camera located at the scene to be monitored in the current cycle includes:

[0061] The two-dimensional turntable of the photoelectric device located at the scene to be monitored rotates, causing the optical camera to rotate and tilt. The step angle of the photoelectric camera ensures that there is a set range of overlap between adjacent optical photos. During each step of the photoelectric camera, an optical photo of the scene to be monitored is taken by the optical camera, ultimately achieving a 360°×90° coverage of the scene to be monitored and obtaining multiple optical photos.

[0062] Preferably, the full-domain three-dimensional deformation remote sensing radar monitoring method further includes:

[0063] S206: At the beginning of each cycle, the position of the optical camera and the position of the radar are set to be at the same point and aligned at zero angle. The step angle of the rotation of the optical camera is set to be the same as the step angle of the rotation of the radar, so that the position of the optical photo obtained each time is the same as the position of the scene to be monitored corresponding to the aperture radar image.

[0064] Preferably, S203: Based on the change in monitoring quantity between the radar panoramic image of the current period and the radar panoramic image of the previous period, a radar monitoring quantity image of the scene to be monitored is obtained, including:

[0065] When the monitored quantity is deformation, a stable point is selected from the radar panoramic image of the current period and the radar panoramic image of the previous period.

[0066] The interferometric synthetic aperture radar method is used to perform phase interference between the current period image and the previous period image at each stable point to obtain the interferometric phase difference. The interferometric phase difference is then inverted into the deformation in the line-of-sight direction, and the accuracy of the deformation is less than or equal to 0.1 mm.

[0067] At the stable point of the radar panoramic image, the cumulative deformation in the line-of-sight direction corresponding to the stable point is used to construct the radar deformation field image in the direction of pitch angle and azimuth angle. The cumulative deformation refers to the sum of the cumulative deformation of all two adjacent cycles from the start of monitoring to the current cycle.

[0068] When the monitored quantity is the deformation rate, after obtaining the deformation quantity, the deformation rate is calculated based on the deformation quantity, and a deformation rate field image is obtained.

[0069] Preferably, the radar deformation field images in the elevation and azimuth directions are constructed using the deformation along the line-of-sight direction corresponding to the stable point, including:

[0070] The upper and lower limits of the deformation of the scene to be monitored are set, and the deformation of the radar resolution unit in the line-of-sight direction is represented by color scheme on the radar panoramic image; wherein, the representation of the deformation of the radar resolution unit in the line-of-sight direction by color scheme on the radar panoramic image specifically includes:

[0071] On the radar panoramic image, the stable points of intangible variables are represented by a first color, the stable points of negative deformation variables are represented by a second color, and the changes in the magnitude of negative deformation variables are represented by a gradient of the second color. The stable points of positive deformation variables are represented by a third color, and the changes in the magnitude of positive deformation variables are represented by a gradient of the third color. The unstable points are made completely transparent, thus constructing a radar deformation field image.

[0072] Preferably, S204: performing multiple location matching and marking operations between the radar monitoring image of the current period and the visual panoramic image, including:

[0073] A radar resolution unit is set for the radar monitoring image and the corresponding serial number is marked. The visual panoramic image is divided into multiple adjacent optical image grids according to the radar imaging resolution unit, and a corresponding number is set for each optical image grid. The number of each optical image grid is associated with the number of the corresponding radar resolution unit. The radar resolution is represented by the horizontal rotation angle.

[0074] Preferably, S205: The radar monitoring image and the visual panoramic image within the field of view are respectively cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image, including:

[0075] The radar monitoring images within the field of view are cropped according to radar resolution units to form radar monitoring slices, and the visual panoramic images within the field of view are cropped according to optical image grids to form visual image slices.

[0076] Based on the correspondence between radar resolution unit numbers and optical image grid numbers, radar monitoring slices are superimposed on the corresponding optical image grids. The alpha channel in RGBA is set to adjust the transparency of the radar monitoring slice colors, forming a fused image.

[0077] The technical solutions of the present invention will be described in detail below with reference to specific application examples. For technical details not described in the implementation process, please refer to the relevant descriptions above.

[0078] This invention presents a full-domain three-dimensional deformation remote sensing radar monitoring method for monitoring scenarios such as engineering inspections (construction monitoring or emergency monitoring), including structures like large foundation pits, tunnels, and buildings. It addresses the problems of existing technologies, such as the significant environmental impact on engineering monitoring, difficulty in accurately obtaining real-time monitoring data (e.g., deformation information), and insufficient visualization. This method improves the accuracy and efficiency of engineering monitoring, providing strong support for the analysis and handling of engineering disasters. It solves the pixel-level correspondence problem caused by low radar resolution, simplifying algorithm design and radar deployment. The three-dimensional scanning radar has wide angular coverage, three-dimensional modeling capabilities, high deformation measurement accuracy unaffected by harsh environments, and integrated visualization facilitates operator location of potential hazard areas. This method is of great significance for engineering disaster analysis and handling and the promotion of AR technology.

[0079] like Figure 3 As shown, the global three-dimensional deformation remote sensing radar monitoring method of this invention includes the following steps:

[0080] I. Optical Panoramic Image Generation: Optical cameras using a two-dimensional optical rotating gimbal capture images, with step angles ensuring sufficient overlap between adjacent optical photographs (e.g., 2 / 3 overlap). Then, open-source software or SDKs (such as Hugin) are used to generate a radar panoramic image of the hemispherical airspace of the scene to be monitored. This shooting method and the setting of the overlap area ensure the integrity and accuracy of the panoramic image of the monitored scene, while the use of open-source software or SDKs improves the efficiency and quality of panoramic image generation.

[0081] II. Formation of Radar Deformation Field

[0082] 1. Rotate the 2D radar pan-tilt unit (i.e., 2D radar turntable) to cover 360°×90°, that is, rotate 360° horizontally and 90° in pitch, to generate a panoramic radar image of the hemispherical airspace of the scene to be monitored. For example, if the scene to be monitored is a tunnel, the panoramic radar image is an image of the entire tunnel captured by the radar.

[0083] 2. Methods such as Amplitude Difference (ADI) can be used to screen stable points (PS) in radar panoramic images. A stable point is an observation target that exhibits stable backscattering characteristics after radar wave emission; only stable points can be used for subsequent deformation extraction. The screened PS points are then subjected to phase interferometry using the PS Interferometric Synthetic Aperture Radar (PS-InSAR) method, and the resulting deformation is inverted into the deformation along the line of sight. Specifically, the interferometric method involves subtracting the phase of each pixel in two consecutive images to obtain the interferometric phase difference.

[0084] 3. The deformation corresponding to point PS is used to construct a two-dimensional image in the elevation and azimuth angle directions, which is the radar deformation field image.

[0085] Specifically, upper and lower limits of the deformation field (d) are set. min d max Each pixel (or radar resolution unit) is represented using RGBA, for example, using the Jet color scheme. min d is represented in blue. max Using red to represent pixels with no effective backscattering, pixels are made fully transparent, thus processing the deformation field corresponding to the radar deformation field into a radar deformation field image. That is, the deformation image uses color to represent the magnitude of deformation based on the given deformation value; for example, red represents negative deformation, blue represents positive deformation, and green represents almost no deformation. This preprocessing method clearly displays the information of the radar deformation field, facilitating subsequent fusion processing. RGBA is a color space representing Red, Green, Blue, and Alpha. The jet color mapping in Matplotlib's data visualization tool has been widely discussed due to controversies in science communication.

[0086] III. Creating a 3D Radar Model

[0087] The radar reflection points have distance (due to the high-precision absolute distance measurement function), elevation angle, and azimuth angle information, which can generate a three-dimensional spatial point cloud; the absolute distance measurement of a point in the scene to be monitored obtained by radar measurement helps to determine the position of the aperture radar image.

[0088] Preprocessing point clouds, such as outlier removal, point cloud filtering, and surface extraction, can yield a 3D scene model, which can then be used to determine the nature of the scene.

[0089] Fourth, the two-dimensional turntable of the optoelectronic equipment rotates to obtain a visual panoramic image of the hemispherical airspace with the same coverage area as the radar;

[0090] The photoelectric turntable also rotates in two dimensions, pitch and azimuth, to generate images at certain step angle intervals (one photo is taken every time an angle is passed), and uses panoramic stitching software or tools to generate a hemispherical aerial panoramic visual image.

[0091] V. Image Grid Division and Numbering Correspondence: The visual panoramic image is divided into optical image grids according to the radar imaging resolution. The numbers of the optical image grids are then mapped to the numbers of the radar resolution units. For example, a radar resolution unit corresponds to a radar resolution of 0.9°. This correspondence allows for precise matching of radar deformation field information with the visual panoramic image.

[0092] VI. AR Application Display

[0093] When used in AR applications, the AR device acquires real-time pitch and azimuth angle information of the operator's or viewer's head movements. Based on the pitch, azimuth, and field-of-view information, the radar deformation field and hemispherical panoramic visual images are cropped to form radar deformation field image slices and visual image slices. The radar image slices are then proportionally expanded or interpolated to the same size and overlaid (overlay display combines a semi-transparent radar deformation field image with a photograph). A fused image of the real-time viewpoint is output, and the fused image within the operator's or viewer's field of view is displayed. This allows the operator or viewer to more intuitively observe the deformation of the surrounding environment through the AR device, improving the convenience and accuracy of monitoring.

[0094] Specifically, image fusion refers to superimposing the color of each radar resolution unit in the radar deformation field image onto the corresponding optical image grid, and adjusting the transparency of the radar deformation field image by setting the Alpha channel in RGBA to form a fused image, making both the deformation field and the optical image visible simultaneously. This fusion method preserves the real scene information of the radar panoramic image while clearly displaying the radar deformation field information, facilitating observation and analysis by operators.

[0095] The beneficial effects achieved by the embodiments of the present invention are as follows:

[0096] 1. Considering the low resolution (beamwidth approximately 2.5-3°) of 3D scanning deformation measurement radar, an algorithm was designed to solve the problem of not being able to obtain accurate pixel-level correspondences. Through reasonable image grid division and numbering correspondence, precise matching of radar deformation field information and panoramic images was achieved, improving the accuracy of monitoring.

[0097] 2. Assuming the radar and optical camera are located at the same point and aligned at zero angle, the algorithm design and radar deployment are simplified within a certain error range, with minimal impact on the actual designation of potential hazard areas. This simplified design reduces system complexity and cost, while improving system practicality and operability.

[0098] 3. The 3D scanning radar has an angular coverage of 360°×90° hemispherical airspace, possessing 3D scene modeling capabilities. It can obtain real-time deformation area information, accurately providing indicators such as the location, size, deformation amount (color), and deformation rate (color, alternating) of the deformation area. This plays a crucial role in the analysis and handling of engineering disasters. Operators can take timely measures based on this information to prevent the occurrence or escalation of disasters.

[0099] 4. The deformation measurement accuracy is better than 0.1mm, and it is unaffected by rain, snow, fog, smoke, dust, fire, etc., making it applicable to the construction and emergency monitoring of structural stability in large foundation pits, tunnels, buildings, etc. This allows the invention to function normally in various harsh environments, giving it broad application prospects.

[0100] 5. The fusion and visualization of panoramic and deformation field images greatly facilitates the rapid location of potential hazard areas by on-site operators, and is also of great significance for the subsequent research and development and promotion of augmented reality (AR) technology. Operators can intuitively see the location and deformation of potential hazard areas through the fused images, improving work efficiency and accuracy.

[0101] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0102] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0103] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0104] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0105] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0106] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0107] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0108] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0109] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A global three-dimensional deformation remote sensing radar monitoring system, characterized in that, include: The first image acquisition unit is used to acquire multiple aperture radar images of the scene to be monitored generated sequentially by the radar in the current period, and to construct a radar panoramic image of the scene to be monitored by the multiple aperture radar images in the current period. The second image acquisition unit is used to acquire multiple optical photos of the scene to be monitored taken sequentially by an optical camera located in the scene to be monitored in the current period, and to construct a visual panoramic image of the scene to be monitored by the multiple optical photos in the current period. The visual panoramic image and the radar panoramic image involve the same range of the scene to be monitored. The radar monitoring quantity image construction unit is used to obtain the radar monitoring quantity image of the scene to be monitored based on the change in monitoring quantity between the radar panoramic image of the current period and the radar panoramic image of the previous period. An image labeling unit is used to match and label multiple locations between the radar monitoring image of the current period and the visual panoramic image. The image fusion construction unit is used to obtain the real-time pitch and azimuth angles of the operator's head movement, as well as the operator's field of view, through the AR device when the operator is viewing the image of the scene to be monitored via the AR device. as well as Based on the operator's real-time head movement, pitch angle, azimuth angle, and field of view range, determine the radar monitoring image and visual panoramic image within the field of view range. The radar monitoring image and the visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image. The fused image is used to display to the operator in real time.

2. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 1, characterized in that, The second image acquisition unit is specifically used for: The two-dimensional turntable of the optoelectronic device located at the scene to be monitored rotates, causing the optical camera to rotate and tilt. The step angle of the optical camera ensures that there is a set range of overlap between adjacent optical photos. During each step of the optical camera, an optical photo of the scene to be monitored is taken, ultimately achieving a 360°×90° coverage of the scene to be monitored and obtaining multiple optical photos.

3. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 1, characterized in that, Also includes: An alignment unit is used to set the position of the optical camera and the position of the radar to be at the same point and aligned at zero angle at the beginning of each cycle, and to set the step angle of the rotation of the optical camera to be the same as the step angle of the rotation of the radar, so that the position of the optical image obtained each time is the same as the position of the scene to be monitored corresponding to the aperture radar image.

4. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 1, characterized in that, The radar monitoring image construction unit includes: The first monitoring quantity construction subunit is used to filter out stable points from the radar panoramic image of the current period and the radar panoramic image of the previous period when the monitoring quantity is deformation. The interferometric synthetic aperture radar method is used to perform phase interference between the current period image and the previous period image at each stable point to obtain the interferometric phase difference. The interferometric phase difference is then inverted into the deformation in the line-of-sight direction, and the accuracy of the deformation is less than or equal to 0.1 mm. At the stable point of the radar panoramic image, the cumulative deformation in the line-of-sight direction corresponding to the stable point is used to construct the radar deformation field image in the direction of pitch angle and azimuth angle. The cumulative deformation refers to the sum of the cumulative deformation of the stable point from the start of monitoring to the current cycle. The second monitoring quantity construction subunit is used to obtain the deformation rate after obtaining the deformation quantity when the monitoring quantity is the deformation rate, and to obtain the deformation rate field image.

5. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 4, characterized in that, The first monitoring quantity construction subunit is specifically used for: The upper and lower limits of the deformation of the scene to be monitored are set, and the cumulative deformation of the radar resolution unit in the line-of-sight direction is represented by color scheme on the radar panoramic image; wherein, the representation of the deformation of the radar resolution unit in the line-of-sight direction by color scheme on the radar panoramic image specifically includes: On the radar panoramic image, the stable points of intangible variables are represented by a first color, the stable points of negative deformation variables are represented by a second color, and the changes in the magnitude of negative deformation variables are represented by a gradient of the second color. The stable points of positive deformation variables are represented by a third color, and the changes in the magnitude of positive deformation variables are represented by a gradient of the third color. The unstable points are made completely transparent, thus constructing a radar deformation field image.

6. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 1, characterized in that, The image labeling unit is specifically used for: A radar resolution unit is set for the radar monitoring image and the corresponding serial number is marked. The visual panoramic image is divided into multiple adjacent optical image grids according to the radar imaging resolution unit, and a corresponding number is set for each optical image grid. The number of each optical image grid is associated with the number of the corresponding radar resolution unit. The radar resolution is represented by the horizontal rotation angle.

7. The full-domain three-dimensional deformation remote sensing radar monitoring system according to claim 6, characterized in that, The fused image construction unit is specifically used for: The radar monitoring images within the field of view are cropped according to radar resolution units to form radar monitoring slices, and the visual panoramic images within the field of view are cropped according to optical image grids to form visual image slices. Based on the correspondence between radar resolution unit numbers and optical image grid numbers, radar monitoring slices are superimposed on the corresponding optical image grids. The alpha channel in RGBA is set to adjust the transparency of the radar monitoring slice colors, forming a fused image.

8. A method for monitoring three-dimensional deformation using a global remote sensing radar, characterized in that, include: Acquire multiple aperture radar images of the scene to be monitored in the current period generated sequentially by the radar located in the scene to be monitored, and construct a radar panoramic image of the scene to be monitored by the multiple aperture radar images in the current period; Multiple optical photos of the scene to be monitored are acquired sequentially by an optical camera located in the scene to be monitored in the current period. The multiple optical photos in the current period are used to construct a visual panoramic image of the scene to be monitored. The visual panoramic image covers the same range of the scene to be monitored as the radar panoramic image. Based on the change in the monitoring quantity of the radar panoramic image in the current period and the radar panoramic image in the previous period, a radar monitoring quantity image of the scene to be monitored is obtained. The radar monitoring image of the current period is matched and marked with multiple locations in the visual panoramic image; When the operator views an image of the scene to be monitored through an AR device, the AR device acquires the operator's real-time pitch and azimuth angles of head movement, as well as the operator's field of view. Based on the operator's real-time head movement, pitch angle, azimuth angle, and field of view range, determine the radar monitoring image and visual panoramic image within the field of view range. The radar monitoring image and the visual panoramic image within the field of view are cropped to form radar monitoring image slices and visual image slices. The radar monitoring image slices are superimposed on the visual image slices at corresponding positions to obtain a fused image. The fused image is used to display to the operator in real time.

9. The method for monitoring full-domain three-dimensional deformation remote sensing radar according to claim 8, characterized in that, The process of the optical camera located in the scene to be monitored taking multiple optical photos of the scene to be monitored sequentially in the current period includes: The two-dimensional turntable of the optoelectronic device located at the scene to be monitored rotates, causing the optical camera to rotate and tilt. The step angle of the optical camera ensures that there is a set range of overlap between adjacent optical photos. During each step of the optical camera, an optical photo of the scene to be monitored is taken, ultimately achieving a 360°×90° coverage of the scene to be monitored and obtaining multiple optical photos.

10. The method for monitoring full-domain three-dimensional deformation remote sensing radar according to claim 8, characterized in that, Also includes: At the beginning of each cycle, the position of the optical camera and the position of the radar are set to be at the same point and aligned at zero angle. The rotation step angle of the optical camera is set to be the same as the rotation step angle of the radar, so that the position of the optical image captured each time is the same as the position of the scene to be monitored corresponding to the aperture radar image.

Images