A method for monitoring three-dimensional structural deformation based on machine vision and laser ranging

Abstract

This invention relates to a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging, belonging to the field of engineering structural deformation monitoring technology. A machine vision subsystem is used to identify and locate the target in two dimensions, while a laser ranging subsystem acquires precise radial distance information. An intelligent galvanometer control system enables vision-guided precise laser pointing and automatic multi-point switching. Finally, a multi-sensor data fusion algorithm integrates two-dimensional image displacement and one-dimensional laser ranging information into complete three-dimensional displacement data. This invention solves the technical problems of insufficient accuracy, incomplete dimensions, and poor environmental adaptability in existing long-distance three-dimensional displacement measurements. It has advantages such as high measurement accuracy, strong environmental adaptability, and high degree of automation, and is suitable for monitoring structural deformation of large infrastructure such as hydraulic structures, slopes, bridges, and tunnels.

Description

Technical Field

[0001] This application relates to the field of engineering structure deformation monitoring technology, and in particular to a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging. Background Technology

[0002] Structural health monitoring of large-scale infrastructure is of great significance for ensuring public safety. Currently, the main monitoring technologies include contact sensors, such as strain gauges, displacement sensors, and accelerometers, as well as non-contact measurement technologies, such as GPS, radar, laser interferometers, and purely visual systems.

[0003] These existing technologies have obvious limitations: the contradiction between measurement accuracy and distance is prominent, and the accuracy drops sharply when measuring at long distances; most methods can only obtain one-dimensional or two-dimensional displacement information and lack complete three-dimensional displacement data; they have poor environmental adaptability and are greatly affected by factors such as temperature, vibration and weather; the system calibration is complicated and requires professional personnel to operate; the real-time performance and multi-point synchronous measurement capabilities are insufficient; and the cost-effectiveness is low, with a large number of sensors required for multi-point monitoring.

[0004] Therefore, there is an urgent need for an advanced monitoring technology that can achieve long-distance millimeter-level precision measurement, acquire complete three-dimensional displacement information, have multi-point synchronous monitoring capabilities, strong environmental adaptability, and support real-time monitoring and intelligent early warning. Summary of the Invention

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] According to a first aspect of the present invention, the present invention claims protection for a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging, comprising the following steps:

[0007] S1, System initialization and calibration: Joint calibration is performed through the machine vision subsystem and the laser ranging subsystem to establish the transformation relationship between the visual coordinate system and the laser coordinate system, and to set the reference position of the target and the initial measurement benchmark.

[0008] S2, the machine vision subsystem acquires a two-dimensional image of the target, a deep learning-based target detection algorithm is used to automatically identify and locate the target, and a sub-pixel positioning algorithm is used to improve the two-dimensional positioning accuracy;

[0009] S3, Visual-guided laser ranging: Based on the two-dimensional image coordinates of the target, the pointing angle of the laser beam is calculated by the intelligent galvanometer control system, and the laser ranging subsystem is controlled to point to each target in sequence, and the radial displacement distance of the target is obtained.

[0010] S4, calculate the three-dimensional displacement by fusing the displacement of the target's two-dimensional image with the radial displacement distance obtained by laser ranging, and calculate the preliminary three-dimensional displacement of the target based on the coordinate transformation model, including horizontal displacement, vertical displacement and radial displacement.

[0011] S5, perform anomaly detection, smoothing and reliability assessment on the preliminary three-dimensional displacement, and output the final three-dimensional deformation monitoring result through a multi-sensor data fusion algorithm.

[0012] Furthermore, step S2 also includes:

[0013] An adaptive target detection algorithm based on improved YOLOv5 is adopted. The network structure includes a CSPDarknet53 backbone network with added attention mechanism, a PANet neck network with enhanced feature fusion, and a multi-scale detection head.

[0014] Data augmentation strategies and custom loss functions are used to optimize the model to adapt to target recognition under different distance, lighting and occlusion conditions.

[0015] Furthermore, step S2 also includes:

[0016] The subpixel positioning algorithm employs a multi-algorithm fusion strategy, including grayscale centroid method, parabolic surface fitting method, and template matching method. It uses normalized cross-correlation coefficients for matching verification to achieve positioning accuracy below the pixel level.

[0017] Furthermore, step S3 also includes:

[0018] Intelligent galvanometer control technology is adopted, including measurement path optimization based on the traveling salesman problem algorithm to reduce measurement time, PID control algorithm and feedforward compensation to achieve precise positioning of the galvanometer, and input shaping technology to suppress residual vibration of the galvanometer.

[0019] Furthermore, step S4 also includes:

[0020] The coordinate transformation model includes distortion correction, intrinsic parameter transformation, extrinsic parameter transformation, and temperature compensation;

[0021] The extrinsic parameter transformation converts the two-dimensional displacement in the visual coordinate system into a three-dimensional displacement in the world coordinate system through a rotation matrix and translation vector obtained by joint calibration.

[0022] Furthermore, step S5 also includes:

[0023] Data fusion is performed using weighted least squares and robust estimation algorithms. The weights are dynamically adjusted based on the reliability of the measurement data and environmental conditions, and Kalman filtering is used to smooth out outlier data.

[0024] Furthermore, the method also includes an environment adaptation step:

[0025] Adapt to changes in lighting conditions through multispectral imaging, automatic exposure control, and image enhancement algorithms;

[0026] The lens heating system and raindrop elimination algorithm are used to cope with severe weather.

[0027] The effects of temperature and vibration are suppressed by using a thermal drift compensation algorithm and a three-dimensional vibration isolation platform.

[0028] Furthermore, the method supports multi-point synchronous monitoring, and realizes automatic switching measurement of multiple targets through intelligent galvanometer control system. The measurement path is dynamically planned based on the target position. A single system can monitor 8-16 measurement points at the same time, and the measurement frequency is 1-10Hz.

[0029] Furthermore, the PID control algorithm, combined with feedback from a high-precision photoelectric encoder, is expressed by the following control formula: Where u(t) represents the control output signal, which is the driving voltage or current of the galvanometer; K_p is the proportional gain, which controls the response speed to the current error; e(t) represents the current error, which is the difference between the target angle and the actual angle; K_i is the integral gain, which eliminates steady-state error; ∫e(t)dt is the time integral of the error; K_d is the differential gain, which predicts future errors; de(t) / dt is the differential term of the rate of change of the error. The control accuracy of the galvanometer in dynamic environments is improved by real-time system identification and adaptive adjustment.

[0030] Furthermore, step S1 also includes:

[0031] GPS timing and hardware triggering are used to achieve spatiotemporal synchronization between machine vision and laser ranging, with a time synchronization accuracy of ±100μs. Precision geometric calibration ensures that the spatial alignment accuracy is within ±0.1mm@100m.

[0032] The beneficial technical effects achieved by this invention include:

[0033] (1) Achieved long-distance, high-precision three-dimensional displacement measurement: By integrating two-dimensional sub-pixel positioning of machine vision with one-dimensional precise ranging of laser ranging, a three-dimensional displacement measurement accuracy of millimeters at a distance of hundreds of meters was achieved, solving the problem of insufficient long-distance measurement accuracy of traditional methods;

[0034] (2) It provides complete three-dimensional deformation information: It overcomes the limitation of most existing technologies that can only obtain one-dimensional or two-dimensional displacement, and can simultaneously and from the same source measure the horizontal, vertical and radial line-of-sight displacement of the target, providing comprehensive three-dimensional deformation data for structural safety assessment;

[0035] (3) Strong environmental adaptability: Through comprehensive measures such as multispectral imaging, automatic exposure, image enhancement, lens heating, raindrop elimination, thermal drift compensation and three-dimensional vibration isolation, it can effectively adapt to complex working conditions such as light changes, rain and fog weather, temperature fluctuations and environmental vibrations, and ensure the long-term stable operation of the system in the field.

[0036] (4) Achieved high degree of automation and intelligence: Based on the improved deep learning algorithm, the target is automatically identified and located without human intervention; the laser is automatically, quickly and accurately pointed under visual guidance through the intelligent galvanometer control system, which greatly improves the measurement efficiency and automation level;

[0037] (5) Accurate spatiotemporal synchronization and system calibration were achieved: GPS timing and hardware triggering ensured the time synchronization accuracy of visual and laser data acquisition; accurate coordinate transformation relationship was established through precise joint calibration, ensuring spatial alignment accuracy and laying the foundation for data fusion.

[0038] (6) Improved dynamic response and control performance of the system: The advanced PID control algorithm combining feedforward compensation and input shaping technology is adopted to achieve fast and accurate positioning of the galvanometer and suppress residual vibration, ensuring the ability to capture dynamic deformation such as wind-induced vibration at high measurement frequencies.

[0039] (7) Reduced system deployment and maintenance complexity and cost: Non-contact measurement is used, eliminating the need to lay a large number of cables on the structure under test; a single device can cover multiple measurement points, which reduces the overall cost and simplifies installation and maintenance compared to deploying a large number of individual sensors. Attached Figure Description

[0040] Figure 1 This invention seeks to protect the flowchart of a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0042] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0043] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0044] According to a first embodiment of the present invention, the present invention claims protection for a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging, referring to... Figure 1 This includes the following steps:

[0045] S1, System initialization and calibration: Joint calibration is performed through the machine vision subsystem and the laser ranging subsystem to establish the transformation relationship between the visual coordinate system and the laser coordinate system, and to set the reference position of the target and the initial measurement benchmark.

[0046] S2, the machine vision subsystem acquires a two-dimensional image of the target, a deep learning-based target detection algorithm is used to automatically identify and locate the target, and a sub-pixel positioning algorithm is used to improve the two-dimensional positioning accuracy;

[0047] S3, Visual-guided laser ranging: Based on the two-dimensional image coordinates of the target, the pointing angle of the laser beam is calculated by the intelligent galvanometer control system, and the laser ranging subsystem is controlled to point to each target in sequence, and the radial displacement distance of the target is obtained.

[0048] S4, calculate the three-dimensional displacement by fusing the displacement of the target's two-dimensional image with the radial displacement distance obtained by laser ranging, and calculate the preliminary three-dimensional displacement of the target based on the coordinate transformation model, including horizontal displacement, vertical displacement and radial displacement.

[0049] S5, perform anomaly detection, smoothing and reliability assessment on the preliminary three-dimensional displacement, and output the final three-dimensional deformation monitoring result through a multi-sensor data fusion algorithm.

[0050] In this embodiment, a machine vision system is used for two-dimensional displacement measurement, a laser ranging system for radial distance measurement, and an intelligent galvanometer system for a collaborative working mode with multi-point switching. The system achieves high-precision three-dimensional displacement measurement through innovative vision-guided laser ranging technology, multi-sensor spatiotemporal synchronization technology, adaptive environmental algorithms, and intelligent galvanometer control technology.

[0051] The machine vision subsystem is employed, including:

[0052] Optical imaging unit: includes a high-precision fixed-focus lens (focal length 5-120mm optional), image sensor (12-megapixel full-frame CMOS), filter system (visible light + near-infrared filter), and lens heating system;

[0053] Image processing unit: Embedded AI processor (computing power ≥ 8 TOPS), image preprocessing module, deep learning inference engine, sub-pixel localization algorithm;

[0054] Control and communication unit: embedded host controller (ARM Cortex-A78), network communication module (gigabit Ethernet + 4G / 5G), storage unit, power management.

[0055] A laser ranging subsystem is employed, including:

[0056] Laser ranging unit: 1550nm eye-safe laser, phase method ranging, ranging range 10-400 meters, accuracy ±2mm@400m;

[0057] Two-dimensional galvanometer system: electromagnetic fast-swing mirror, deflection range ±30° horizontally and ±30° vertically, positioning accuracy ≤5 arcseconds;

[0058] Galvanometer control system: PID + feedforward control algorithm, high-precision photoelectric encoder feedback.

[0059] The main control system adopts a distributed control architecture and uses the LinuxRT real-time operating system.

[0060] The calibration and excitation system performs geometric calibration, laser calibration, combined calibration, and online calibration.

[0061] The environmental adaptation system includes an IP67 protection rating, an active temperature control system, and a three-dimensional vibration isolation platform.

[0062] For target systems, including

[0063] Target design: 3M retroreflective film material, circular or cross-shaped design, 500mm in diameter at 400m;

[0064] Installation system: magnetic, adhesive or bolt fixing, three-dimensional fine adjustment device.

[0065] The three-dimensional measurement of this invention is based on the fusion measurement principle of "two-dimensional visual displacement + one-dimensional laser distance":

[0066] Horizontal displacement measurement: ΔX = (Δpx × D × Psize) / f;

[0067] Vertical displacement measurement: ΔY = (Δpy × D × Psize) / f;

[0068] Radial displacement measurement: ΔZ = D(t) - D(0);

[0069] Where Δpx and Δpy are pixel displacements, D is the target distance, Psize is the pixel physical size, and f is the lens focal length; where ΔZ is the radial displacement, the physical displacement of the target in the Z direction, in mm; D(t) is the ranging distance at the current time t, with an accuracy of ±2mm@400m; D(0) is the ranging distance reference value at the initial reference time.

[0070] The machine vision system identifies the target position in real time, calculates the angle at which the laser beam should point using intelligent algorithms, and controls the galvanometer to precisely point at the target. This innovative technology achieves true three-dimensional measurement, overcoming the limitations of a single sensor.

[0071] By employing GPS timing, hardware triggering, and software compensation, a time synchronization accuracy of ±100μs is achieved; and by establishing coordinate transformation relationships through precise calibration, a spatial alignment accuracy of ±0.1mm@100m is achieved.

[0072] Deep learning-based environment adaptation algorithms include illumination adaptation (automatic exposure control + HDR imaging + dynamic range extension), weather adaptation (multispectral imaging + image enhancement + atmospheric attenuation compensation), and vibration suppression (active vibration isolation + digital filtering + motion compensation).

[0073] Intelligent galvanometer control based on trajectory optimization uses the TSP algorithm to optimize the measurement path, reducing measurement time by 30%; PID + feedforward control achieves a positioning accuracy of 5 arcseconds; and input shaping technology eliminates residual vibration.

[0074] Perform system initialization, including:

[0075] Camera intrinsic parameter calibration: The camera intrinsic parameter matrix and lens distortion coefficients were calibrated using Zhang Zhengyou's calibration method;

[0076] Laser ranging calibration: Multi-point calibration is performed using a standard ranging block to establish a temperature compensation model;

[0077] Joint calibration: Establish the transformation relationship between the visual coordinate system and the laser coordinate system: P_laser = R_vl × P_vision + T_vl;

[0078] Where P_laser is the point position in the laser coordinate system, which is a three-dimensional vector; R_vl is a 3x3 rotation matrix, obtained by joint calibration); P_vision is the point position in the vision coordinate system, which is a two-dimensional or three-dimensional vector; T_vl is a 3x1 translation vector, obtained by joint calibration;

[0079] Set the target reference position and initial measurement baseline.

[0080] Furthermore, step S2 also includes:

[0081] An adaptive target detection algorithm based on improved YOLOv5 is adopted. The network structure includes a CSPDarknet53 backbone network with added attention mechanism, a PANet neck network with enhanced feature fusion, and a multi-scale detection head.

[0082] Data augmentation strategies and custom loss functions are used to optimize the model to adapt to target recognition under different distance, lighting and occlusion conditions.

[0083] In this embodiment,

[0084] Furthermore, step S2 also includes:

[0085] The subpixel positioning algorithm employs a multi-algorithm fusion strategy, including grayscale centroid method, parabolic surface fitting method, and template matching method. It uses normalized cross-correlation coefficients for matching verification to achieve positioning accuracy below the pixel level.

[0086] An adaptive target detection algorithm based on improved YOLOv5 is adopted:

[0087] Network structure optimization: The CSPDarknet53 backbone network adds an attention mechanism, the PANet neck network enhances feature fusion, and the three-scale detection head adapts to targets at different distances;

[0088] Data augmentation strategies: geometric transformation, color space adjustment, noise injection, occlusion simulation;

[0089] Loss function design: L_total = L_bbox + L_conf + L_class + L_center;

[0090] Where L_total represents the total loss value, used for backpropagation to optimize model parameters; L_bbox represents the bounding box loss, which uses GIoU Loss to calculate the overlap between the predicted box and the ground truth box; L_conf represents the confidence loss, which uses BinaryCross Entropy to calculate; L_class represents the class loss, which uses Focal Loss to handle class imbalance; and L_center represents the custom center point localization loss to improve localization accuracy.

[0091] Sub-pixel localization combining multiple algorithms:

[0092] Gray-scale centroid method: x_center = Σ(x_i×I) / Σ(I);

[0093] Where x_center is the x-coordinate of the target center with sub-pixel precision, x_i is the x-coordinate of the i-th pixel; I is the grayscale intensity of the pixel, ranging from 0 to 255; Σ is the summation of all pixels within the target area;

[0094] Parabolic fitting method: ;

[0095] Where I(x,y) is the pixel gray value at position (x,y); a,b,c are quadratic coefficients describing the curvature of the surface; d,e are linear coefficients describing the linear offset; and f is a constant term describing the baseline gray value.

[0096] Template matching method: using normalized cross-correlation coefficient (NCC).

[0097] Furthermore, step S3 also includes:

[0098] Intelligent galvanometer control technology is adopted, including measurement path optimization based on the traveling salesman problem algorithm to reduce measurement time, PID control algorithm and feedforward compensation to achieve precise positioning of the galvanometer, and input shaping technology to suppress residual vibration of the galvanometer.

[0099] In this embodiment, the corresponding galvanometer deflection angle is calculated based on the target image coordinates, and the galvanometer system is controlled to point sequentially at each target. Intelligent galvanometer control technology is employed.

[0100] Motion trajectory planning: The TSP algorithm optimizes the measurement path and minimizes the measurement time;

[0101] Vibration suppression algorithm: Input shaping technique u_shaped(t)=Σ(A_i×u(t-t_i));

[0102] Where u_shaped(t) represents the shaped control input signal; A_i is the amplitude weight of the i-th delayed signal; u(t - t_i) is the value of the original input signal after a delay time t_i; Σ represents the summation of multiple delay terms.

[0103] Furthermore, step S4 also includes:

[0104] The coordinate transformation model includes distortion correction, intrinsic parameter transformation, extrinsic parameter transformation, and temperature compensation;

[0105] The extrinsic parameter transformation converts the two-dimensional displacement in the visual coordinate system into a three-dimensional displacement in the world coordinate system through a rotation matrix and translation vector obtained by joint calibration.

[0106] Furthermore, step S5 also includes:

[0107] Data fusion is performed using weighted least squares and robust estimation algorithms. The weights are dynamically adjusted based on the reliability of the measurement data and environmental conditions, and Kalman filtering is used to smooth out outlier data.

[0108] Furthermore, the method also includes an environment adaptation step:

[0109] Adapt to changes in lighting conditions through multispectral imaging, automatic exposure control, and image enhancement algorithms;

[0110] The lens heating system and raindrop elimination algorithm are used to cope with severe weather.

[0111] The effects of temperature and vibration are suppressed by using a thermal drift compensation algorithm and a three-dimensional vibration isolation platform.

[0112] Furthermore, the method supports multi-point synchronous monitoring, and realizes automatic switching measurement of multiple targets through intelligent galvanometer control system. The measurement path is dynamically planned based on the target position. A single system can monitor 8-16 measurement points at the same time, and the measurement frequency is 1-10Hz.

[0113] Furthermore, the PID control algorithm, combined with feedback from a high-precision photoelectric encoder, is expressed by the following control formula: Where u(t) represents the control output signal, which is the driving voltage or current of the galvanometer; K_p is the proportional gain, which controls the response speed to the current error; e(t) represents the current error, which is the difference between the target angle and the actual angle; K_i is the integral gain, which eliminates steady-state error; ∫e(t)dt is the time integral of the error; K_d is the differential gain, which predicts future errors; de(t) / dt is the differential term of the rate of change of the error. The control accuracy of the galvanometer in dynamic environments is improved by real-time system identification and adaptive adjustment.

[0114] Furthermore, step S1 also includes:

[0115] GPS timing and hardware triggering are used to achieve spatiotemporal synchronization between machine vision and laser ranging, with a time synchronization accuracy of ±100μs. Precision geometric calibration ensures that the spatial alignment accuracy is within ±0.1mm@100m.

[0116] The present invention will be described below with reference to specific embodiments:

[0117] Example 1: Bridge Health Monitoring

[0118] Application scenario: Displacement monitoring of the main girder of a long-span suspension bridge

[0119] System Configuration:

[0120] Measurement distance: 300 meters;

[0121] Lens option: 85mm prime lens;

[0122] Number of monitoring points: 12;

[0123] Measurement accuracy: ±1.5mm;

[0124] Sampling frequency: 5Hz.

[0125] Installation plan:

[0126] The monitoring equipment is installed on the platform at the top of the bridge pier;

[0127] The targets are installed below the main beam at intervals of 25 meters.

[0128] Data is transmitted to the monitoring center via fiber optic network.

[0129] Monitoring results:

[0130] Static displacement monitoring accuracy: ±1.2mm;

[0131] Dynamic response measurement accuracy: ±2.0mm;

[0132] System stability: After 180 days of continuous operation, availability is >99.5%.

[0133] Example 2: Dam Deformation Monitoring

[0134] Application scenario: Horizontal displacement monitoring of a gravity dam

[0135] System Configuration:

[0136] Measurement distance: 150 meters;

[0137] Lens option: 50mm prime lens;

[0138] Number of monitoring points: 8;

[0139] Measurement accuracy: ±1.0mm;

[0140] Sampling frequency: 1Hz.

[0141] Technical solution:

[0142] The equipment is installed on the observation station on the opposite bank;

[0143] Targets were deployed along the dam crest to cover key monitoring areas;

[0144] 24-hour continuous monitoring and real-time data analysis.

[0145] Monitoring results:

[0146] Temperature deformation monitoring: daily variation ±3.5mm;

[0147] Reservoir water level change response: ±2.1 mm / m water level;

[0148] Long-term stability: Annual drift < ±0.5 mm.

[0149] Example 3: Monitoring the Sway of High-Rise Buildings

[0150] Application scenario: Monitoring wind-induced sway at the top of a super high-rise building

[0151] System Configuration:

[0152] Measurement distance: 400 meters;

[0153] Lens option: 120mm prime lens;

[0154] Number of monitoring points: 4 (four corners of the building);

[0155] Measurement accuracy: ±2.0mm;

[0156] Sampling frequency: 10Hz.

[0157] Monitoring Plan:

[0158] The equipment is installed on the roof of an adjacent building;

[0159] The target is installed on the exterior wall of the target building;

[0160] The focus is on monitoring responses during periods of strong winds.

[0161] Application results:

[0162] Wind-induced vibration frequency: 0.15Hz;

[0163] Maximum oscillation: ±45mm (Force 10 wind);

[0164] Damping ratio: 2.3%.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0166] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0167] The specific embodiments of the invention have been described in detail above, but they are only examples, and this application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of this application. Therefore, all equivalent changes, modifications, and improvements made without departing from the spirit and principles of this application should be covered within the scope of this application.

Claims

1. A method for monitoring three-dimensional deformation of a structure based on machine vision and laser ranging, characterized in that, Includes the following steps: S1, System initialization and calibration: Joint calibration is performed through the machine vision subsystem and the laser ranging subsystem to establish the transformation relationship between the visual coordinate system and the laser coordinate system, and to set the reference position of the target and the initial measurement benchmark. S2, the machine vision subsystem acquires a two-dimensional image of the target, a deep learning-based target detection algorithm is used to automatically identify and locate the target, and a sub-pixel positioning algorithm is used to improve the two-dimensional positioning accuracy; S3, Visual-guided laser ranging: Based on the two-dimensional image coordinates of the target, the pointing angle of the laser beam is calculated by the intelligent galvanometer control system, and the laser ranging subsystem is controlled to point to each target in sequence, and the radial displacement distance of the target is obtained. S4, calculate the three-dimensional displacement by fusing the displacement of the target's two-dimensional image with the radial displacement distance obtained by laser ranging, and calculate the preliminary three-dimensional displacement of the target based on the coordinate transformation model, including horizontal displacement, vertical displacement and radial displacement. S5, perform anomaly detection, smoothing and reliability assessment on the preliminary three-dimensional displacement, and output the final three-dimensional deformation monitoring result through a multi-sensor data fusion algorithm. 2.The structural three-dimensional deformation monitoring method based on machine vision and laser ranging according to claim 1, characterized in that, Step S2 further includes: An adaptive target detection algorithm based on improved YOLOv5 is adopted. The network structure includes a CSPDarknet53 backbone network with added attention mechanism, a PANet neck network with enhanced feature fusion, and a multi-scale detection head. Data augmentation strategies and custom loss functions are used to optimize the model to adapt to target recognition under different distance, lighting and occlusion conditions.

3. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, Step S2 further includes: The subpixel positioning algorithm employs a multi-algorithm fusion strategy, including grayscale centroid method, parabolic surface fitting method, and template matching method. It uses normalized cross-correlation coefficients for matching verification to achieve positioning accuracy below the pixel level.

4. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, Step S3 further includes: Intelligent galvanometer control technology is adopted, including measurement path optimization based on the traveling salesman problem algorithm to reduce measurement time, PID control algorithm and feedforward compensation to achieve precise positioning of the galvanometer, and input shaping technology to suppress residual vibration of the galvanometer.

5. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, Step S4 further includes: The coordinate transformation model includes distortion correction, intrinsic parameter transformation, extrinsic parameter transformation, and temperature compensation; The extrinsic parameter transformation converts the two-dimensional displacement in the visual coordinate system into a three-dimensional displacement in the world coordinate system through a rotation matrix and translation vector obtained by joint calibration.

6. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, Step S5 further includes: Data fusion is performed using weighted least squares and robust estimation algorithms. The weights are dynamically adjusted based on the reliability of the measurement data and environmental conditions, and Kalman filtering is used to smooth out outlier data.

7. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, The method also includes an environment adaptation step: Adapt to changes in lighting conditions through multispectral imaging, automatic exposure control, and image enhancement algorithms; The lens heating system and raindrop elimination algorithm are used to cope with severe weather. The effects of temperature and vibration are suppressed by using a thermal drift compensation algorithm and a three-dimensional vibration isolation platform.

8. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, The method supports multi-point synchronous monitoring and realizes automatic switching measurement of multiple targets through intelligent galvanometer control system. The measurement path is dynamically planned based on the target position. A single system can monitor 8-16 measurement points at the same time, and the measurement frequency is 1-10Hz.

9. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 4, characterized in that, The PID control algorithm, combined with feedback from a high-precision photoelectric encoder, is expressed by the following control formula: Where u(t) represents the control output signal, which is the driving voltage or current of the galvanometer; K_p is the proportional gain, which controls the response speed to the current error; e(t) represents the current error, which is the difference between the target angle and the actual angle; K_i is the integral gain, which eliminates steady-state error; ∫e(t)dt is the time integral of the error; K_d is the differential gain, which predicts future errors; and de(t) / dt is the differential term of the rate of change of the error, which improves the control accuracy of the galvanometer in dynamic environments through real-time system identification and adaptive adjustment.

10. The method for monitoring three-dimensional structural deformation based on machine vision and laser ranging according to claim 1, characterized in that, Step S1 further includes: GPS timing and hardware triggering are used to achieve spatiotemporal synchronization between machine vision and laser ranging, with a time synchronization accuracy of ±100μs. Precision geometric calibration ensures that the spatial alignment accuracy is within ±0.1mm@100m.

Images