A method for measuring three-dimensional displacement of a structure using a single target

Abstract

This application relates to a method for measuring the three-dimensional displacement of a structure using a single target. The method outputs the three-dimensional displacement of the target in the gravity coordinate system by acquiring an image from the single target. The steps include: acquiring a target image using an imaging device; calculating the displacement of the target's corner pixels; detecting the pixel position of a laser spot in the current image; calculating the Euclidean distance between the center of the laser spot and the center of the target to determine the position of the laser spot until it is determined that the laser spot is aligned with the target center; emitting a laser and receiving the reflected signal using a laser displacement meter to measure the target distance; and combining the data obtained in the preceding steps with the device attitude data output by an inclinometer to synthesize a three-dimensional displacement vector in the gravity coordinate system. This application uses visual closed-loop feedback to replace reliance on the gimbal angle accuracy, thus eliminating dependence on the gimbal's angle accuracy.

Description

Technical Field

[0001] This application relates to a method for measuring three-dimensional displacement of a structure using a single target, which can be used in the technical field of building health monitoring. Background Technology

[0002] In building structural health monitoring, it is typically necessary to acquire the three-dimensional displacement of key measuring points in real time, that is, the displacement of key measuring points in three orthogonal directions, in order to determine whether the building structure has suffered damage or failure. Existing monitoring solutions include the following:

[0003] (1) Using contact sensors, such as dial gauges and displacement gauges, requires installing sensors and reference brackets on the structure being measured, which is difficult to set up. Furthermore, the reference point itself may shift, affecting the accuracy of the measurement. This method is not suitable for long-term unattended monitoring of existing buildings.

[0004] (2) Total station has high accuracy in automated monitoring, but the equipment is expensive, has strict requirements for line-of-sight conditions, and has a low sampling frequency, usually ranging from minutes to hours, making it difficult to capture dynamic displacement processes;

[0005] (3) Pure visual displacement monitoring, such as binocular vision and digital image correlation, can only acquire two-dimensional displacement in the plane and cannot directly measure displacement in the depth direction outside the plane; although binocular vision can recover three-dimensional information, the baseline length is limited and the depth accuracy at long distances drops sharply.

[0006] CN121655413A discloses a method for monitoring three-dimensional structural deformation based on machine vision and laser ranging. It uses a machine vision subsystem to identify and locate a target in two dimensions, a laser ranging subsystem to obtain precise radial distance information, and an intelligent galvanometer control system to achieve vision-guided laser pointing and automatic multi-point switching. Finally, a multi-sensor data fusion algorithm integrates two-dimensional image displacement and one-dimensional laser ranging information into complete three-dimensional displacement data. This patent uses a combination of "galvanometer + laser + vision" technology; however, the laser propagation through the galvanometer causes optical path divergence, affecting the accuracy of long-distance ranging. Furthermore, the galvanometer is relatively expensive.

[0007] CN118939007A discloses a gimbal control method and system for deformation monitoring. The method involves acquiring the relative displacement changes of each observation point within the monitoring range along the X and Z axes using a vision device on the gimbal; acquiring the distance to each observation point within the monitoring range using a laser rangefinder on the gimbal; calculating the horizontal and vertical angles of the gimbal corresponding to each observation point using trigonometric functions based on the acquired relative displacement changes and distances; and controlling the gimbal to rotate accordingly based on the calculated horizontal and vertical angles, aligning the laser rangefinder on the gimbal with the center of the target to achieve three-dimensional deformation measurement. This application employs visual measurement technology and laser ranging technology, combined with intelligent algorithms, to quickly and accurately acquire the horizontal and vertical angles of the gimbal. High-precision control of the gimbal is achieved based on these angles, thereby improving the accuracy and reliability of deformation measurement. This patent uses a combination of "gimbal + laser + vision" technology, with the operating logic being "calculate the angle → rotate." However, because the gimbal rotation angle is calculated using trigonometric functions and then executed in an open-loop manner, the execution accuracy of the gimbal must match the calculation accuracy. Furthermore, gear wear can easily lead to long-term cumulative errors, and there is no search and recovery mechanism; the coordinate system used is the device coordinate system, which cannot achieve automatic correction of coordinate values ​​to eliminate various errors and obtain more realistic and accurate results.

[0008] Therefore, how to achieve low-cost, long-term reliable, and automatically compensated three-dimensional displacement monitoring without relying on the accuracy of the gimbal angle is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] The purpose of this application is to provide a method for measuring the three-dimensional displacement of a structure using a single target, which replaces the dependence on the angle accuracy of the gimbal with visual closed-loop feedback, and does not depend on the angle accuracy of the gimbal.

[0010] This application relates to a method for measuring the three-dimensional displacement of a structure using a single target, which outputs the three-dimensional displacement of the target's gravity coordinate system by acquiring images from the single target, and includes the following steps:

[0011] (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, high threshold global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the spot.

[0012] (2) In the image output by the target extraction channel, the outermost maximum closed contour is tracked and locked within the target RoI, and the contour is simplified and approximated by polygons to initially extract the vertex coordinates of the target at the current time; sub-pixel level fine positioning is performed on each detected coarse corner point to obtain the corner coordinates with sub-pixel precision distortion removal.

[0013] (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of multiple corner points is taken as the overall pixel displacement of the single target. , ;

[0014] (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The positions of the light spots are compared to determine the position of the light spot until it is determined that the light spot is aligned with the center of the target.

[0015] (5) The laser displacement meter emits a laser and receives the reflected signal to perform laser ranging to measure the distance to the target. ;

[0016] (6) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to form a three-dimensional displacement vector in the gravity coordinate system.

[0017] In step (2), the method for obtaining the corner coordinates of sub-pixel precision distortion correction is as follows: a search window is established with the coarse corner as the center, and the edge normal equation is fitted by the iterative weighted least squares method based on the image gradient distribution of the two edge directions in the neighborhood of the corner, and the precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates; the method for corner distortion correction is as follows: using the pre-calibrated camera intrinsic parameter matrix and distortion coefficient Distortion correction is performed independently on the coordinates of the aforementioned sub-pixel reference vertices to obtain their true physical projection coordinates on the distortion-free ideal imaging surface.

[0018] In step (4), if And if the light spot is within the target area, then it is determined that the light spot is aligned with the target center; if If the light spot is within the target range, it is determined that the light spot is on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center; if the light spot is outside the target range, coarse search and fine search are performed in sequence until it is determined that the light spot is aligned with the target center.

[0019] The coarse search includes the following steps:

[0020] (4.1) The control unit drives the two-axis gimbal to move the laser displacement meter and scan outward in a radial pattern from the current pointing angle.

[0021] (4.2) During the scanning step, the vision processing module acquires images in real time; once the light spot is detected to have invaded the target RoI area, the control unit immediately interrupts the scanning and locks the current gimbal angle coordinates;

[0022] (4.3) Perform cross-validation of visual ranging and laser ranging. If the cross-validation is true and the spot does not yet meet the center threshold, then perform a fine search.

[0023] Furthermore, the refined search includes the following steps:

[0024] (5.1) The laser displacement meter emits a laser and receives the reflected signal, measures the distance in the current optical path direction, performs cross-verification of visual ranging and laser ranging, and ensures that the absolute difference between the two is less than the set threshold.

[0025] (5.2) Using the pixel distance between the light spot and the target center as the objective function, the adaptive step size gradient approximation method is adopted to alternately adjust the horizontal angle and pitch angle of the gimbal to gradually move the light spot to the target center;

[0026] (5.3) After reaching the precision threshold of fine search, perform cross-validation again; until the verification is true, then perform laser ranging.

[0027] In step (6), the synthesis of the three-dimensional displacement vector in the gravity coordinate system includes the following steps:

[0028] (6.1) The obtained pixel displacement Combined with the measured target distance This is converted into physical displacement within the visual plane;

[0029] (6.2) Convert the displacement values ​​of the device coordinate system to the gravity coordinate system based on the camera tilt angle.

[0030] In step (6.1), a perspective distortion correction term is introduced. and The corrected formula for the physical displacement of the equipment coordinate system is:

[0031]

[0032]

[0033] calculate Absolute longitudinal displacement in the direction:

[0034]

[0035] Perspective distortion correction item and The calculation formula is as follows:

[0036]

[0037]

[0038] in, , These are the equivalent focal lengths of the camera in the horizontal and vertical directions, respectively, in pixels; , The pixel coordinates of the camera's optical center in the image; The initial reference value for calculating the displacement of the target in the Z direction; These are the coordinates of the initial reference pixel.

[0039] In step (6.2), the initial tilt angle data is used. and As a fixed transformation reference for the equipment coordinate system relative to the gravity coordinate system, when the deviation of the current tilt angle from the initial tilt angle exceeds the threshold, the result of this cycle is low-confidence failure data.

[0040] Rotation matrix First rotate around the camera's Y-axis Then rotate around the X-axis Sequential construction:

[0041]

[0042] Convert device coordinates to gravity coordinates:

[0043]

[0044] After conversion For horizontal and lateral displacement, For vertical settlement displacement, These represent horizontal and longitudinal displacements, and the three directly correspond to the monitoring direction of the equipment.

[0045] The method for measuring the three-dimensional displacement of a structure using a single target according to this application has the following beneficial technical effects:

[0046] (1) This application uses visual feedback, closed-loop search, and step-by-step approximation to complete displacement measurement. Compared with the prior art, which uses trigonometric function calculation and then open-loop execution, it does not rely on the angle execution accuracy of the gimbal. It can effectively avoid the cumulative alignment error caused by gear wear and transmission error in open-loop execution, thereby reducing the accuracy requirements and cost of gimbal hardware, while improving the alignment reliability and measurement stability of long-term operation.

[0047] (2) This application has low requirements for the accuracy of the gimbal; it is only necessary to be able to see the position of the light spot, and there is no need for a gimbal with high angular accuracy.

[0048] (3) When the observation equipment deviates from the target, this application can re-search and align with the target after each deviation. Compared with the method of rotation, this avoids the wear of the rotating mechanism, thereby avoiding the cumulative error caused by wear and the phenomenon of being unable to align.

[0049] (4) The method of “visual coarse distance + laser ranging cross-verification” and “multi-strategy search + backoff mechanism” can be used to deal with the situation where the identified light spot deviates;

[0050] (5) This application uses an inclinometer for compensation, which can realize a real gravity coordinate system. Compared with the prior art, which uses the equipment as the coordinate system and has no compensation, it has obvious technical innovation. In addition, the inclinometer of this application does not need to be placed on a horizontal surface and can automatically compensate, avoiding the leveling operation required in the early stage of the existing equipment and simplifying the operation difficulty.

[0051] (6) This application proposes a three-dimensional displacement fusion calculation method. This method uses the two-dimensional pixel displacement of the target extracted by the vision module as the planar displacement information, the distance change measured by the laser displacement meter as the line-of-sight displacement information, and combines the equipment attitude data output by the inclinometer to convert the displacement in the equipment coordinate system to the gravity coordinate system. At the same time, perspective distortion correction is used to reduce the pseudo displacement error when imaging at a distance, thereby obtaining a more complete and accurate three-dimensional displacement result. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the measurement method of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

[0054] According to this application, a method for measuring the three-dimensional displacement of a structure using a single target operates on the logic of "searching through visual feedback → finding → approximating," without relying on the angular accuracy of the gimbal. This application utilizes an industrial camera to extract the two-dimensional displacement (X, Y) of the target within its plane and its distance from the optical path direction (Z), while simultaneously detecting the laser spot position. This distance can be a relatively low-precision value, which is sufficient to meet the requirements. Then, a laser displacement meter is used to measure the distance (Z) in the optical path direction. The gimbal in this application only needs to support the laser displacement meter and adjust the illumination direction, but high precision is not required. This application uses an inclinometer to measure the device's attitude angle, converting the device coordinate system to a gravity coordinate system, and employs a visual feedback search algorithm to achieve closed-loop control of the gimbal's alignment.

[0055] This application provides a system for measuring the three-dimensional displacement of a structure using a single target, comprising the following devices:

[0056] (1) Imaging equipment, such as an industrial camera: fixed on a rigid bracket, with the optical axis direction remaining unchanged after installation. The camera must support manual focus and have a focus lock function, and the resolution should meet the clear imaging requirements of the farthest target.

[0057] (2) Two-axis gimbal: Mounted on the same rigid bracket, it drives the laser displacement gauge to rotate in two degrees of freedom, horizontal and pitch, and receives angle commands from the control unit. The gimbal stepping accuracy should meet the requirements of coarse search step length constraint and fine search alignment accuracy.

[0058] (3) Laser displacement gauge: Fixed on the gimbal and rotates with the gimbal. Used to emit laser and receive reflected signals to measure the slant distance from the imaging device to the target surface. The measurement accuracy should meet the error requirements for structural displacement monitoring.

[0059] (4) Inclinometer: It is fixed to a rigid support and outputs the tilt angle along the vertical direction of the camera in real time. and tilt angle along the horizontal direction of the camera During installation, ensure that the two measuring axes of the inclinometer are aligned with the camera's coordinate axes. Corresponding to the tilt component in the vertical direction of the camera, The tilt component corresponding to the horizontal direction of the camera.

[0060] (5) Target: Mounted on the surface of the structure to be monitored. The target consists of two rigidly connected patterns: a matrix-coded identity feature area and a geometric measurement pattern. The matrix-coded identity feature area can be, for example, a QR code or AprilTag code, encoding a unique identifier ID; the geometric measurement pattern can be a rectangular high-contrast black and white outline. The physical side length of the rectangle must be known in advance as a reference for visual distance estimation. The target surface must be able to effectively reflect the laser emitted by the laser displacement meter. The target mounting surface should be approximately perpendicular to the camera's line of sight.

[0061] (6) Rigid bracket: Supports the industrial camera, two-axis gimbal and tilt meter, ensuring that the relative positions of the three remain stable during long-term operation. The bracket should have sufficient rigidity and vibration resistance.

[0062] (7) Storage module: Persistently stores camera intrinsic parameters, target reference data, tilt baseline and target status for displacement calculation.

[0063] According to the present application, a method for measuring the three-dimensional displacement of a structure using a single target involves acquiring images from a single target mounted on the structure to output the three-dimensional displacement of the target's gravity coordinate system, comprising the following steps:

[0064] (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, high threshold global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the bright spot.

[0065] High-threshold global thresholding is an image segmentation technique used to extract the most salient targets in an image. It uses a high, uniform grayscale threshold applicable to the entire image, which can effectively filter noise and weak signals. The threshold can be set empirically. That is, the entire image is binarized with a uniform threshold higher than that of the target white block and the background grayscale, retaining the overexposed pixel with the highest grayscale and filtering out the rest as background.

[0066] (2) In the image output by the above target extraction channel, track and lock the outermost maximum closed contour within the target RoI; use the RDP polygon approximation algorithm to simplify and approximate the contour with polygons, take the simplified approximate polygon with 4 vertices, and initially extract the coordinates of the 4 vertices of the rectangular target at the current time as coarse corner points; perform sub-pixel level fine positioning for each detected coarse corner point to obtain the corner coordinates of the distortion-free corner with sub-pixel precision.

[0067] The sub-pixel precision distortion correction of corner coordinates involves establishing a search window centered on the coarse corner point. Based on the image gradient distribution along the two edge directions within the corner's neighborhood, an iterative weighted least squares method is used to fit the edge normal equations, and the precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates. This method eliminates discrete gridding errors and serves as the physical source data input for subsequent displacement extraction. Then, corner distortion correction is performed using a pre-calibrated camera intrinsic parameter matrix. and distortion coefficient Distortion correction is performed independently on the coordinates of the four sub-pixel reference vertices to obtain their true physical projection coordinates on the distortion-free ideal imaging surface.

[0068] (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the overall pixel displacement of the single target. , .

[0069] (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The location of the light spot is determined by comparison.

[0070] (5) If And if the light spot is within the target area, then it is determined that the light spot is aligned with the target center; if If the light spot is within the target range, it is determined that the light spot is on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center; if the light spot is outside the target range, coarse search and fine search are performed in sequence until it is determined that the light spot is aligned with the target center.

[0071] (6) The laser displacement meter emits a laser and receives the reflected signal to measure the target distance. .

[0072] (7) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to form a three-dimensional displacement vector in the gravity coordinate system.

[0073] This application combines visual recognition with laser ranging. When the vision module detects a light spot entering the target area, the system does not immediately assume the laser has hit the target. Instead, it further compares the visually estimated distance with the measured laser distance. Only when the difference between the two meets a set threshold is the system confirmed that the light spot has actually landed on the target surface. This method reduces misjudgments caused by environmental reflections, background illumination errors, and stray light sources, thus improving the reliability of laser alignment.

[0074] The following will describe each step of this application in more detail. Specifically, a method for measuring the three-dimensional displacement of a structure using a single target, according to this application, includes the following steps:

[0075] S1. Initialization and Calibration Process: After the system is installed or redeployed for the first time, perform the following initialization steps to establish a measurement baseline.

[0076] S11. Equipment Installation and Initial Adjustment. Perform the following sub-steps:

[0077] S111. Bracket Installation and Equipment Fixing. Mount the industrial camera, two-axis gimbal (including laser displacement gauge), and inclinometer onto the same rigid bracket. The inclinometer is fixed to the bracket; during installation, ensure that the two measuring axes of the inclinometer are aligned with the camera's coordinate axes. Corresponding to the tilt component in the vertical direction of the camera, The tilt component corresponding to the horizontal direction of the camera.

[0078] S112. Camera Positioning and Focusing. Adjust the support posture so that the camera's field of view covers the target to be monitored. Adjust the camera's focus ring, and lock the focus ring after adjustment.

[0079] S113. Initial laser pointing adjustment. Control the gimbal to rotate so that the laser spot is projected near the center area of ​​the camera image. This position serves as the starting reference for subsequent visual search, reducing the search range.

[0080] S114. Inclination Baseline Recording. After the equipment is installed and stabilized, record the inclinometer reading at this time. As the initial attitude reference for the device.

[0081] S12. Camera intrinsic parameter calibration. Zhang Zhengyou's calibration method can be used, employing a checkerboard calibration board to capture multiple images in different poses, and then solving for the camera intrinsic parameter matrix. and distortion coefficient The calibration results are stored in the system configuration and used as the basic parameters for displacement extraction.

[0082]

[0083]

[0084] in , These are the equivalent focal lengths in the horizontal and vertical directions, respectively, in pixels; , The pixel coordinates of the camera's optical center in the image. The tilt coefficient; , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

[0085] S13. Target Reference Registration. Identify and confirm the unique target to be monitored within the current field of view. Perform the following sub-steps on the target to be monitored to establish and save the initial reference data for the target:

[0086] S131. Target Feature Recognition and Decoding. The vision module performs region localization and recognition on the currently acquired calibration image. First, it locates and retrieves the matrix-coded identity feature region in the target, decodes it to obtain the target's unique ID, and then expands outward from this region to retrieve the geometric measurement pattern rigidly connected to it.

[0087] S132. Establishing the Region of Interest (RoI) and Extracting Vertex Corners Based on Contour Approximation. A rectangular RoI is established centered on the identified target. Edge connectivity analysis and contour tracking are performed within the RoI to identify the largest closed contour of the target's outermost edge. The RDP polygon approximation algorithm (Ramer-Douglas-Peucker algorithm) is used to simplify this contour, and the distance tolerance parameter is adjusted. (A fixed proportion of the outline perimeter can be used), and an approximate polygon with 4 vertices after simplification is selected. The pixel coordinates of the four reference vertices of the rectangular target are initially extracted. This RoI is dynamically translated according to the target position in the previous frame during the measurement cycle, so that the target remains within the RoI after displacement.

[0088] S133. Sub-pixel corner localization. Based on the image gradient distribution along two edge directions within the corner's neighborhood, an iterative weighted least squares method is used to fit the edge normal equations, and the precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates. This method can eliminate discrete gridding errors and serve as the physical source data input for subsequent displacement extraction.

[0089] S134, Corner distortion correction. This is achieved using a pre-calibrated camera intrinsic parameter matrix. and distortion coefficient Distortion correction is performed independently on the coordinates of the four sub-pixel reference vertices to obtain their true physical projection coordinates on the distortion-free ideal imaging surface.

[0090] S135. Initial reference centroid coordinate calculation. Calculate the geometric center (i.e., the arithmetic mean) of the four reference vertices after distortion correction, and use this center as the initial reference pixel coordinates for the target. Stored in the system as the absolute reference zero point for all subsequent displacement difference calculations.

[0091] S136. Initial Distance Estimation and Recording. Based on the known physical side length of the target. and the pixel side length extracted from the image The approximate distance from the target to the camera is estimated using camera calibration parameters. .

[0092]

[0093] The manual operation controls the gimbal to align the laser spot with the center of the target, while the laser displacement meter measures the initial distance. .Will and Record at the same time. This serves as the initial reference value for calculating the Z-direction displacement of the target.

[0094] S2, the main process of single target measurement, completes the entire process from acquiring an image from a target to outputting the three-dimensional displacement of the target's gravity coordinate system.

[0095] S21. Image Acquisition and Preprocessing. The industrial camera acquires the current image. Due to the extreme grayscale contrast between the black and white target pattern and the high-energy laser spot in the image, the system copies the image and inputs it into two parallel preprocessing channels:

[0096] The first channel, namely target extraction, performs bilateral filtering for noise reduction and adopts adaptive local mean thresholding to transform the black and white boundary into a high-contrast binary image of the target.

[0097] The second channel, namely laser spot extraction, uses high-threshold global threshold segmentation to filter out the background and output a binarized image containing only the core domain of the bright spot.

[0098] S22. Target identification and 2D displacement extraction of the target. Perform the following sub-steps:

[0099] S221. Vertex extraction based on edge contour approximation. In the image output by the first channel, the outermost maximum closed contour is tracked and locked within the target RoI; the RDP polygon approximation algorithm is used to simplify the contour into a polygon, and an approximate polygon with 4 vertices after simplification is taken to initially extract the coordinates of the 4 vertices of the rectangular target at the current time.

[0100] S222, execute S133, S134 workflow to obtain the corner coordinates of subpixel precision distortion correction.

[0101] S223, Pixel displacement calculation. Let the initial pixel coordinates of the target corner be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the overall pixel displacement of the target. , .

[0102] S23, Laser spot status detection.

[0103] S231. The vision module detects the pixel position of the laser spot in the current image and calculates the Euclidean distance between the center of the laser spot and the center of the target RoI. :

[0104]

[0105] in, The current pixel coordinates of the light spot. The coordinates of the center pixel of the target ROI; a threshold for the Euclidean distance can be set. This serves as the standard for determining whether the light spot meets the center threshold.

[0106] S232. The position of the light spot is determined, and it is divided into the following three cases:

[0107] (1) If If the light spot is within the target area, it is determined that the light spot is aligned with the target center. The threshold for allowing the light spot to deviate from the target center can be specified according to the actual size of the target. At this point, S234 is first entered to perform cross-validation of visual ranging and laser ranging. If the validation is true, S24 laser ranging is then executed.

[0108] (2) If If the light spot is within the target area, it is determined that the light spot falls on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center.

[0109] (3) If the light spot is outside the target range, trigger coarse search processing.

[0110] S233, Coarse Search. The control unit drives the two-axis gimbal to move the laser displacement meter, scanning outward in a radial pattern along a preset Archimedean spiral trajectory from the current pointing angle. During the scanning process, the vision processing module acquires images in real time; once a light spot is detected intruding into the target RoI area, the control unit immediately interrupts the scanning and locks the current gimbal angle coordinates, then proceeds to S234 to perform cross-validation of visual ranging and laser ranging. If the cross-validation is true and the light spot has not yet met the center threshold, then proceeds to S235 to perform a fine search; if the cross-validation is false, then proceeds to S236 for backtracking.

[0111] S234. The laser displacement meter emits a laser and receives the reflected signal to measure the distance in the current optical path direction. Cross-validation of visual ranging and laser ranging is performed to ensure that the absolute difference between the two is less than a set threshold. ,Right now If the detected light spot is confirmed to be generated by laser irradiation on the target surface, rather than ambient reflection or other light sources, the cross-validation is returned as true; otherwise, the cross-validation is returned as false.

[0112] S235, Fine-tuning Search. Using the pixel distance between the light spot and the target center as the objective function, an adaptive step-size gradient approximation method is employed, alternately adjusting the pan / tilt angle and elevation angle to gradually move the light spot to the target center. After reaching the fine-tuning search accuracy threshold, S234 cross-validation is performed again; if the validation is true, S24 laser ranging is performed. If the light spot is lost during the fine-tuning search, or if any S234 cross-validation returns false, the process proceeds to S236 for backtracking.

[0113] S236. Rollback Processing. The system rolls back to the coarse search in S233 according to the preset state machine (the initial search radius is expanded during rollback to avoid oscillation), and records the number of rollbacks. When the number of rollbacks exceeds 3, the target state is marked as LOST and an alarm is triggered.

[0114] S24. Laser Ranging. A laser displacement meter emits a laser and receives the reflected signal to measure the distance to a target. .

[0115] S25. Physical displacement calculation and 3D data fusion, 3D displacement synthesis. The data obtained in the preceding steps are synthesized into a 3D displacement vector in the gravity coordinate system, executed according to the following sub-steps:

[0116] S251, Physical displacement calculation. Calculate the pixel displacement obtained in S223. Combined with the laser distance measured by S24 This is converted into physical displacement within the visual plane. Considering the longitudinal displacement of the target along the camera's line of sight in long-distance monitoring of large-span or tall structures... This can cause perspective distortion in the image, also known as the "near objects appear larger, far objects appear smaller" effect, resulting in pseudo-displacement errors in the lateral direction. To further improve the accuracy of 3D displacement calculation, the system introduces a perspective distortion correction term into the pixel displacement conversion formula. and The revised formula for the physical displacement of the device coordinate system is:

[0117]

[0118]

[0119] Simultaneously, calculation Absolute longitudinal displacement in the direction:

[0120]

[0121] Among them, perspective distortion correction item and The calculation formula is as follows:

[0122]

[0123]

[0124] S252. Convert the equipment coordinate system displacement value to the gravity coordinate system based on the camera tilt angle. Under normal operating conditions where the equipment support is in a stable posture, use the initial tilt angle data. and This serves as a fixed transformation reference for the device coordinate system relative to the gravity coordinate system; when the deviation of the current tilt angle from the initial tilt angle exceeds a threshold, the results for this cycle are marked as low-confidence failure data. The rotation matrix is ​​first rotated around the camera's Y-axis. Then rotate around the X-axis Sequential construction:

[0125]

[0126] Convert device coordinates to gravity coordinates:

[0127]

[0128] After conversion For horizontal and lateral displacement, For vertical settlement displacement, These represent horizontal and vertical displacements, and the three directly correspond to the equipment displacement in the monitoring direction.

[0129] In step S251, the slant distance measured by the laser is directly replaced by the vertical depth, and the geometric error boundary introduced is as follows.

[0130] Let the optical center of the camera be the origin of the device coordinate system, and the optical axis of the camera be... Axis. When the target deviates from the exact center of the image, let the angle between the target center and the optical axis be θ. At this point, the true slant distance measured by the laser displacement meter is... The vertical projection depth required for a rigorous pinhole model is The system then approximates the slant range as depth, introducing an absolute depth error. for:

[0131]

[0132] According to the lateral physical displacement calculation formula in step S251 The absolute error transmitted by this depth error to the lateral physical displacement solution. for:

[0133]

[0134] relative error for:

[0135]

[0136] Taking a real-world engineering example, in a scenario where the target is 30m away, the system uses a standard 1 / 1.8-inch image sensor with a half-diagonal of approximately 4.47mm and a 50mm lens. The maximum edge deviation angle of the image under this setup is approximately 5.1°. Substituting this into the formula, if the structure shifts by 10mm, the absolute error introduced by the non-orthogonality of the laser is 0.040mm, which is within an acceptable range. For measurement scenarios exceeding 30m, a longer focal length lens is used to meet imaging resolution requirements. This further reduces the system's field of view, correspondingly decreasing the maximum edge deviation angle, and further converging the upper bound of the error introduced by the non-orthogonality of the laser.

[0137] Although the embodiments disclosed in this application are as described above, the content is merely for the purpose of facilitating understanding of this application and is not intended to limit this application. Any person skilled in the art to which this application pertains may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application; however, the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for measuring the three-dimensional displacement of a structure using a single target, characterized in that, Outputting the three-dimensional displacement of a target in its gravity coordinate system by acquiring images from a single target includes the following steps: (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, high threshold global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the spot. (2) In the image output by the target extraction channel, the outermost maximum closed contour is tracked and locked within the target RoI, and the contour is simplified and approximated by polygons to initially extract the vertex coordinates of the target at the current time; sub-pixel level fine positioning is performed on each detected coarse corner point to obtain the corner coordinates with sub-pixel precision distortion removal. (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of multiple corner points is taken as the overall pixel displacement of the single target. , ; (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The positions of the light spots are compared to determine the position of the light spot until it is determined that the light spot is aligned with the center of the target. (5) The laser displacement meter emits a laser and receives the reflected signal to perform laser ranging to measure the distance to the target. ; (6) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to form a three-dimensional displacement vector in the gravity coordinate system.

2. The method according to claim 1, characterized in that, In step (2), the method for obtaining the corner coordinates of sub-pixel precision distortion correction is as follows: a search window is established with the coarse corner as the center, and the edge normal equation is fitted by the iterative weighted least squares method based on the image gradient distribution of the two edge directions in the neighborhood of the corner, and the precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates. The method for corner distortion correction is as follows: using a pre-calibrated camera intrinsic parameter matrix and distortion coefficient Distortion correction is performed independently on the coordinates of the aforementioned sub-pixel reference vertices to obtain their true physical projection coordinates on the distortion-free ideal imaging surface.

3. The method according to claim 1 or 2, characterized in that, In step (4), if And if the light spot is within the target area, then it is determined that the light spot is aligned with the target center; if If the light spot is within the target range, it is determined that the light spot is on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center; if the light spot is outside the target range, coarse search and fine search are performed in sequence until it is determined that the light spot is aligned with the target center.

4. The method according to claim 3, characterized in that, The coarse search includes the following steps: (4.1) The control unit drives the two-axis gimbal to move the laser displacement meter and scan outward in a radial pattern from the current pointing angle. (4.2) During the scanning step, the vision processing module acquires images in real time; once the light spot is detected to have invaded the target RoI area, the control unit immediately interrupts the scanning and locks the current gimbal angle coordinates; (4.3) Perform cross-validation of visual ranging and laser ranging. If the cross-validation is true and the spot does not yet meet the center threshold, then perform a fine search.

5. The method according to claim 4, characterized in that, The refined search includes the following steps: (5.1) The laser displacement meter emits a laser and receives the reflected signal, measures the distance in the current optical path direction, performs cross-verification of visual ranging and laser ranging, and ensures that the absolute difference between the two is less than the set threshold. (5.2) Using the pixel distance between the light spot and the target center as the objective function, the adaptive step size gradient approximation method is adopted to alternately adjust the horizontal angle and pitch angle of the gimbal to gradually move the light spot to the target center; (5.3) After reaching the precision threshold of fine search, perform cross-validation again; until the verification is true, then perform laser ranging.

6. The method according to claim 1, 2, 4, or 5, characterized in that, In step (6), the synthesis of the three-dimensional displacement vector in the gravity coordinate system includes the following steps: (6.1) The obtained pixel displacement Combined with the measured target distance This is converted into physical displacement within the visual plane; (6.2) Convert the displacement values ​​of the device coordinate system to the gravity coordinate system based on the camera tilt angle.

7. The method according to claim 6, characterized in that, In step (6.1), a perspective distortion correction term is introduced. and The corrected formula for the physical displacement of the equipment coordinate system is: calculate Absolute longitudinal displacement in the direction: Perspective distortion correction item and The calculation formula is as follows: in, , These are the equivalent focal lengths of the camera in the horizontal and vertical directions, respectively, in pixels; , The pixel coordinates of the camera's optical center in the image; The initial reference value for calculating the displacement of the target in the Z direction; These are the coordinates of the initial reference pixel.

8. The method according to claim 7, characterized in that, In step (6.2), the initial tilt angle data is used. and As a fixed transformation reference for the equipment coordinate system relative to the gravity coordinate system, when the deviation of the current tilt angle from the initial tilt angle exceeds the threshold, the result of this cycle is low-confidence failure data. Rotation matrix First rotate around the camera's Y-axis Then rotate around the X-axis Sequential construction: Convert device coordinates to gravity coordinates: After conversion For horizontal and lateral displacement, For vertical settlement displacement, These represent horizontal and longitudinal displacements, and the three directly correspond to the monitoring direction of the equipment.

Images