A method for measuring three-dimensional displacement of a structure using multiple targets

By using a vision module and multi-target synchronous recognition technology, a single device can monitor the three-dimensional displacement of multiple targets, solving the problems of complexity and high cost of traditional monitoring systems, reducing equipment requirements, and improving monitoring efficiency and accuracy.

CN123065353APending Publication Date: 2026-11-10CABR TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611356381.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-03
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Traditional building structural health monitoring requires an independent laser ranging device for each measuring point, resulting in high costs and complex systems, making it difficult to efficiently monitor the three-dimensional displacement of multiple measuring points.

Method used

Employing the global perception capability of the vision module and the technology of simultaneous recognition and ROI binding of multiple targets, the system achieves the cyclic monitoring of multiple targets through a single set of equipment. It uses the vision module to identify the features of all targets within the field of view, establishes an independent ROI region for each target, and realizes the three-dimensional displacement measurement of multiple targets through path planning and state machine management.

Benefits of technology

It enables single-device monitoring of multiple measurement points, reduces system costs, balances search speed and alignment accuracy, is suitable for long-distance and multi-target monitoring scenarios, can provide reliability judgment of measurement results, and reduces the impact of abnormal situations such as equipment tilting, settlement and target loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123065353A_ABST
    Figure CN123065353A_ABST
Patent Text Reader

Abstract

This application relates to a method for measuring the three-dimensional displacement of a structure using multiple targets, comprising the following steps: loading pre-established target coordinate mapping information and generating a target traversal sequence for path planning; maintaining an independent state machine for each target to mark the target's working state; after the scheduler selects the next target i along the traversal sequence, performing a fast presence detection before laser switching to avoid wasting alignment time on targets not tracked by vision, until target i is detected to be visible in the field of view; performing a complete displacement measurement on target i according to the single-target measurement process, and marking the target state as under normal monitoring and outputting displacement data normally.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to a method for measuring three-dimensional displacement of a structure using multiple targets, which can be used in the technical field of building health monitoring. Background Technology

[0002] In building structural health monitoring, it is typically necessary to acquire the three-dimensional displacement of key measuring points in real time, that is, the displacement of key measuring points in three orthogonal directions, in order to determine whether the building structure has suffered damage or failure. In actual engineering, a monitoring station usually needs to monitor multiple targets at multiple measuring points simultaneously. Traditional solutions require an independent laser ranging device for each measuring point, which is costly and complex.

[0003] Therefore, how to use as few devices as possible to measure the three-dimensional displacement of a structure through multiple targets is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0004] The purpose of this application is to provide a method for measuring the three-dimensional displacement of a structure using multiple targets. By utilizing the global perception capability of a vision module and employing multi-target synchronous identification and ROI binding technology, a single set of equipment can perform roving monitoring of multiple targets.

[0005] This application relates to a method for measuring the three-dimensional displacement of a structure using multiple targets, comprising the following steps:

[0006] S31. Load the pre-established target coordinate mapping information and generate the target traversal sequence for path planning; each target maintains an independent state machine to mark the target's working state.

[0007] S32. After the scheduler selects the next target i along the traversal sequence, it performs a fast presence detection before the laser switches to avoid wasting alignment time on targets that are not tracked by vision, until the target i is detected to be visible in the field of view.

[0008] S33. Following the single target measurement procedure, perform a complete displacement measurement on target i, and mark the target status as being under normal monitoring and output displacement data normally.

[0009] In step S31, the working state of the target is defined as follows:

[0010] ACTIVE: Under normal monitoring, it is being rotated and dispatched and outputs displacement data normally;

[0011] SEARCHING: The laser spot is being aligned and there is no displacement output at the moment;

[0012] LOST: If no target information is identified in the visual image before the round trip, the target will be skipped.

[0013] In step S31, the path planning method includes: calculating the target rotation angle of the gimbal in the horizontal and pitch directions, and driving the gimbal to rotate; starting from the current position, selecting the nearest unvisited target as the next target each time, and generating a traversal sequence.

[0014] Step S32 includes:

[0015] S321, Trigger the camera to capture a frame of image, and search for the features of target i in the image;

[0016] S322. If target i is detected, it means that the target is visible in the field of view. Proceed to step S33 to execute the complete single target measurement process and mark the target status as SEARCHING.

[0017] S323. If target i is not detected, it means that the target has disappeared from the field of view. Mark the target status as LOST and report it to the system. The scheduler skips the target, returns immediately and selects the next target in the traversal sequence.

[0018] Preferably, the system may also include step S4: during operation, the system monitors the equipment tilt alarm index, equipment settlement alarm index and target loss alarm index in parallel to perform abnormal monitoring and early warning. Among them, the equipment tilt alarm refers to the inclinometer outputting the current tilt angle θ along the vertical direction of the camera and the tilt angle φ along the horizontal direction of the camera in real time. After each measurement cycle, the system compares the change of the current reading with the initial reference. When the change is greater than the set threshold, the equipment tilt alarm is triggered, and the three-dimensional displacement calculation result of the current measurement cycle is marked as low-confidence failure data. The equipment settlement alarm refers to all or most of the remote targets showing a relative displacement ΔY in the Y direction with similar magnitude in the same direction in the equipment coordinate system. The system uses this feature to indirectly detect equipment settlement. After each round cycle, the ΔY value of all ACTIVE targets is compared. If the ΔY direction of all or most of the ACTIVE targets is consistent and the magnitude is similar, the equipment settlement alarm is triggered, and the three-dimensional displacement calculation result of the current measurement cycle is marked as low-confidence failure data. The target loss alarm refers to the target loss alarm being triggered when the target status is marked as LOST in the pre-detection step S32 before the switching of the multi-target round cycle process.

[0019] The path planning method includes the following steps:

[0020] (1) Take the current gimbal position as the starting point;

[0021] (2) Calculate the rotation angle distance from the starting point to each unvisited target, and select the closest one as the next target;

[0022] (3) Drive the gimbal to the reference pointing angle of the target and perform single target measurement;

[0023] (4) Mark the target as visited, and use its pointing angle as the new starting point to repeat steps (2) and (3) until all targets have been visited.

[0024] Preferably, the gimbal rotation angle distance between targets Defined as the Euclidean distance between the reference pointing angles of two targets:

[0025]

[0026] in, and Here, the reference pointing angles are for targets i and j, respectively. The difference in gimbal rotation angle between adjacent targets is estimated using a small-angle approximation formula:

[0027] , ;

[0028] In the formula, , fx and fy represent the pixel coordinate difference between the two targets in the image, and fx and fy are the equivalent focal lengths of the camera, in pixels.

[0029] The single-target measurement process involves acquiring images from a single target and outputting the three-dimensional displacement of the target's gravity coordinate system, including the following steps:

[0030] (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the spot.

[0031] (2) In the image output by the target extraction channel, track and lock the outermost maximum closed contour within the target RoI; perform polygon simplification approximation on the contour to initially extract the vertex coordinates of the target at the current moment; perform sub-pixel level fine localization on each detected coarse corner point to obtain the corner coordinates with sub-pixel precision distortion removal.

[0032] (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the displacement of the entire pixel of the single target. , ;

[0033] (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The positions of the light spots are compared to determine the position of the light spot until it is determined that the light spot is aligned with the center of the target.

[0034] (5) The laser displacement meter emits a laser and receives the reflected signal to perform laser ranging to measure the distance to the target. ;

[0035] (6) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to synthesize the data.

[0036] This is a three-dimensional displacement vector in the gravity coordinate system.

[0037] The method for measuring the three-dimensional displacement of a structure using multiple targets according to this application has the following technical advantages:

[0038] (1) This application uses a multi-target measurement structure to measure the three-dimensional displacement. The vision module can simultaneously identify the features of all targets in the field of view, establish a separate ROI region for each target, bind the ROI of each target to its target number, maintain a target coordinate mapping table, and extract the two-dimensional displacement of each target in parallel without interfering with each other.

[0039] (2) In the multi-target scenario, this application establishes an independent state machine for each target and incorporates the target detection, laser alignment, measurement success, target loss, backtracking exceeding limits, equipment tilting and suspected equipment subsidence into the scheduling logic; the system can decide whether to output measurement results, skip the current target, reduce the confidence of the results and trigger an alarm according to different states; it can not only complete automatic round-robin measurement, but also provide information on whether the results are reliable at the same time;

[0040] (3) In view of the problems of laser spot misalignment, overshooting or loss during multi-target switching, this application adopts a control method that combines coarse search, fine search and back-off processing; it can simultaneously take into account search speed and alignment accuracy, and is suitable for long-distance and multi-target monitoring scenarios.

[0041] (4) This application can realize single device monitoring of multiple measurement points, that is, one set of equipment can monitor multiple targets in turn, without the need to equip each measurement point with an independent laser device, which greatly reduces the system cost; the vision module, as a "global sensor", monitors the position of all targets at the same time, and can directly locate the new target when switching, without blindly searching. Attached Figure Description

[0042] Figure 1This is a flowchart illustrating the multi-target measurement method of this application.

[0043] Figure 2 This is a schematic diagram of the target anomaly early warning process of this application.

[0044] Figure 3 This is a schematic diagram of the equipment anomaly early warning process in this application.

[0045] Figure 4 This is a flowchart illustrating the single-target measurement method of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

[0047] This application provides a method for measuring the three-dimensional displacement of a structure using multiple targets. Multiple targets are set up in the target area, and a single device is used to monitor the multiple targets in rotation. This application uses multiple targets to measure the three-dimensional displacement of the structure. The vision module can simultaneously identify the features of all targets within the field of view, and a separate region of interest (ROI) is established for each target. The ROI of each target is bound to its target number. The system maintains a target coordinate mapping table, and the two-dimensional displacements (X, Y) of each target can be extracted in parallel. This application allows for selection of coarse search or fine search based on pre-positioning accuracy, balancing efficiency and reliability. The coarse search is used to quickly relocate the target, while the fine search is used to further move the light spot to the target center. Backtracking is used to re-enter the search process in case of verification failure or light spot loss.

[0048] In this application, when the laser needs to switch from one target to another for monitoring, the vision module identifies the pixel coordinates of the target in the current frame; based on the positional offset of the new target in the frame, it roughly calculates the horizontal and vertical angles that the gimbal needs to adjust using trigonometric functions, and controls the gimbal to rotate to the estimated approximate position. Based on the pre-positioned state, the following search strategy can be selected:

[0049] Coarse search strategy: If the laser spot does not enter the ROI range of the new target after pre-positioning, the coarse search process is triggered, and the spot position is found through visual feedback closed-loop control.

[0050] Fine-grained search strategy: If the laser spot has entered the ROI range of the new target after pre-positioning, but is not in the center, the coarse search is skipped, and the fine-grained search process is directly initiated. An approximation algorithm is used to move the spot to the center of the target. After alignment, cross-validation is performed using visual coarse distance and laser range measurement. Once confirmed to be correct, the target is put into continuous monitoring mode.

[0051] This application provides a method for measuring the three-dimensional displacement of a structure using multiple targets, employing a multi-target rotation method for displacement monitoring. When multiple targets are positioned within the visual range and their displacements need to be measured, the system sequentially executes a complete single-target measurement process for each target. Compared to calling each target independently, multi-target rotation adds scheduling management, path optimization, and state coordination. Specifically, the equipment used in this application includes:

[0052] (1) Imaging equipment, such as an industrial camera: fixed on a rigid bracket, with the optical axis direction remaining unchanged after installation. The camera must support manual focus and have a focus lock function, and the resolution should meet the clear imaging requirements of the farthest target.

[0053] (2) Two-axis gimbal: Mounted on the same rigid bracket, it drives the laser displacement gauge to rotate in two degrees of freedom, horizontal and pitch, and receives angle commands from the control unit. The gimbal stepping accuracy should meet the requirements of coarse search step length constraint and fine search alignment accuracy.

[0054] (3) Laser displacement gauge: Fixed on the gimbal and rotates with the gimbal. Used to emit laser and receive reflected signals to measure the slant distance from the imaging device to the target surface. The measurement accuracy should meet the error requirements for structural displacement monitoring.

[0055] (4) Inclinometer: Fixed to a rigid bracket, it outputs the tilt angle θ along the vertical direction of the camera and the tilt angle φ along the horizontal direction of the camera in real time. During installation, ensure that the two measuring axes of the inclinometer are aligned with the camera's coordinate axes, where θ corresponds to the tilt component in the vertical direction of the camera, and φ corresponds to the tilt component in the horizontal direction of the camera. After the equipment is installed stably, record the inclinometer readings at this time. As the initial attitude reference for the device.

[0056] (5) Target: Installed on the surface of each structure to be monitored. Each target consists of two rigidly connected patterns: a matrix-coded identity feature area and a geometric measurement pattern. The matrix-coded identity feature area can be, for example, a QR code or AprilTag code, encoding a unique identifier ID; the geometric measurement pattern can be a rectangular high-contrast black and white outline. The physical side length of the rectangle must be known in advance as a reference for visual distance estimation. The target surface must be able to effectively reflect the laser emitted by the laser displacement meter. The target mounting surface should be approximately perpendicular to the camera's line of sight.

[0057] (6) Rigid bracket: Supports the industrial camera, two-axis gimbal and tilt meter, ensuring that the relative positions of the three remain stable during long-term operation. The bracket should have sufficient rigidity and vibration resistance.

[0058] (7) Storage module: Persistently stores camera intrinsic parameters, reference data of each target, tilt baseline and target status for use in round-robin scheduling and displacement calculation.

[0059] Because the two-axis gimbal drives the laser displacement meter to perform multi-target cyclic searching, the laser emission optical path, under non-zero rotation angle, has an angle that is not completely orthogonal to the camera optical axis (i.e., the vertical depth axis in the pinhole imaging model). Considering that the installation distance between the optical center of the industrial camera and the emission center of the laser displacement meter on the rigid support is much smaller than the actual monitoring distance, the two can be approximated as a concentric system in the error boundary analysis.

[0060] Specifically, a method for measuring the three-dimensional displacement of a structure using multiple targets, according to this application, includes the following steps:

[0061] S31. Load target mapping information and path planning. Load pre-established target coordinate mapping information, which is established after target baseline registration during system installation and deployment. The table records the ID, RoI region, and initial baseline position of each target. Its establishment process is consistent with the initialization process of single-target measurement. The scheduler generates a target traversal sequence based on this for path planning. Each target maintains an independent state machine (FSM) to mark the target's working state. The aforementioned state machine represents a mathematical model of a finite number of states and the transitions and actions between these states. The system can only be in one state at any given time. Specifically, this includes:

[0062] S311. Load the pre-established target coordinate mapping information. The table records the ID, RoI region, and initial reference position of each target. .

[0063] S312. Establish an independent state machine for each target. For example, the target state can be defined as follows:

[0064] ACTIVE: Under normal monitoring, it is being rotated and dispatched and outputs displacement data normally;

[0065] SEARCHING: The laser spot is being aligned and there is no displacement output at the moment;

[0066] LOST: If no target information is identified in the visual image before the round trip, the target will be skipped.

[0067] S313. Path Planning. Calculate the target rotation angles of the gimbal in the horizontal and pitch directions, and drive the gimbal to rotate. Starting from the current position, select the nearest unvisited target as the next target each time, generating a traversal sequence to reduce the total rotation time and mechanical wear of the gimbal.

[0068] S32. Pre-switch detection. After the scheduler selects the next target i along the traversal sequence, it performs a fast presence detection before laser switching to avoid wasting alignment time on targets not tracked by vision. Specifically, this includes:

[0069] S321, Trigger the camera to capture a frame of image, and search for the features of target i in the image.

[0070] S322. If target i is detected: This indicates that the target is visible in the field of view. Proceed to S33 to execute the complete single target measurement process and mark the target status as SEARCHING.

[0071] S323. If target i is not detected, it means that the target has disappeared from the field of view. Perform the following processing: mark the target status as LOST and report to the system. The scheduler skips the target and immediately returns to S32 to select the next target in the traversal sequence.

[0072] S33. Perform single target measurement. Following the single target measurement procedure, perform a complete displacement measurement on target i and mark the target state as ACTIVE.

[0073] S4. Anomaly Monitoring and Early Warning Process. During operation, the system monitors the following indicators in parallel. Once triggered, an alarm is output to notify maintenance personnel for intervention.

[0074] S41, Equipment Tilt Alarm. The inclinometer outputs the current tilt angle θ along the camera's vertical direction and the tilt angle φ along the camera's horizontal direction in real time. After each measurement cycle, the system compares the current reading with the initial reference value. When the change... or If this occurs, it indicates that the equipment support has experienced a non-negligible tilt, triggering a tilt alarm and marking the three-dimensional displacement calculation results of the current measurement cycle as low-confidence failure data. , The threshold is set based on the distance measured from the target.

[0075] S42. Equipment Settlement Alarm. When the equipment itself experiences overall settlement or elevation, all or most of the remote targets will exhibit relative displacements in the same direction and with similar amplitudes in the Y-direction within the equipment coordinate system. The system utilizes this characteristic to indirectly detect equipment settlement: after each cycle, the ΔY values ​​of all ACTIVE targets are compared. If the ΔY directions of all or most ACTIVE targets are consistent and their amplitudes are similar, the equipment triggers a settlement alarm and marks the three-dimensional displacement calculation results of the current measurement cycle as low-confidence failure data.

[0076] S43, Target Loss Alarm. A target loss alarm is triggered when the target status is marked as LOST in the pre-detection step S32 before switching to the multi-target rotation process.

[0077] S44. When the number of backtracking records in S236 exceeds 3, an alarm is triggered, and the target displacement reading is skipped, proceeding to the next target. When the number of backtracking records in S236 does not exceed 3, the process returns to S33 to continue measurement.

[0078] To reduce the ineffective rotation time and mechanical wear of the gimbal between targets, the access order of each target needs to be optimized. The following describes a method for planning multi-target round-robin paths, where the scheduler uses a nearest neighbor greedy algorithm to generate the traversal sequence, including the following steps:

[0079] (1) Take the current gimbal position as the starting point;

[0080] (2) Calculate the rotation angle distance from the starting point to each unvisited target, and select the closest one as the next target;

[0081] (3) Drive the gimbal to the reference pointing angle of the target and perform single target measurement;

[0082] (4) Mark the target as visited, and use its pointing angle as the new starting point. Repeat (2) and (3) until all targets have been visited.

[0083] The distance of the gimbal rotation angle between targets Defined as the Euclidean distance between the reference pointing angles of two targets:

[0084]

[0085] in, and Here, represents the reference pointing angles for targets i and j, respectively. The difference in gimbal rotation angle between adjacent targets can be estimated using a small-angle approximation formula: , In the formula, , fx and fy represent the pixel coordinate difference between the two targets in the image, and fx and fy are the equivalent focal lengths of the camera, in pixels.

[0086] The single-target measurement mentioned above in this application includes the following steps:

[0087] (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, high threshold global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the bright spot.

[0088] High-threshold global thresholding is an image segmentation technique used to extract the most salient targets in an image. It employs a high, uniform grayscale threshold applicable to the entire image, effectively filtering noise and weak signals. The threshold can be set empirically; that is, a uniform threshold higher than the grayscale of the target white patch and the background is used to segment the entire image. Figure 2 Value-enhanced processing retains the overexposed pixels with the highest grayscale, while the rest are filtered out and treated as background.

[0089] (2) In the image output by the target extraction channel, track and lock the outermost maximum closed contour within the region of interest (RoI); the RDP (Ramer-Douglas-Peucker) polygon approximation algorithm can be used to simplify and approximate the contour into a polygon, and take the simplified approximate polygon with 4 vertices to initially extract the coordinates of the 4 vertices of the rectangular target at the current moment as coarse corner points; perform sub-pixel level fine localization on each detected coarse corner point to obtain the corner coordinates with sub-pixel precision distortion removal.

[0090] The sub-pixel precision distortion correction of corner coordinates involves establishing a search window centered on the coarse corner point. Based on the image gradient distribution along the two edge directions within the corner's neighborhood, an iterative weighted least squares method is used to fit the edge normal equations, and the precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates. This method eliminates discrete gridding errors and serves as the physical source data input for subsequent displacement extraction. Then, corner distortion correction is performed using a pre-calibrated camera intrinsic parameter matrix. and distortion coefficient Distortion correction is performed independently on the coordinates of the four sub-pixel reference vertices to obtain their true physical projection coordinates on the distortion-free ideal imaging surface. Among these,

[0091]

[0092]

[0093] in , These are the equivalent focal lengths in the horizontal and vertical directions, respectively, in pixels; , The pixel coordinates of the camera's optical center in the image. The tilt coefficient; , , The radial distortion coefficient is... , denoted as the tangential distortion coefficient.

[0094] (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the overall pixel displacement of the single target. , .

[0095] (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The location of the light spot is determined by comparison.

[0096] (5) If And if the light spot is within the target area, then it is determined that the light spot is aligned with the target center; if If the light spot is within the target range, it is determined that the light spot is on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center; if the light spot is outside the target range, coarse search and fine search are performed in sequence until it is determined that the light spot is aligned with the target center.

[0097] (6) The laser displacement meter emits a laser and receives the reflected signal to measure the target distance. .

[0098] (7) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to form a three-dimensional displacement vector in the gravity coordinate system.

[0099] This application, through the aforementioned single-target measurement main process, can complete the entire process from acquiring an image of a target to outputting the three-dimensional displacement of the target in its gravity coordinate system. Specifically, it includes the following steps:

[0100] S21. Image Acquisition and Preprocessing. The industrial camera acquires the current image. Due to the extreme grayscale contrast between the black and white target pattern and the high-energy laser spot in the image, the system copies the image and inputs it into two parallel preprocessing channels:

[0101] The first channel, namely target extraction, performs bilateral filtering for noise reduction and adopts adaptive local mean thresholding to transform the black and white boundary into a high-contrast binary image of the target.

[0102] The second channel, namely laser spot extraction, uses high-threshold global threshold segmentation to filter out the background and output a binarized image containing only the core domain of the bright spot.

[0103] S22. Target identification and 2D displacement extraction of the target. Perform the following sub-steps:

[0104] S221. Vertex extraction based on edge contour approximation. In the image output by the first channel, the outermost maximum closed contour is tracked and locked within the target RoI; the RDP polygon approximation algorithm is used to simplify the contour into a polygon, and an approximate polygon with 4 vertices after simplification is taken to initially extract the coordinates of the 4 vertices of the rectangular target at the current time.

[0105] S222. Based on the image gradient distribution of the two edge directions in the neighborhood of the corner point, the edge normal equation is fitted by the iterative weighted least squares method. The precise intersection of the two edge lines is obtained as the sub-pixel corner coordinates. Distortion correction is performed on the above four sub-pixel reference vertex coordinates to obtain the initial state of the corner coordinates of the sub-pixel precision distortion correction.

[0106] S223, Pixel displacement calculation. Let the initial pixel coordinates of the target corner be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the overall pixel displacement of the target. , .

[0107] S23, Laser spot status detection.

[0108] S231. The vision module detects the pixel position of the laser spot in the current image and calculates the Euclidean distance between the center of the laser spot and the center of the target RoI. :

[0109]

[0110] in The current pixel coordinates of the light spot. The coordinates of the center pixel of the target ROI.

[0111] S232. The position of the light spot is determined, and the following three cases are identified:

[0112] (1) If If the light spot is within the target area, it is determined that the light spot is aligned with the target center. The threshold for allowing the light spot to deviate from the target center can be specified according to the actual size of the target. At this point, S234 is first entered to perform cross-validation of visual ranging and laser ranging. If the validation is true, S24 laser ranging is then executed.

[0113] (2) If If the light spot is within the target area, it is determined that the light spot falls on the target but deviates from the target center, and a fine search is directly triggered to gradually move the light spot to the target center.

[0114] (3) If the light spot is outside the target range, trigger coarse search processing.

[0115] S233, Coarse Search. The control unit drives the two-axis gimbal to move the laser displacement meter, scanning outward in a radial pattern along a preset Archimedean spiral trajectory from the current pointing angle. During the scanning process, the vision processing module acquires images in real time; once a light spot is detected intruding into the target RoI area, the control unit immediately interrupts the scanning and locks the current gimbal angle coordinates, then proceeds to S234 to perform cross-validation of visual ranging and laser ranging. If the cross-validation is true and the light spot has not yet met the center threshold, then proceeds to S235 to perform a fine search; if the cross-validation is false, then proceeds to S236 for backtracking.

[0116] S234. The laser displacement meter emits a laser and receives the reflected signal to measure the distance in the current optical path direction. Cross-validation of visual ranging and laser ranging is performed to ensure that the absolute difference between the two is less than a set threshold. ,Right now If the detected light spot is confirmed to be generated by laser irradiation on the target surface, rather than ambient reflection or other light sources, the cross-validation is returned as true; otherwise, the cross-validation is returned as false.

[0117] S235, Fine-tuning Search. Using the pixel distance between the light spot and the target center as the objective function, an adaptive step-size gradient approximation method is employed, alternately adjusting the pan / tilt angle and elevation angle to gradually move the light spot to the target center. After reaching the fine-tuning search accuracy threshold, S234 cross-validation is performed again; if the validation is true, S24 laser ranging is performed. If the light spot is lost during the fine-tuning search, or if any S234 cross-validation returns false, the process proceeds to S236 for backtracking.

[0118] S236. Rollback Processing. The system rolls back to the coarse search in S233 according to the preset state machine (the initial search radius is expanded during rollback to avoid oscillation), and records the number of rollbacks. When the number of rollbacks exceeds 3, the target state is marked as LOST and an alarm is triggered.

[0119] S24. Laser Ranging. A laser displacement meter emits a laser and receives the reflected signal to measure the distance to a target. .

[0120] S25. Physical displacement calculation and 3D data fusion, 3D displacement synthesis. The data obtained in the preceding steps, combined with the equipment attitude data output from the inclinometer, are synthesized into a 3D displacement vector in the gravity coordinate system, executed according to the following sub-steps:

[0121] S251, Physical displacement calculation. Calculate the pixel displacement obtained in S223. Combined with the laser distance measured by S24 This is converted into physical displacement within the visual plane. Considering the longitudinal displacement of the target along the camera's line of sight in long-distance monitoring of large-span or tall structures... This can cause perspective distortion in the image, also known as the "near objects appear larger, far objects appear smaller" effect, resulting in pseudo-displacement errors in the lateral direction. To further improve the accuracy of 3D displacement calculation, the system introduces a perspective distortion correction term into the pixel displacement conversion formula. and The revised formula for the physical displacement of the device coordinate system is:

[0122]

[0123]

[0124] Simultaneously, calculation Absolute longitudinal displacement in the direction:

[0125]

[0126] Among them, perspective distortion correction item and The calculation formula is as follows:

[0127]

[0128]

[0129] S252. Convert the displacement values ​​of the equipment coordinate system to the gravity coordinate system based on the camera tilt angle. Under normal operating conditions where the equipment support attitude is stable, the initial tilt angle data θ0 and φ0 are used as the fixed transformation reference of the equipment coordinate system relative to the gravity coordinate system; when the deviation of the current tilt angle from the initial tilt angle exceeds the threshold, the results of this cycle are marked as low-confidence failure data according to S41. The rotation matrix is ​​constructed in the order of first rotating φ0 around the camera Y-axis, and then rotating θ0 around the X-axis:

[0130]

[0131] Convert device coordinates to gravity coordinates:

[0132]

[0133] After conversion For horizontal and lateral displacement, For vertical settlement / uplift displacement, These represent horizontal and longitudinal displacements, and the three directly correspond to the monitoring direction of the equipment.

[0134] Although the embodiments disclosed in this application are as described above, the content is merely for the purpose of facilitating understanding of this application and is not intended to limit this application. Any person skilled in the art to which this application pertains may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application; however, the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for measuring the three-dimensional displacement of a structure using multiple targets, characterized in that, Includes the following steps: S31. Load the pre-established target coordinate mapping information and generate the target traversal sequence for path planning; each target maintains an independent state machine to mark the target's working state. S32. After the scheduler selects the next target i along the traversal sequence, it performs a fast presence detection before the laser switches to avoid wasting alignment time on targets that are not tracked by vision, until the target i is detected to be visible in the field of view. S33. Following the single target measurement procedure, perform a complete displacement measurement on target i, and mark the target status as being under normal monitoring and output displacement data normally.

2. The method according to claim 1, characterized in that, In step S31, the working state of the target is defined as follows: ACTIVE: Under normal monitoring, it is being rotated and dispatched and outputs displacement data normally; SEARCHING: The laser spot is being aligned and there is no displacement output at the moment; LOST: If no target information is identified in the visual image before the round trip, the target will be skipped.

3. The method according to claim 2, characterized in that, In step S31, the path planning method includes: calculating the target rotation angle of the gimbal in the horizontal and pitch directions, and driving the gimbal to rotate; starting from the current position, selecting the nearest unvisited target as the next target each time, and generating a traversal sequence.

4. The method according to claim 2, characterized in that, Step S32 includes: S321, Trigger the camera to capture a frame of image, and search for the features of target i in the image; S322. If target i is detected, it means that the target is visible in the field of view. Proceed to step S33 to execute the complete single target measurement process and mark the target status as SEARCHING. S323. If target i is not detected, it means that the target has disappeared from the field of view. Mark the target status as LOST and report it to the system. The scheduler skips the target, returns immediately and selects the next target in the traversal sequence.

5. The method according to any one of claims 2-4, characterized in that, It also includes step S4: During the operation of the system, the system monitors the equipment tilt alarm index, equipment settlement alarm index and target loss alarm index in parallel to carry out abnormal monitoring and early warning.

6. The method according to claim 5, characterized in that, The equipment tilt alarm refers to the inclinometer outputting the current tilt angle θ along the vertical direction of the camera and the tilt angle φ along the horizontal direction of the camera in real time. After each measurement cycle, the system compares the current reading with the change of the initial reference. When the change is greater than the set threshold, the equipment tilt alarm is triggered, and the three-dimensional displacement calculation result of the current measurement cycle is marked as low confidence failure data. Equipment settlement alarm refers to the fact that all or most of the remote targets exhibit relative displacement ΔY in the Y direction with similar magnitudes in the same direction in the equipment coordinate system. The system uses this feature to indirectly detect equipment settlement; after each patrol cycle, the ΔY values ​​of all ACTIVE targets are compared. If all or most of the ACTIVE targets have the same ΔY direction and similar amplitude, the equipment settlement alarm will be triggered, and the three-dimensional displacement calculation results of the current measurement cycle will be marked as low-confidence failure data. The target loss alarm refers to the alarm being triggered when the target status is marked as LOST in the pre-detection step S32 before the switching of the multi-target rotation process.

7. The method according to any one of claims 1-4 and 6, characterized in that, The path planning method includes the following steps: (1) Take the current gimbal position as the starting point; (2) Calculate the rotation angle distance from the starting point to each unvisited target, and select the closest one as the next target; (3) Drive the gimbal to the reference pointing angle of the target and perform single target measurement; (4) Mark the target as visited, and use its pointing angle as the new starting point to repeat steps (2) and (3) until all targets have been visited.

8. The method according to claim 7, characterized in that, The distance of the gimbal rotation angle between targets Defined as the Euclidean distance between the reference pointing angles of two targets: in, and Here, the reference pointing angles are for targets i and j, respectively. The difference in gimbal rotation angle between adjacent targets is estimated using a small-angle approximation formula: , ; In the formula, , fx and fy represent the pixel coordinate difference between the two targets in the image, and fx and fy are the equivalent focal lengths of the camera, in pixels.

9. The method according to any one of claims 1-4, 6, and 8, characterized in that, The single-target measurement process involves acquiring images from a single target and outputting the three-dimensional displacement of the target's gravity coordinate system, including the following steps: (1) The target image is acquired using an imaging device. The target image includes a black and white target pattern and a laser spot. The image is copied and input into parallel preprocessing channels, namely the target extraction channel and the laser spot extraction channel. Bilateral filtering is performed in the target extraction channel to reduce noise, and adaptive local mean threshold segmentation is used to convert the black and white boundary into a high-contrast binary image of the target. In the laser spot extraction channel, global threshold segmentation is used to filter out the background and output a binary image containing only the core domain of the spot. (2) In the image output by the target extraction channel, track and lock the outermost maximum closed contour within the target RoI; perform polygon simplification approximation on the contour to initially extract the vertex coordinates of the target at the current moment; perform sub-pixel level fine localization on each detected coarse corner point to obtain the corner coordinates with sub-pixel precision distortion removal; (3) Calculate the displacement of the target corner pixel: Let the initial state of the target corner pixel coordinates be... Currently Then pixel displacement , The average displacement of the four corner points is taken as the displacement of the entire pixel of the single target. , ; (4) Detect the pixel position of the laser spot in the current image and calculate the Euclidean distance between the center of the spot and the center of the target RoI. and the calculated Euclidean distance The set Euclidean distance threshold The positions of the light spots are compared to determine the position of the light spot until it is determined that the light spot is aligned with the center of the target. (5) The laser displacement meter emits a laser and receives the reflected signal to perform laser ranging to measure the distance to the target. ; (6) Combine the data obtained in the preceding steps with the equipment attitude data output by the inclinometer to form a three-dimensional displacement vector in the gravity coordinate system.