A structural displacement monitoring method and system based on a zooming pan-tilt camera

Abstract

The application discloses a kind of structural displacement monitoring method and system based on zooming head camera, method includes: along the target measurement area of the structure to be monitored different position of longitudinal arrangement measuring point target;With electric steering engine control zooming camera is turned, identifies all target of target structure;Based on the mapping relationship between pixel coordinates and space coordinates in each target pattern known dot coordinates and perspective geometric constraint establishes self-calibration model, correction;Visual displacement monitoring is carried out, and corresponding displacement time series data is generated;Realize monitoring process self-feedback and self-correction;According to the order of measuring point, vibration video at all measuring point positions is cyclically collected, and displacement response data of measuring point at different positions of structure is calculated.The application can expand the field of view by turning the camera, and expand the depth of field by zooming. Combined with deep learning, the target can be automatically identified. The application realizes large-scale, multi-point structural displacement automatic replacement monitoring, and improves the measurement efficiency and accuracy.

Description

Technical Field

[0001] This invention belongs to the field of visual measurement and relates to the technology of monitoring the health of civil engineering structures, specifically to a method and system for monitoring structural displacement based on a zoom gimbal camera. Background Technology

[0002] Currently, visual measurement methods are increasingly being applied to vibration monitoring of large structures such as bridges and buildings due to their advantages of being non-contact and easy to deploy. Most existing vision-based structural displacement measurement methods employ fixed cameras for long-term monitoring of a specific location on the structure. However, limitations in camera resolution and depth of field prevent them from simultaneously achieving high precision and wide-area measurement. With the increasing demand for intelligent construction and structural health monitoring, there is an urgent need for a visual measurement method and system capable of multi-point, high-precision, and automated monitoring to improve measurement efficiency and reduce the burden of manual operation. Summary of the Invention

[0003] Purpose of the invention: To address the problem that existing visual measurement technologies are limited by camera depth of field and cannot achieve high-precision global displacement measurement of large structures, this invention provides a structural displacement monitoring method and system based on a zoom gimbal camera. By rotating the camera to expand the field of view and zooming to expand the depth of field, and by combining deep learning to automatically identify targets, this invention enables automatic succession monitoring of structural displacement over a wide range and at multiple measurement points, thereby improving measurement efficiency and accuracy.

[0004] Technical Solution: To achieve the above objectives, this invention provides a method for monitoring structural displacement based on a zoom gimbal camera, comprising the following steps:

[0005] S1: Set up measurement point targets at different positions along the longitudinal direction of the measurement area of ​​the monitored structural target;

[0006] S2: Uses an electric servo motor to control the zoom camera to turn and scans and monitors all targets in the target structure across the entire focal length range;

[0007] S3: During camera turning and zooming, identify the target and its number, and record the servo motor angle and focal length information at this time;

[0008] S4: After completing all target recognition, a self-calibration model is established based on the known dot coordinates and perspective geometric constraints in each target pattern. The model compensates in real time for changes in the relative pose of the camera and the target scale factor of the measurement point position caused by zoom and angle changes, and corrects the mapping relationship between pixel coordinates and spatial coordinates.

[0009] S5: Based on the target number sequence or the acquisition priority set by the system, visual displacement monitoring is performed on each measuring point in sequence to generate corresponding displacement time series data;

[0010] S6: Real-time evaluation of image clarity, target detection confidence, and servo motor execution stability during monitoring, enabling self-feedback and self-correction during the monitoring process;

[0011] S7: Based on the servo angle and camera focal length information of each target recorded in the first scan, collect vibration videos of all measurement points in a loop according to the measurement point sequence, and generate multi-point displacement response data.

[0012] Furthermore, in step S1, two types of dots are set on the target: a coding point located at the center of the target and self-calibration points arranged at equal intervals around the coding point. Different targets are configured with different arc-shaped marks. The coding point and the arc-shaped marks constitute the coding mark. Each coding mark corresponds to a different unique code value, which is used to identify the measurement point number. The self-calibration points are used for camera self-calibration.

[0013] Furthermore, in step S3, a deep learning target detection algorithm is used to automatically search, locate, and focus on targets that can be clearly imaged within the field of view, and identify the target number.

[0014] Furthermore, in step S4, the self-calibration model identifies the position of the self-calibration point on each target and the known physical size position information, thereby realizing automatic update of the pixel scale factor under focal length changes. At the same time, it identifies the relative pose between the camera and the target, and calculates the displacement result through geometric position relationship when the relative angle between the camera imaging plane and the target plane is greater than a set value.

[0015] Furthermore, the self-calibration model operation in step S4 includes:

[0016] Constructing a basic projection model:

[0017]

[0018] in, This represents the three-dimensional coordinates of the target in the structural space coordinate system. This represents the pixel coordinates of the target in the image plane. As a scaling factor, This is the camera's internal parameter matrix, along with the focal length. Related; For the horizontal angle of the servo motor and vertical angle The determined rotation matrix, The translation vector of the camera's optical center in space;

[0019] In a structural displacement measurement scenario, assuming that each measuring point only generates displacement parallel to the target plane, and the displacement amplitude of the measuring point is less than the distance from the camera to the measuring point, a homography relationship is formed between the real physical coordinate plane of the measuring point and the pixel coordinate plane:

[0020]

[0021] By identifying and matching the pixel coordinates of self-calibrated points on a physical target at a specific focal length, the homography transformation matrix H, which represents the homography relationship, is solved globally.

[0022] Furthermore, in step S4, the new relative physical coordinates of the measuring point during the measurement process... Or changes in physical coordinates Through its new pixel coordinates Or pixel coordinate changes The calculation, specifically expressed as follows:

[0023] or .

[0024] Furthermore, in step S6, when target occlusion, abnormal focus, or servo deviation is detected and evaluated, refocusing, rescanning, or servo correction operations are automatically triggered to achieve self-feedback and self-correction in the monitoring process.

[0025] Furthermore, the self-feedback execution in step S6 includes:

[0026] When a decrease in image sharpness is detected, a micro-focusing operation is automatically performed;

[0027] When a target is detected to be missing or obstructed, the servo angle is automatically adjusted to search again.

[0028] The present invention also provides a structural displacement monitoring system based on a zoom gimbal camera, comprising:

[0029] A zoom gimbal camera is used to acquire video images of target structures on their surfaces from different directions and focal lengths.

[0030] Electric servo motor, used to control the steering of the zoom gimbal camera;

[0031] Targets are used to locate different measurement points;

[0032] The data processing unit is used to realize displacement monitoring and self-feedback and self-correction during the monitoring process.

[0033] Furthermore, the data processing unit includes:

[0034] The target detection module is used for target detection and identification.

[0035] The camera self-calibration module is used to compensate for imaging distortion and scaling changes caused by zoom and angle of view changes in real time, and to correct the mapping relationship between pixel coordinates and spatial coordinates.

[0036] The displacement measurement module is used to calculate displacement data;

[0037] The anomaly detection module is used to perform anomaly detection.

[0038] Servo control module, used to control electric servo motors;

[0039] Servo calibration module, used to calibrate electric servos.

[0040] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0041] 1. Camera turning is achieved through gimbal servo control, significantly expanding the monitoring range;

[0042] 2. The zoom function expands the depth of field coverage, ensuring accurate measurement of both near and far measurement points;

[0043] 3. Utilize deep learning target detection algorithms to achieve automatic target identification, reduce manual intervention, and improve the degree of automation in measurement;

[0044] 4. It can realize automatic succession monitoring of structural displacement over a large area and at multiple measurement points, while taking into account both measurement accuracy and coverage, providing an efficient and reliable measurement method for long-term health monitoring of large structures such as bridges and buildings. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the architecture of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of the coded mark; Detailed Implementation

[0047] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0048] Example 1:

[0049] like Figure 1 As shown, this embodiment provides a structural displacement monitoring method based on a zoom gimbal camera, including the following steps:

[0050] S1: Set up measurement point targets at different positions along the longitudinal direction of the measurement area of ​​the monitored structural target;

[0051] The target can be placed at the measurement point of the structure of interest. The target must face the camera and be in line with the camera (i.e., the camera can capture the target).

[0052] The target has two types of dots: a coding point located at the center of the target and self-calibration points arranged at equal intervals around the coding point, such as... Figure 2 As shown, different targets are equipped with different arc-shaped marks. The coding points and arc-shaped marks constitute the coding marks. Each coding mark corresponds to a different unique code value, which can be used for measurement point number identification. In this embodiment, each target is equipped with 12 self-calibration points. The 12 self-calibration points are distributed around the coding points in a square structure. The coding marks are used to identify the measurement point number, and the 12 self-calibration points are used to calculate and identify the angle between the camera imaging plane and the target plane (i.e., camera self-calibration).

[0053] S2: Use an electric servo motor to control the zoom camera to turn horizontally or vertically, and scan all targets within the visible area of ​​the target structure across the entire focal length range;

[0054] S3: During camera turning and zooming, a deep learning target detection algorithm is used to automatically search, locate and focus on targets that can be clearly imaged within the field of view, identify the target number and record the servo motor rotation angle and focal length information at this time.

[0055] Deep learning object detection algorithms include SSD, YOLO, or other applicable deep learning algorithms for object detection;

[0056] S4: After completing all target recognition, a self-calibration model is established based on the known dot coordinates and perspective geometric constraints in each target pattern. The model compensates in real time for changes in the relative pose of the camera and the target scale factor of the measurement point position caused by zoom and angle changes, and corrects the mapping relationship between pixel coordinates and spatial coordinates.

[0057] The self-calibration model automatically updates the pixel scale factor under focal length changes by identifying the position of the self-calibration point on each target and the known physical size position information. It can also identify the relative pose between the camera and the target. When the relative angle between the camera imaging plane and the target plane is large, more accurate displacement results can be obtained by calculating the geometric position relationship.

[0058] In a zoom gimbal camera system, the mapping relationship between the pixel coordinate system and the spatial coordinate system changes with the focal length and the servo motor angle adjustment. The self-calibration model, without relying on an external calibration plate, automatically identifies the position information of multiple white dots with known spatial geometric relationships distributed on the target, uses them as constraint points to perform perspective calculations, dynamically reconstructs the projection matrix, and obtains proportional information that can be used to calculate the physical displacement of the structure.

[0059] Specifically, for the basic projection model:

[0060]

[0061] This represents the three-dimensional coordinates of the target in the structural space coordinate system. This represents the pixel coordinates of the target in the image plane. It is a scaling factor (homogeneous coordinate scale). This is the camera's internal parameter matrix, along with the focal length. Related, For the horizontal angle of the servo motor and vertical angle The determined rotation matrix, The translation vector of the camera's optical center in space.

[0062] In structural displacement measurement scenarios, assuming that each measuring point only generates displacement parallel to the target plane, and the displacement amplitude of the measuring point is much smaller than the distance from the camera to the measuring point, then a homography relationship can be formed between its true physical coordinate plane and the pixel coordinate plane.

[0063]

[0064] By identifying and matching the pixel coordinates of self-calibrated points on a physical target at a specific focal length, the homography transformation matrix H representing this homography relationship can be solved globally. During the measurement process, the new relative physical coordinates of the measuring points are... Or changes in physical coordinates It can be obtained through its new pixel coordinates Or pixel coordinate changes calculate:

[0065] or

[0066] S5: Based on the target number sequence or the acquisition priority set by the system, visual displacement monitoring is performed on each measuring point in sequence to generate corresponding displacement time series data;

[0067] S6: During the monitoring process, image clarity, target detection confidence and servo execution stability are evaluated in real time. When target occlusion, abnormal focus or servo deviation is detected, refocusing, rescanning or servo correction operations are automatically triggered to realize self-feedback and self-correction in the monitoring process.

[0068] In this embodiment, image clarity, target detection confidence, and servo motor execution stability can all be evaluated by setting thresholds, thereby ensuring the normal progress of the measurement process.

[0069] Target occlusion, abnormal focus, or servo deviation correspond to the condition that the displacement result cannot be calculated normally. The algorithm will report an error in the calculation result of this frame, and then start refocusing and measurement.

[0070] The self-feedback process includes:

[0071] When a decrease in image sharpness is detected, a micro-focusing operation is automatically performed;

[0072] When a target is detected to be missing or obstructed, the servo angle is automatically adjusted to search again.

[0073] Self-correction refers to the refocusing and measurement after servo motor correction and camera focus adjustment.

[0074] S7: Based on the servo angle and camera focal length information of each target recorded in the first scan, collect all vibration videos in a loop according to the measurement point sequence, and generate multi-point displacement response data.

[0075] Example 2:

[0076] Reference Figure 1 This embodiment provides a structural displacement monitoring system based on a zoom gimbal camera, including:

[0077] A zoom gimbal camera is used to acquire video images of target structures on their surfaces from different directions and focal lengths.

[0078] The electric servo motor is used to control the rotation of the zoom gimbal camera, enabling the camera to rotate 360° horizontally and adjust ±90° vertically.

[0079] Targets are used to locate different measurement points;

[0080] The data processing unit is used to realize displacement monitoring and self-feedback and self-correction during the monitoring process.

[0081] The data processing unit includes:

[0082] The target detection module is used for target detection and identification.

[0083] The camera self-calibration module is used to compensate for imaging distortion and scaling changes caused by zoom and angle of view changes in real time, and to correct the mapping relationship between pixel coordinates and spatial coordinates.

[0084] The displacement measurement module is used to calculate displacement data;

[0085] The anomaly detection module is used to perform anomaly detection, including target occlusion, focusing anomalies, or servo deviation.

[0086] Servo control module, used to control electric servo motors;

[0087] Servo calibration module, used to calibrate electric servos.

Claims

1. A method for monitoring structural displacement based on a zoom gimbal camera, characterized in that, Includes the following steps: S1: Set up measurement point targets at different positions along the longitudinal direction of the measurement area of ​​the monitored structural target; S2: Uses an electric servo motor to control the zoom camera to turn and scans and monitors all targets in the target structure across the entire focal length range; S3: During camera turning and zooming, identify the target and its number, and record the servo motor angle and focal length information at this time; S4: After completing all target recognition, a self-calibration model is established based on the known dot coordinates and perspective geometric constraints in each target pattern. The model compensates in real time for changes in the relative pose of the camera and the target scale factor of the measurement point position caused by zoom and angle changes, and corrects the mapping relationship between pixel coordinates and spatial coordinates. S5: Based on the target number sequence or the acquisition priority set by the system, visual displacement monitoring is performed on each measuring point in sequence to generate corresponding displacement time series data; S6: Real-time evaluation of image clarity, target detection confidence, and servo motor execution stability during monitoring, enabling self-feedback and self-correction during the monitoring process; S7: Based on the servo angle and camera focal length information of each target recorded in the first scan, collect vibration videos of all measurement points in a loop according to the measurement point sequence, and generate multi-point displacement response data; In step S4, the self-calibration model identifies the position of the self-calibration point on each target and the known physical size position information, thereby achieving automatic update of the pixel scale factor under focal length changes. Simultaneously, it identifies the relative pose between the camera and the target, and calculates the displacement result through geometric positional relationships when the relative angle between the camera's imaging plane and the target plane is greater than a set value. The self-calibration model operation in step S4 includes: Constructing a basic projection model: ； in, This represents the three-dimensional coordinates of the target in the structural space coordinate system. This represents the pixel coordinates of the target in the image plane. As a scaling factor, This is the camera's internal parameter matrix, along with the focal length. Related; For the horizontal angle of the servo motor and vertical angle The determined rotation matrix, The translation vector of the camera's optical center in space; In a structural displacement measurement scenario, assuming that each measuring point only generates displacement parallel to the target plane, and the displacement amplitude of the measuring point is less than the distance from the camera to the measuring point, a homography relationship is formed between the real physical coordinate plane of the measuring point and the pixel coordinate plane: ； By identifying and matching the pixel coordinates of self-calibrated points on a physical target at a specific focal length, the homography transformation matrix H, which represents the homography relationship, is solved globally. The new relative physical coordinates of the measuring point during step S4. Or changes in physical coordinates Through its new pixel coordinates Or pixel coordinate changes The calculation, specifically expressed as follows: or .

2. The structural displacement monitoring method based on a zoom gimbal camera according to claim 1, characterized in that, In step S1, two types of dots are set on the target: a coding point located at the center of the target and self-calibration points arranged at equal intervals around the coding point. Different targets are equipped with different arc-shaped marks. The coding point and the arc-shaped marks constitute the coding mark. Each coding mark corresponds to a different unique code value, which is used to identify the measurement point number. The self-calibration points are used for camera self-calibration.

3. The structural displacement monitoring method based on a zoom gimbal camera according to claim 1, characterized in that, In step S3, a deep learning target detection algorithm is used to automatically search, locate, and focus on targets that can be clearly imaged within the field of view, and identify the target number.

4. The structural displacement monitoring method based on a zoom gimbal camera according to claim 1, characterized in that, In step S6, when target obstruction, abnormal focus, or servo deviation is detected and evaluated, refocusing, rescanning, or servo correction operations are automatically triggered to achieve self-feedback and self-correction in the monitoring process.

5. The structural displacement monitoring method based on a zoom gimbal camera according to claim 4, characterized in that, The self-feedback execution in step S6 includes: When a decrease in image sharpness is detected, a micro-focusing operation is automatically performed; When a target is detected to be missing or obstructed, the servo angle is automatically adjusted to search again.

6. A structural displacement monitoring system based on a zoom gimbal camera, characterized in that, For implementing the method of claim 1, the system comprises: A zoom gimbal camera is used to acquire video images of target structures on their surfaces from different directions and focal lengths. Electric servo motor, used to control the steering of the zoom gimbal camera; Targets are used to locate different measurement points; The data processing unit is used to realize displacement monitoring and self-feedback and self-correction during the monitoring process.

7. The structural displacement monitoring system according to claim 6, characterized in that, The data processing unit includes: The target detection module is used for target detection and identification. The camera self-calibration module is used to compensate for imaging distortion and scaling changes caused by zoom and angle of view changes in real time, and to correct the mapping relationship between pixel coordinates and spatial coordinates. The displacement measurement module is used to calculate displacement data; The anomaly detection module is used to perform anomaly detection. Servo control module, used to control electric servo motors; Servo calibration module, used to calibrate electric servos.

Images