A method and system for monitoring water engineering displacement

Abstract

The application discloses a kind of water engineering displacement monitoring method and system, it is related to engineering monitoring: by monitoring component, the image sequence of multiple targets on the structure to be measured in different focusing positions is acquired;Each target area is fused and handled;Target detection algorithm is used to carry out target identification to depth of field composite image;The target detection result of multiple time points obtained according to time sequence is processed by multi-target tracking, and the track correlation of each target at different time is established;According to the motion track of each target, the displacement of each target relative to the reference is calculated with the target position in the first image as reference. In the prior art, the dam surface displacement monitoring is limited by the camera depth of field, and a single set of visual monitoring equipment cannot simultaneously capture the point image at different distances of the dam body clearly, resulting in large data synchronization error in segmented monitoring by multiple devices. The application realizes the high-precision displacement monitoring of the whole section of the dam that can be completed by traditional multiple sets of equipment with a single set of equipment.

Description

Technical Field

[0001] This application relates to the field of engineering monitoring, and in particular to a method and system for monitoring displacement in water engineering projects. Background Technology

[0002] Currently, my country has constructed a vast number of reservoirs, dams, hydropower stations, and other water conservancy projects, forming a nationwide water engineering system. As crucial water infrastructure, the structural safety of dams directly impacts the safety of life and property and socio-economic development downstream. Traditional dam displacement monitoring methods primarily rely on manual inspections and contact sensors, which suffer from limitations such as limited monitoring points, poor real-time performance, and high labor costs. Particularly for high dams and large reservoirs, traditional monitoring methods struggle to achieve continuous monitoring of the entire dam surface, and their monitoring capabilities under extreme weather conditions are severely inadequate, becoming a significant bottleneck restricting the intelligent management of dam safety.

[0003] In recent years, machine vision technology has been applied as a non-contact measurement method in the field of dam displacement monitoring. Machine vision monitoring systems utilize multiple feature targets deployed on the dam surface, periodically capturing images of these targets with high-resolution industrial cameras. By combining image processing and pattern recognition algorithms, the displacement changes of each target are calculated, enabling automated monitoring of dam surface deformation. However, in practical applications, due to limitations in optical imaging principles, the depth of field of a single camera is limited. When monitoring a dam with a large span, targets at different distances cannot all be clearly imaged from the same focusing position. Specifically, when the camera focuses on a nearby target, distant targets are blurred; when focusing on a distant target, nearby targets are out of focus. This depth-of-field limitation means that a single vision monitoring device can only clearly capture target images of a localized area of ​​the dam, failing to achieve synchronous monitoring of the entire dam surface.

[0004] To cover the entire dam body, existing technologies typically employ a multi-camera segmented monitoring approach, where multiple monitoring devices are installed at different locations, each responsible for monitoring targets within a certain distance. However, this approach has significant drawbacks: first, it is difficult to ensure strict synchronization when multiple cameras acquire images, and inconsistencies caused by time differences affect the accuracy of displacement calculations; second, the accumulation of errors during the independent calibration and coordinate transformation of each camera reduces the overall monitoring accuracy; and third, the costs of equipment procurement, installation, and maintenance increase exponentially, significantly raising the construction and operation costs of the monitoring system.

[0005] Therefore, how to overcome the limitations of camera depth of field and achieve clear imaging and high-precision displacement monitoring of measuring points at different distances on the dam surface using a single set of equipment has become a technical problem that urgently needs to be solved in the field of dam safety monitoring. Summary of the Invention

[0006] In addressing the limitations of camera depth of field in existing dam surface displacement monitoring technologies, where a single visual monitoring device cannot simultaneously and clearly capture images of measuring points at different distances along the dam, resulting in large data synchronization errors due to segmented monitoring by multiple devices, this application provides a method and system for monitoring displacement in hydraulic engineering projects. This system enables a single monitoring device to simultaneously and clearly image measuring points at different distances along the entire dam body through focus bracketing and depth-of-field synthesis. Furthermore, it uses deep learning algorithms to accurately identify and track displacement changes at each measuring point, thereby achieving high-precision full-section displacement monitoring of the dam using a single device, a feat that traditionally requires multiple devices.

[0007] One aspect of this application provides a method for monitoring displacement in a hydraulic engineering project, comprising: S1, acquiring a sequence of multiple images of targets at different focus positions on a structure under test using a monitoring component; S2, extracting the corresponding target regions from each frame of images and fusing the target regions to obtain a depth-of-field composite image; S3, using a target detection algorithm to identify targets in the depth-of-field composite image, extracting the position coordinates of each target in the image, and obtaining the target detection result at the current moment; S4, performing multi-target tracking processing on the target detection results acquired in the time series, establishing the trajectory correlation between each target at different moments, and obtaining the motion trajectory of each target; S5, based on the motion trajectory of each target, using the target position in the first frame of the image as a reference, calculating the displacement of each target relative to the reference at each moment, and monitoring the structural deformation state of the dam based on the displacement; wherein the displacement includes horizontal and vertical displacement.

[0008] Among them, depth-of-field composite images refer to a single, clear image generated by fusing multiple frames of images of the same scene taken at different focusing distances. In dam displacement monitoring, because the targets on the dam are distributed at different distances from near to far, a single focusing shot cannot simultaneously capture all targets in sharp focus. By extracting the target area in optimal focus from each frame and reconstructing these sharp areas pixel-by-pixel according to their original spatial positions, a composite image is finally obtained that overcomes the physical depth-of-field limitations, making all targets at different distances clearly distinguishable.

[0009] Multi-target tracking refers to the process of simultaneously identifying, locating, and tracking multiple target objects in a continuous time series of images, establishing the identity correspondence between each target at different times. In dam monitoring scenarios, it is necessary to simultaneously track the motion status of dozens or even hundreds of targets on the dam body. By combining algorithms such as target detection, feature extraction, and motion prediction, each target can be accurately identified at each moment and correctly associated with targets at historical moments, forming an independent motion trajectory for each target. Even when targets are temporarily occluded or image quality degrades, the continuity and accuracy of tracking can be maintained.

[0010] Furthermore, the monitoring components include an industrial camera equipped with a telephoto lens and a focusing motor; multiple targets are arranged at preset distances in different areas of the structure to be measured; the focusing motor drives the telephoto lens to change the focusing position, so that the industrial camera focuses on the targets at different distances and acquires images of each target on different focal planes.

[0011] Unlike ordinary cameras, industrial cameras have higher pixel resolution (usually over 20 million pixels), more stable image quality, and stronger environmental adaptability, enabling them to work continuously 24 hours a day in complex outdoor climates such as high temperature, low temperature, rain, and fog.

[0012] The focusing motor is a precision electric motor drive device mounted on the lens focusing mechanism. It can automatically control the rotation of the lens focusing ring according to a preset program to achieve precise adjustment of the focusing distance. In dam monitoring scenarios, the focusing motor receives control signals and drives the telephoto lens to perform continuous or step-by-step focusing scans across the entire distance range from the nearest target to the farthest target.

[0013] Images of targets on different focal planes refer to a sequence of images containing multiple targets, captured at different focal positions by changing the lens's focusing distance. Due to the physical limitations of optical depth of field, when the lens is focused at a specific distance, only targets within a certain range near that distance can be clearly imaged, while targets at other distances appear blurred to varying degrees. By driving the lens through the entire monitoring range using a focusing motor, a set of images can be obtained, in which some targets in each image are in sharp focus. This multi-focal-plane image sequence contains clear information of all targets at their respective optimal focusing distances, providing the necessary raw data for subsequent depth-of-field synthesis.

[0014] Furthermore, the target is an infrared target; the industrial camera is equipped with an infrared photosensitive element. This infrared photosensitive element is the core imaging device inside the industrial camera, capable of sensing and converting infrared light signals into electrical signals. It typically employs a CMOS or CCD sensor with high quantum efficiency in the near-infrared band (700-1000nm). In dam displacement monitoring, using an infrared photosensitive element in conjunction with an infrared target can effectively overcome environmental interference problems associated with visible light imaging. Even under adverse lighting conditions such as fog, backlight, and nighttime, the specific wavelength of infrared light emitted by the infrared target can still be clearly captured by the photosensitive element, forming a high-contrast target image. This active emission imaging method greatly improves the all-weather working capability of the monitoring system and the reliability of target identification.

[0015] Furthermore, S2, the target regions are fused, including: detecting the target region in each frame of the image sequence to determine the target position in each frame; calculating the sharpness score of each target region in each frame; for each target, selecting the image region with the highest sharpness score from multiple frames as the sharp region of the corresponding target; and stitching the sharp regions of all targets pixel by pixel according to the original position of the target in the image to generate a depth-of-field composite image containing all targets.

[0016] The sharpness score is a numerical indicator that quantitatively evaluates the focus quality of a specific target area in an image, used to determine whether the area is in optimal focus. During depth-of-field synthesis, the sharpness score, reflecting the image sharpness, is obtained by calculating characteristic parameters such as the gradient intensity, Laplacian operator response value, or high-frequency components of each target area in each frame of the image.

[0017] Pixel-level stitching refers to the process of aligning and merging multiple clear target areas from different frames of images according to their precise positions in the original image coordinate system, pixel by pixel.

[0018] Furthermore, the object detection algorithm uses the YOLO algorithm; the loss function expression of the YOLO algorithm is: ;in, S represents the total number of grids used to divide the feature map, where S is the grid side length. For YOLOv8, the default S=13. ; This is an indicator function; it is 1 if the i-th grid contains the target center, and 0 otherwise. This is an indicator function; it is 1 if the i-th grid does not contain the target center, and 0 otherwise. Normalized coordinates of the predicted bounding box center point, relative to the grid cell; These are the normalized coordinates of the center point of the true bounding box; The confidence score of the predicted bounding box represents the probability that the target exists. This represents the confidence level of the true label; it is 1 if the target exists, and 0 otherwise. This is the probability distribution vector of the predicted target's category; The one-hot encoded vector of the true category label; This is the weighting coefficient for location loss, used to balance the importance of location loss; a typical value is 5. This is the weighting coefficient for the confidence loss, balancing the importance of the confidence loss; a typical value is 1. Let be the cross-entropy loss function, used to calculate the classification loss.

[0019] Furthermore, S4, multi-target tracking processing is performed on the target detection results obtained from multiple time-series acquisitions, including: extracting the bounding box information of the target detection output from the depth-of-field synthetic image based on the YOLO algorithm, extracting the center coordinates and confidence score of the bounding box for each target, and constructing the target detection set at the current time t. Each test result Includes target location information and confidence level; for the target detection set Detection results for each target in The ReID algorithm is used to extract the feature vector of the corresponding target. ; Calculate the target detection result at the current time t eigenvectors The set of trajectories at the previous time t-1 Feature vectors of the target in the last frame of each trajectory Cosine similarity between ; For the set of trajectories at the previous moment For each trajectory in the dataset, the Kalman filter algorithm is used to predict the position coordinates of the corresponding trajectory at the current time t. ; Calculate the target detection position at the current time t Predicted position coordinates of the trajectory Mahalanobis distance between Assess the consistency of target motion; based on cosine similarity. and Mahal distance Construct the cost matrix C between target detection and trajectory, and the matrix elements... Let C represent the association cost between the i-th target detection result and the j-th trajectory; use the Hungarian algorithm to solve the bipartite graph optimal matching problem on the cost matrix C, determine the historical trajectory corresponding to each target detection result, and obtain the optimal matching pair set of detection-trajectory; update the trajectory set according to the optimal matching pair set.

[0020] Among them, the ReID algorithm, or Re-Identification algorithm, is a target feature extraction algorithm based on deep learning. In dam displacement monitoring, the ReID algorithm uses a convolutional neural network to perform deep feature learning on the image region of each target, extracting high-dimensional feature representations that can characterize the unique identity of the target. Even under conditions of changing illumination, partial occlusion, or degraded image quality, the ReID algorithm can still stably extract the essential features of the target, such as subtle differences in the target's geometric shape pattern, surface texture features, and infrared reflectance intensity distribution. This deep feature extraction capability enables the system to accurately distinguish numerous targets on the dam that look similar but are located differently.

[0021] Feature vector This is the high-dimensional numerical vector representation extracted by the ReID algorithm for the i-th target at time t, typically a 128-dimensional or 256-dimensional floating-point vector. In the feature space, feature vectors of the same target at different times are relatively close, while feature vectors of different targets are relatively far apart. By calculating the similarity between feature vectors (such as cosine similarity), the probability that two detected targets belong to the same target can be quantitatively assessed, thereby achieving target identity association across time frames.

[0022] Mahalanobis distance, in this scheme, is used to evaluate the degree of deviation between the actual detected position of the target and the predicted position by the Kalman filter. Unlike Euclidean distance, Mahalanobis distance takes into account the uncertainty of position prediction (characterized by the covariance matrix of the Kalman filter) and can more accurately reflect the degree of anomaly in the target motion.

[0023] The cost matrix C has rows corresponding to the targets detected at the current time and columns corresponding to the historical trajectories. It integrates information from two dimensions: appearance similarity (calculated by cosine similarity of ReID feature vectors) and motion consistency (calculated by Mahalanobis distance).

[0024] In particular, this application ingeniously integrates deep learning-based appearance feature extraction (ReID) and physical model-based motion prediction (Kalman filtering) to form a complementary dual verification mechanism. ReID learns the subtle appearance differences (such as shape, texture, and light reflection characteristics) of each target through a deep neural network, extracting a distinctive 128-dimensional feature vector even when targets are highly similar in appearance. Kalman filtering, on the other hand, establishes a state-space model based on the target's historical motion trajectory and estimates the target's motion trend through a prediction-update mechanism. By calculating appearance similarity (cosine similarity) and motion consistency (Mahaviran distance) and using the Hungarian algorithm to solve for the optimal match globally, it ensures that even in complex conditions such as temporary target occlusion, image blurring, or dense target distribution, the identity of each target can still be accurately identified and continuously tracked.

[0025] Furthermore, the trajectory set is updated based on the optimal matching pair set, including: for successfully matched detection-trajectory pairs, the current target detection result is added to the corresponding historical trajectory; for target detection results that do not match historical trajectories, a new target trajectory is initialized; for historical trajectories that do not match the current detection, they are marked as temporarily lost, and the corresponding trajectory is terminated if there are 3 consecutive frames without a match.

[0026] Further, in step S5, the displacement of each target relative to the reference frame at each time step is calculated, including: selecting the first frame in the time series image as the reference frame; comparing the subsequent depth-of-field composite image with the reference frame; extracting the position coordinates of each target at each time step based on the motion trajectory of each target obtained by multi-target tracking; and calculating the horizontal displacement ΔX and vertical displacement ΔY of each target in each frame relative to the corresponding target in the reference frame.

[0027] Furthermore, the structural deformation status of the dam is monitored based on the displacement, including: determining whether there are one or more targets that meet the warning conditions of ΔX>X or ΔY>Y; if so, the corresponding target number, location information and displacement are recorded, and a warning log is generated.

[0028] Another aspect of this application provides a dam displacement monitoring system, comprising: a target assembly including multiple targets, the multiple targets being arranged at preset distances in different areas of the structure to be measured; a monitoring assembly including an industrial camera equipped with a telephoto lens and a focusing motor, the focusing motor driving the telephoto lens to change the focusing position, enabling the industrial camera to acquire a multi-frame image sequence of each target on different focal planes; a depth-of-field synthesis module, which performs target region detection on each frame of the image sequence, calculates the sharpness score of each target region, selects the image region with the highest sharpness score for each target, and stitches the sharp regions of all targets pixel-level to generate a depth-of-field composite image; and a target detection module, which uses the YOLO algorithm to identify targets in the depth-of-field composite image. The system extracts the position coordinates of each target in the image to obtain the target detection result at the current moment; the multi-target tracking module processes the target detection results obtained in multiple time series, extracts the target feature vectors through the ReID algorithm, and establishes the trajectory association relationship of each target at different times by combining Kalman filter prediction and Hungarian algorithm; the displacement calculation module calculates the horizontal displacement ΔX and vertical displacement ΔY of each target relative to the reference at each moment based on the motion trajectory of each target and the target position in the first frame image; the early warning module determines whether the target displacement exceeds the warning value based on the preset horizontal deformation warning value X and vertical deformation warning value Y, and generates an early warning log when the limit is detected.

[0029] Compared to existing technologies, the advantages of this application are:

[0030] This application is the first to integrate focus bracketing logic control and depth-of-field fusion algorithms into a machine vision device for dam displacement monitoring, breaking through the physical depth-of-field limitations of traditional cameras. Traditional single-focus imaging is limited by the optical depth-of-field formula. Constraints exist, particularly in long-focus monitoring scenarios where depth of field is extremely limited. By using a focusing motor to drive the lens at different focusing positions according to preset logic, a sequence of images covering the entire depth of the dam is continuously acquired. Then, a depth-of-field fusion algorithm intelligently extracts the target area with the best clarity from each frame and performs pixel-level stitching and reconstruction, ultimately generating a composite image where targets at both near and far distances are clearly distinguishable. This "multi-focus temporal acquisition + intelligent fusion reconstruction" technical approach enables a single monitoring device to achieve the equivalent of multiple devices relaying monitoring with an ultra-large depth-of-field coverage capability. While ensuring monitoring accuracy, it reduces equipment costs and fundamentally resolves the contradiction between the long span of the dam and the limited depth of field of the camera. Attached Figure Description

[0031] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0032] Figure 1 This is an exemplary flowchart of a method for monitoring displacement in a water conservancy project, according to some embodiments of this application;

[0033] Figure 2 These are schematic diagrams illustrating the working scenario of a field monitoring system according to some embodiments of this application;

[0034] Figure 3 This is a simplified schematic diagram illustrating the working principle of displacement calculation in a displacement management center according to some embodiments of this application.

[0035] Explanation of the labels in the diagram:

[0036] 1. Intelligent industrial camera; 2. Power supply components; 3. Target components; 4. Displacement management platform; 5. Dam body. Detailed Implementation

[0037] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] Example 1

[0039] This application provides an intelligent displacement monitoring system and method designed for large-scale engineering projects such as dams, embankments, and slopes. The intelligent displacement monitoring system includes a monitoring component, a target component, and a displacement data management platform. The target component includes a feature target, which is installed on the structure to be measured. The monitoring component includes an intelligent industrial camera and a telephoto lens, and is installed at a position that allows observation of all targets. The monitoring component integrates a focus bracketing logic control module and a depth-of-field synthesis algorithm, which can synthesize multiple frames of images from different focus positions into a single image in which all measuring points at near and far are clearly distinguishable. This image is then uploaded to the displacement data management platform to accurately observe all measuring points at near and far. The displacement data management platform includes feature extraction, target tracking, and displacement calculation programs. By performing feature extraction, target tracking, and displacement calculation on the images uploaded by the monitoring system, the displacement data of each measuring point is calculated, achieving synchronous monitoring of the displacement of different measuring points on the engineering surface. The advantage of this application is that the equipment as a whole achieves synchronous monitoring of monitoring points at different locations in long-distance projects through the fusion of multiple algorithms, improves the deformation monitoring capability of a single set of equipment, and greatly reduces the monitoring cost of long-distance projects such as dams, embankments, and slopes.

[0040] Furthermore, the target component of the intelligent monitoring system should contain multiple targets, which should be placed at a certain distance at multiple characteristic locations in different areas of the building to be monitored. The intelligent industrial camera in the monitoring component should be able to identify the location of all characteristic targets.

[0041] Furthermore, the targets in this system are infrared targets, so that they can be identified by intelligent industrial cameras in harsh environments such as nighttime and rainy / foggy days.

[0042] Furthermore, the intelligent industrial camera in the monitoring component of this intelligent monitoring system is equipped with a telephoto lens to acquire clear images of distant targets, extending the maximum distance that a single intelligent industrial camera can monitor.

[0043] The intelligent monitoring system in this application also has the following features: the intelligent industrial camera in its monitoring component integrates a focus bracketing function module on the basis of traditional visual monitoring equipment. The focus bracketing function module includes a hardware module and algorithm control.

[0044] (1) The hardware module is a camera focusing motor. The camera drive motor drives the lens to change the focusing position and drives the camera to focus on targets at different distances to obtain high-resolution photos of each target on different focal planes.

[0045] (2) The algorithm control module is a logic scheduling algorithm. According to the frequency, accuracy, range and other requirements of specific engineering monitoring, the algorithm can autonomously adjust the number of shots taken by the intelligent industrial camera per unit time, the starting and ending positions of the focus point, and the degree of focus compensation to meet the specific needs of different engineering monitoring.

[0046] Furthermore, the intelligent industrial camera in the system's monitoring component integrates a depth-of-field synthesis algorithm. This algorithm combines multiple frames of images from different focus positions acquired by the intelligent industrial camera into a single image in which all measurement points, near and far, are clearly distinguishable, and then uploads this image to the displacement management center.

[0047] When the intelligent industrial cameras in the system's monitoring components perform depth-of-field synthesis, the resulting images should clearly show all feature targets. The intelligent displacement monitoring system uses numbers or letters to number all feature targets, and the displacement management system calculates the displacement of all target locations at the measurement points.

[0048] Furthermore, the intelligent monitoring system transmits depth-of-field composite photos to the displacement management platform via wireless 4G / 5G.

[0049] Furthermore, the displacement management platform in this intelligent monitoring system employs the YOLO series of algorithms for target detection. This algorithm can directly identify the predicted target category and bounding box coordinates on each grid cell of the feature map, achieving rapid feature target identification and coordinate reading, and enabling end-to-end rapid detection. Taking YOLOv8 as an example, its loss function consists of three parts: In the formula, The total number of grids used to divide the feature map (e.g., the default in YOLOv8) but ); This is an indicator function; it is 1 if the i-th grid contains the target center, and 0 otherwise. : Predict the normalized coordinates (relative to the grid cell) of the bounding box center point; These are the normalized coordinates of the center point of the true bounding box; : Confidence of the predicted bounding box (probability of the target existing); The confidence level of the true label (1 if the target exists, 0 otherwise); This is the probability distribution vector of the predicted target's category; : The one-hot encoded vector of the true category label; Weighting coefficients (typical values) to balance position loss and confidence loss ); This is the cross-entropy loss function used for classification tasks. The displacement management system can directly identify the position coordinates of all targets in multiple frames of images uploaded in chronological order using the YOLO algorithm.

[0050] Furthermore, when performing target tracking, the displacement management platform outputs a set of target trajectories in each frame, based on a given sequence of multiple image frames. The multi-target tracking (MOT) task can be formalized as follows: given a sequence of multiple image frames... Output the set of target trajectories in each frame. ,in For the first The goal is at a certain moment The bounding box, This represents the trajectory length. Data association is a crucial step in the entire process; it combines the detection results of the current frame. With existing trajectory Matching is performed. Key technologies for data association consist of appearance feature matching, motion consistency constraints, and association optimization algorithms. Regarding appearance features, this displacement management platform uses a ReID (Person Re-identification) model to extract feature vectors of the detected targets, and measures the similarity between targets by calculating cosine similarity. In hydraulic scenarios, targets on dams and sluices may be affected by various environmental factors, such as dust and rainwater, causing changes in their appearance features. Therefore, selecting a suitable ReID model is crucial for accurately extracting target features. The appearance features are extracted using the ReID model to extract the feature vectors of the detected targets. Calculate the cosine similarity: .

[0051] In this displacement management platform, motion consistency constraints employ Kalman filtering to predict trajectory positions. Motion consistency is evaluated by calculating the IoU (Intersection over Union) or Mahalanobis distance between the detection box and the predicted box. Mahalanobis distance takes into account the covariance of the data. In long-distance hydraulic engineering scenarios, when the project is subjected to external forces such as water flow impact, the motion state of the deployed targets may change significantly. Mahalanobis distance can more accurately reflect this change, thereby improving the accuracy of motion consistency constraints. The motion consistency constraints utilize Kalman filtering to predict trajectory positions. Calculate the IoU or Mahalanobis distance between the detection box and the predicted box. The Mahalanobis distance is calculated using the following formula: .

[0052] The Kalman filter state prediction equation is: In the formula, In order to be in Time prediction Time-state vector (including position, velocity, etc.); This is the state transition matrix, used to describe the dynamic equations of the motion model; for The estimated target state at time t; For the control input matrix (if there is an external control input, such as acceleration); To control the input vector (usually provided by an external sensor).

[0053] The association optimization algorithm constructs a cost matrix and uses the Hungarian algorithm to find the optimal match. In long-distance engineering scenarios, due to the presence of multiple targets and complex environmental interference, constructing an accurate cost matrix is ​​crucial for achieving precise target tracking. When multiple targets appear in the frame simultaneously and their trajectories intersect, the association optimization algorithm needs to be able to accurately match each detection result with the correct trajectory based on information such as appearance features and motion consistency. The association optimization algorithm constructs a cost matrix and uses the Hungarian algorithm to find the optimal match. The cost matrix is: In the formula, For the first The first detection and the first The association cost of a trajectory is such that the smaller the value, the higher the probability of association. The cosine similarity of appearance features ranges from [-1, 1] and needs to be normalized to the interval [0, 1]. The Mahalanobis distance between the detection box and the trajectory prediction box is used to characterize motion consistency. This is a weighting coefficient, usually taken as 0.7-0.9, emphasizing the importance of appearance.

[0054] The displacement management platform uses the aforementioned correlation optimization algorithm to track the target motion trajectory in multiple frames of images within a given time series.

[0055] Furthermore, the displacement management platform in this intelligent monitoring system adopts a fixed reference frame strategy when calculating displacement. That is, the first frame in the sequence is selected as the benchmark, and each subsequent frame image is compared with the fixed reference frame to calculate the horizontal displacement ΔX and vertical displacement ΔY between each frame and the reference frame, thereby completing the time series of displacement deformation monitoring.

[0056] Furthermore, the displacement management system in this intelligent monitoring system is equipped with a horizontal deformation warning value X and a vertical deformation warning value Y. If, during the monitoring process, the horizontal displacement generated by one or more targets exceeds the horizontal deformation warning value (i.e., ΔX > X warning) or the vertical displacement generated exceeds the vertical displacement deformation warning value (i.e., ΔY > Y warning), the displacement management system will immediately issue an alarm.

[0057] The advantage of this application lies in the fact that it is the first to apply multi-algorithm fusion in a machine vision deformation monitoring device for displacement monitoring. By integrating the focus bracketing logic control module and depth-of-field fusion, it solves the problem of insufficient depth of field in a single video displacement monitoring device, and greatly improves the monitoring capability of a single device for long-distance projects. This application can significantly reduce the cost of displacement monitoring in long-distance projects. A single monitoring device with integrated focus bracketing function can achieve the effect of multiple monitoring devices relaying each other, and complete the synchronous monitoring of targets at different distances.

[0058] Example 2

[0059] Please refer to Figure 2 As shown, this application discloses an intelligent displacement monitoring system and method for monitoring displacement deformation in long-distance engineering projects, referring to... Figure 1 As shown, this example only uses a dam to demonstrate the implementation method of the invention for monitoring the displacement and deformation of the dam crest surface. The specific implementation methods for other long-distance projects are similar to this case.

[0060] Please refer to Figure 2 As shown, the target component 3 in the intelligent displacement monitoring system should contain more than one feature target. Multiple targets [A, B, C, D] should be placed at the feature positions of the project to be measured. For example, in this case, all targets are placed on the surface of the top of the dam body 5 to monitor the displacement deformation of the dam top surface.

[0061] Please refer to Figure 1 As shown, the multiple targets [A, B, C, D] in the target component 3 of the intelligent displacement are spaced apart and placed in different areas of the dam body 5, so that the target monitoring area covers the entire dam top, and the targets are labeled with unique numbers or letters.

[0062] Please refer to Figure 2 As shown, the target in this system is an infrared target, which can be identified by the intelligent industrial camera in harsh environments such as night and rainy / foggy days.

[0063] Please refer to Figure 2 As shown, the monitoring component in this intelligent displacement monitoring system is an intelligent industrial camera 1. The intelligent industrial camera 1 should be installed on one side of the target component 3. The installation position of the intelligent industrial camera 1 should allow the intelligent industrial camera 1 to observe all feature components [A, B, C, D].

[0064] The intelligent industrial camera 1 integrates a focus bracketing logic control module, which can control the number of photos taken by the intelligent industrial camera 1 per unit time under the frequency and accuracy requirements of monitoring work. Driven by the camera focusing motor built into the intelligent industrial camera 1, it can identify the positions of all feature targets [A, B, C, D] at different distances and focus on them, continuously taking multiple high-resolution images of targets located at different focal plane positions.

[0065] Please refer to Figure 2 As shown, the intelligent industrial camera 1 uses a depth-of-field synthesis algorithm to combine multiple photos taken by the intelligent industrial camera 1 that are focused on different targets 3 into a single high-definition image in which all targets 3, whether near or far, are clearly visible.

[0066] Please refer to Figure 2As shown, the power supply component 2 and the intelligent industrial camera 1 form a system to complete the shooting work, and transmit the photos synthesized by depth of field to the displacement management platform 4 wirelessly (such as 4G or 5G).

[0067] Please refer to Figure 3 As shown, when calculating the displacement and deformation of the dam crest in this case, the Y-axis is established parallel to the dam extension direction, and the X-axis is established perpendicular to the dam extension direction.

[0068] Please follow Figure 3 The displacement management platform shown uses the YOLO series of algorithms to directly identify the type of feature target 3 in the feature map and the precise coordinates of all targets 3. Its loss function consists of three parts: In the formula, The total number of grids used to divide the feature map (e.g., the default in YOLOv8) but ); This is an indicator function; it is 1 if the i-th grid contains the target center, and 0 otherwise. : Predict the normalized coordinates (relative to the grid cell) of the bounding box center point; These are the normalized coordinates of the center point of the true bounding box; : Confidence of the predicted bounding box (probability of the target existing); The confidence level of the true label (1 if the target exists, 0 otherwise); This is the probability distribution vector of the predicted target's category; : The one-hot encoded vector of the true category label; Weighting coefficients (typical values) to balance position loss and confidence loss ); Let be the cross-entropy loss function, used in classification tasks.

[0069] The displacement management platform is based on a given sequence of multiple uploaded images in chronological order. Output the set of trajectories of target 3 in each frame. ,in For the first The goal is at a certain moment The bounding box, Given the trajectory length, the above data association optimization algorithm is used to track multiple targets 3 within the monitoring period.

[0070] Within a monitoring cycle, the displacement management system 5 uses the first image uploaded by the intelligent industrial camera 1 as a reference frame and records the coordinates of all targets in the reference frame as initial coordinates. For ease of explanation, taking a single target A as an example, the initial coordinates of target A are (XA_initial, YA_initial). In the second frame, the coordinates of target A are (XA1, YA1), in the third frame, the coordinates of target A are (XA2, YA2), and so on. Within a monitoring period, the coordinates of target A in the t-th frame are (XAt-1, YAt-1). Similar to target A, the displacement management system tracks and records the position coordinates of each feature target in each frame.

[0071] Please refer to Figure 3 As shown, when calculating the displacement deformation of the target, it is necessary to calculate both the horizontal displacement Δx and the vertical displacement Δy. The displacement calculation method of this system is a fixed reference frame strategy, that is, the first frame in the sequence is selected as the reference, and each subsequent frame image is compared with the fixed reference frame to calculate the displacement between each frame and the reference frame. Figure 3 As shown, the initial coordinates of the target are (X_initial, Y_initial), and the coordinates of the target in frame t are (X_t-1, Y_t-1). Therefore, the horizontal displacement ΔX = X_t-1 - X_initial and the vertical displacement ΔY = Y_t-1 - Y_initial generated by the target in the time corresponding to frame t are calculated. Thus, this intelligent displacement system can calculate and record the displacement of all feature targets in each frame, ultimately establishing a time series of displacement deformation, enabling synchronous monitoring of the deformation displacement of multiple targets by a single system.

[0072] Please refer to Figure 2 , three As shown, the displacement management system 5 in this system is equipped with a horizontal deformation warning value X and a vertical deformation warning value Y. During the monitoring process, if the horizontal displacement generated by one or more targets is greater than the horizontal deformation warning value (i.e., ΔX > X warning) or the vertical displacement generated is greater than the vertical deformation warning value (i.e., ΔY > Y warning), the displacement management system will immediately issue an alarm and immediately determine the target number, which is conducive to quickly determining the location and time of dangerous deformation.

[0073] The advantage of this application lies in the fact that it is the first to apply multi-algorithm fusion in displacement monitoring machine vision deformation monitoring equipment. By integrating the focus bracketing logic control module and depth fusion, it solves the problem of insufficient depth of field of a single video displacement monitoring device, and greatly improves the monitoring capability of a single device for long-distance projects. This application can significantly reduce the cost of displacement monitoring for long-distance projects. A single monitoring device with integrated focus bracketing function can achieve the effect of multiple monitoring devices relaying each other, and complete the synchronous monitoring of targets at different distances.

[0074] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for monitoring displacement in a water engineering project, characterized in that, include: S1, acquires a sequence of multiple frames of images at different focus positions on multiple targets on the structure under test through the monitoring component; S2, extract the corresponding target regions from each frame image, and perform fusion processing on each target region to obtain a depth-of-field composite image; S3, using a target detection algorithm to identify targets in the depth-of-field composite image, extracting the position coordinates of each target in the image, and obtaining the target detection result at the current moment; S4. Perform multi-target tracking processing on the target detection results obtained from multiple time-series acquisitions, establish the trajectory correlation between each target at different times, and obtain the motion trajectory of each target. S5. Based on the motion trajectory of each target, and taking the target position in the first frame image as a reference, calculate the displacement of each target relative to the reference at each moment, and monitor the structural deformation state of the dam based on the displacement; wherein, the displacement includes horizontal and vertical displacement.

2. The method for monitoring displacement in water engineering projects according to claim 1, characterized in that: The monitoring components include an industrial camera equipped with a telephoto lens and a focusing motor; Multiple targets are deployed at preset distances in different areas of the structure to be tested; The focusing motor drives the telephoto lens to change the focusing position, enabling the industrial camera to focus on targets at different distances and acquire images of targets on different focal planes.

3. The method for monitoring displacement in water engineering projects according to claim 2, characterized in that: The target is an infrared target; Industrial cameras are equipped with infrared sensors.

4. The method for monitoring displacement in water engineering projects according to claim 2, characterized in that: S2 performs fusion processing on each target region, including: Target region detection is performed on each frame of the image sequence to determine the target location in each frame. Calculate the sharpness score for each target region in each frame of the image; For each target, the image region with the highest sharpness score is selected from multiple frames as the sharp region of the corresponding target; The sharp areas of all targets are stitched together pixel by pixel according to the original positions of the targets in the image to generate a depth-of-field composite image containing all targets.

5. The method for monitoring displacement in water engineering projects according to claim 4, characterized in that: The target detection algorithm uses the YOLO algorithm; The loss function expression for the YOLO algorithm is: ； Where S is the grid side length; This is an indicator function; it is 1 if the i-th grid contains the target center, and 0 otherwise. This is an indicator function; it is 1 if the i-th grid does not contain the target center, and 0 otherwise. Normalized coordinates for predicting the center point of the bounding box; These are the normalized coordinates of the center point of the true bounding box; The confidence level of the predicted bounding box; The confidence level of the true label; This is the probability distribution vector of the predicted target's category; The one-hot encoded vector of the true category label; These are the weighting coefficients for the location loss; The weighting coefficients for confidence loss; This is the cross-entropy loss function.

6. The method for monitoring displacement of a water conservancy project according to any one of claims 2 to 5, characterized in that: S4 performs multi-target tracking processing on the target detection results obtained from multiple time-series acquisitions, including: Based on the bounding box information output by target detection in the depth-synthesized image using the YOLO algorithm, the center coordinates and confidence score of the bounding box for each target are extracted to construct the target detection set at the current time t. Each test result Includes target location information and confidence level; For target detection set Detection results for each target in The ReID algorithm is used to extract the feature vector of the corresponding target. ; Calculate the target detection result at the current time t. eigenvectors The set of trajectories at the previous time t-1 Feature vectors of the target in the last frame of each trajectory Cosine similarity between ; For the set of trajectories at the previous moment For each trajectory in the dataset, the Kalman filter algorithm is used to predict the position coordinates of the corresponding trajectory at the current time t. ; The target detection position at the current time t is calculated. Predicted position coordinates of the trajectory Mahalanobis distance between To assess the consistency of target movement; Based on cosine similarity and Mahal distance Construct the cost matrix C between target detection and trajectory, and the matrix elements... This represents the association cost between the detection result of the i-th target and the j-th trajectory; The Hungarian algorithm is used to solve the bipartite graph optimal matching problem of the cost matrix C, and the historical trajectory corresponding to each target detection result is determined to obtain the optimal matching pair set of detection-trajectory. Update the trajectory set based on the set of optimal matching pairs.

7. The method for monitoring displacement in water engineering projects according to claim 6, characterized in that: Update the trajectory set based on the set of optimal matching pairs, including: For a successfully matched detection-trajectory pair, the current target detection result is added to the corresponding historical trajectory; For target detection results that do not match historical trajectories, initialize new target trajectories; For historical trajectories that do not match the current detection, they are marked as temporarily lost. If there are 3 consecutive frames without a match, the corresponding trajectory is terminated.

8. The method for monitoring displacement in water conservancy projects according to claim 6, characterized in that: S5, calculate the displacement of each target relative to the reference at each time step, including: The first frame in the time series image is selected as the reference frame. Each subsequent frame of the synthesized depth-of-field image is compared with the baseline reference frame; Based on the motion trajectory of each target obtained from multi-target tracking, the position coordinates of each target at each time moment are extracted; Calculate the horizontal displacement ΔX and vertical displacement ΔY of each target in each frame relative to the corresponding target in the reference frame.

9. The method for monitoring displacement in water engineering projects according to claim 8, characterized in that: Monitoring the structural deformation state of the dam based on displacement includes: Determine whether there are one or more targets that satisfy the warning conditions: ΔX>X or ΔY>Y. If so, record the corresponding target number, location information, and displacement, and generate an early warning log.

10. A dam displacement monitoring system, characterized in that, include: The target assembly includes multiple targets, which are arranged at preset distances in different areas of the structure to be tested. The monitoring components include an industrial camera equipped with a telephoto lens and a focusing motor. The focusing motor drives the telephoto lens to change the focusing position, enabling the industrial camera to acquire a multi-frame image sequence of each target on different focal planes. The depth-of-field compositing module performs target region detection on each frame of the image sequence, calculates the sharpness score of each target region, selects the image region with the highest sharpness score for each target, and stitches the sharp regions of all targets pixel by pixel to generate a depth-of-field compositing image. The target detection module uses the YOLO algorithm to identify targets in the depth-of-field composite image, extracts the position coordinates of each target in the image, and obtains the target detection result at the current time. The multi-target tracking module processes the target detection results obtained from multiple time-series data, extracts target feature vectors using the ReID algorithm, and combines Kalman filter prediction and Hungarian algorithm to establish the trajectory correlation relationship between each target at different time points. The displacement calculation module calculates the horizontal displacement ΔX and vertical displacement ΔY of each target relative to the reference at each moment, based on the motion trajectory of each target and using the target position in the first frame image as the reference. The early warning module determines whether the target displacement exceeds the preset horizontal deformation early warning value X and vertical deformation early warning value Y, and generates an early warning log when an over-limit is detected.

Images