A dam settlement visual monitoring method and system based on deep learning

Abstract

The application discloses a kind of dam settlement visual monitoring method and system based on deep learning, belong to water conservancy engineering safety monitoring technical field.The method is by deploying binocular industrial camera group around dam to collect dam surface image, adopts improved YOLOv8 target detection algorithm to identify reference target, monitoring target and virtual target, realizes target center subpixel level positioning in combination with ESPCN subpixel convolution network and weighted centroid method;Through stereomatching algorithm, the three-dimensional coordinates of target are solved and the settlement is calculated;Ordinary Kriging interpolation method is used in combination with structure constraint weight correction to reconstruct dam global settlement field;Using the time series prediction model based on Transformer and the Isolation Forest algorithm, the settlement trend prediction and abnormal early warning are carried out.The system includes image acquisition unit, edge computing unit, data transmission unit, remote monitoring center and power supply unit.The application can improve the precision, stability and automation level of dam settlement monitoring in complex outdoor environment, and is suitable for long-term safety monitoring of various dams.

Description

Technical Field

[0001] This invention belongs to the field of water conservancy project safety monitoring technology, and specifically relates to a deep learning-based visual monitoring method and system for dam settlement. Background Technology

[0002] Dam settlement is an important indicator reflecting the structural stability and operational safety of the dam. Uneven settlement can lead to cracking of the dam body, expansion of seepage channels, and abnormal stress on auxiliary structures, which can seriously affect the overall safe operation of the dam.

[0003] Although the manual leveling method commonly used in existing projects has high accuracy, it has a long measurement cycle and a high degree of reliance on manual labor. A single full dam measurement usually takes several days to complete and is affected by weather, visibility conditions and on-site operating conditions, making it difficult to meet the needs of continuous and real-time monitoring.

[0004] GNSS automated monitoring can achieve high-frequency data acquisition, but the cost of single-point construction is high, and it is susceptible to multipath effects, ionospheric disturbances, and surrounding obstructions. The vertical measurement accuracy is difficult to consistently meet the millimeter-level settlement monitoring requirements of high dams. Contact sensors such as water pipe settlement meters are complex to install and maintain, are easily affected by dam deformation or construction environment, and are mostly used for single-point monitoring, making it difficult to comprehensively reflect the overall deformation pattern of the dam.

[0005] In recent years, non-contact monitoring technology based on machine vision has been applied in engineering structure monitoring. However, traditional image processing methods such as template matching and edge detection are quite sensitive to changes in lighting, rain, snow, fog, haze, target contamination, and viewing angle; some solutions rely on specially made high-contrast targets, which have high deployment and maintenance costs; most existing visual monitoring systems focus on single-point or a small number of measurement points, lacking the ability to reconstruct the settlement field of the entire dam body, and also lacking the ability to combine historical data for trend prediction and early warning.

[0006] Therefore, there is an urgent need for a visual monitoring method and system for dam settlement that can stably identify targets, achieve sub-pixel positioning, reconstruct the entire settlement field, and predict trends in complex outdoor environments. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a deep learning-based visual monitoring method and system for dam settlement, so as to overcome the shortcomings of existing dam settlement monitoring technologies in terms of adaptability to harsh environments, monitoring accuracy, deployment cost, full-area deformation characterization and trend early warning.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A deep learning-based visual monitoring method for dam settlement, comprising the following steps: S1. Deploy a binocular industrial camera group in a stable area around the dam, set up benchmark targets and monitoring targets on the dam surface, and select stable and identifiable natural features of the dam as virtual targets; perform internal and external parameter calibration on the binocular industrial camera group, and establish the mapping relationship between the image coordinate system, the camera coordinate system and the dam engineering coordinate system. S2. Acquire binocular images of the dam surface according to a preset acquisition frequency, and perform noise reduction, enhancement, distortion correction and illumination normalization on the binocular images to obtain a preprocessed image. S3. An improved YOLOv8 target detection algorithm is used to identify the baseline target, monitoring target and virtual target in the preprocessed image, and super-resolution reconstruction and weighted centroid calculation are performed on the identified target region to obtain the sub-pixel level coordinates of the target center; S4. Based on the stereo matching results of the binocular images, camera calibration parameters, and extrinsic parameter matrices, calculate the three-dimensional coordinates of each target in the dam engineering coordinate system; S5. Using the benchmark target as a stable reference, calculate the settlement based on the elevation changes of each monitoring target and virtual target at different times. S6. Reconstruct the settlement field of the entire dam body based on the discrete target settlement data, and constrain and correct the settlement field in combination with the dam body structural zoning and geological conditions; S7. Based on the target sedimentation time series, perform sedimentation trend prediction and anomaly detection. When the sedimentation amount or sedimentation rate meets the preset early warning conditions, output graded early warning information.

[0009] Preferably, in step S1, the reference targets are deployed in stable areas on both sides of the dam that are not affected by dam deformation, and the number is not less than 3; the monitoring targets are deployed in a grid pattern on the dam top and upstream and downstream dam surfaces, with a deployment spacing of 20m to 50m; the virtual targets include at least one of dam section joints, drainage holes, pre-embedded bolts, and permanent markers.

[0010] Preferably, in step S1, the intrinsic parameter calibration of the binocular industrial camera group is completed using the Zhang's calibration method to obtain the focal length, principal point coordinates, and lens distortion coefficients of the left and right cameras; the extrinsic parameter calibration is completed using the PnP algorithm combined with the known engineering coordinates of the reference target to obtain the rotation matrix and translation vector from the camera coordinate system to the dam engineering coordinate system, and the extrinsic parameter calibration results are verified using the reprojection error of the reference target according to a preset period.

[0011] Preferably, in step S2, the denoising is performed using Gaussian filtering, the enhancement is performed using adaptive histogram equalization, the distortion correction is performed based on camera intrinsic parameters and lens distortion coefficients, and the illumination normalization is performed using a multi-scale Retinex algorithm to reduce the impact of noise, shadows, and illumination abrupt changes on target recognition.

[0012] Preferably, in step S3, the improved YOLOv8 target detection algorithm is a detection network formed on the basis of the YOLOv8-s network. The detection network introduces CBAM attention mechanism modules after the C2f-3, C2f-4 and C2f-5 modules of the Backbone, and replaces the original PAN structure with a BiFPN bidirectional feature pyramid structure in the Neck part to perform multi-scale feature fusion detection of the benchmark target, the monitoring target and the virtual target.

[0013] Preferably, in step S3, the identified target region is reconstructed using a sub-pixel convolutional network based on ESPCN with a 4x super-resolution capability, and the sub-pixel coordinates of the target center are calculated based on the gray-level weights of the reconstructed target region. The calculation formula is as follows: ; In the formula, The horizontal subpixel coordinates of the target center. The vertical sub-pixel coordinates of the target center. The set of pixels within the target area. The image coordinates of the pixels within the target area. For pixels The corresponding grayscale value.

[0014] Preferably, in step S4, the SGBM stereo matching algorithm is used to calculate the disparity map between the left and right images, and combined with the focal length, baseline distance, intrinsic parameter matrix, extrinsic parameter matrix and sub-pixel level coordinates of the target center of the binocular camera, the three-dimensional coordinates of each reference target, monitoring target and virtual target in the dam engineering coordinate system are calculated.

[0015] Preferably, in step S5, the settlement amount is calculated according to the following formula: ; In the formula, For the first One target in Settlement at any given time For the first The elevation of the target at the initial moment For the first One target in The measured elevation at any given time. Target number, For monitoring time.

[0016] Preferably, in step S6, ordinary kriging interpolation is used to generate a global settlement field of the dam body based on the discrete target settlement data. The settlement to be interpolated by ordinary kriging interpolation is the sum of the products of the settlement of each discrete target and the corresponding kriging weight. For areas with impermeable walls, core walls, or other rigid structures, the structural constraint weight coefficient is set to 1.2 to 1.5 during the settlement field reconstruction process, and the structural constraint weight coefficient is used as the weighting coefficient of the neighborhood smoothing term of the interpolation points in the rigid structure area to post-process and correct the ordinary kriging interpolation results. For dam sections with poor geological conditions or large historical deformation, the density of interpolation points is increased to improve the accuracy of local settlement field expression.

[0017] Preferably, in step S7, a Transformer-based time-series prediction model is used to analyze the settlement time series of each target. The time-series prediction model includes an Encoder×6 and a Decoder×6. The input is the settlement data of the past 30 days and environmental auxiliary features, and the output is the settlement prediction value for the next 7 days. The environmental auxiliary features include at least one of temperature, humidity, light intensity and rainfall.

[0018] Preferably, in step S7, the isolated forest algorithm is used to detect anomalies in the real-time settlement data, and the warning level is determined according to at least one of the settlement amount threshold, settlement rate threshold, and predicted settlement deviation threshold. The graded warning information includes at least one of the warning location, warning level, abnormal target number, anomaly type, and handling suggestions.

[0019] A deep learning-based visual monitoring system for dam settlement employs the aforementioned deep learning-based visual monitoring method for dam settlement. It comprises an image acquisition unit, an edge computing unit, a data transmission unit, a remote monitoring center, and a power supply unit, all interconnected sequentially. Specifically: the image acquisition unit acquires binocular images of the dam surface and environmental data; the edge computing unit performs image preprocessing, target detection, sub-pixel localization, and 3D coordinate calculation on the binocular images; the data transmission unit transmits settlement data, processing results, and original images to the remote monitoring center; the remote monitoring center stores and manages monitoring data, reconstructs the entire dam settlement field, performs settlement trend prediction, and provides early warning of anomalies; and the power supply unit provides continuous power to the image acquisition unit, edge computing unit, data transmission unit, and remote monitoring center.

[0020] Preferably, the image acquisition unit includes two 20-megapixel global shutter industrial cameras, a 16mm fixed-focus lens, a waterproof and dustproof protective cover, an electric gimbal, and an environmental sensor; the environmental sensor includes at least one of a temperature sensor, a humidity sensor, a light sensor, and a rain sensor.

[0021] Preferably, the edge computing unit adopts an embedded computing platform, which has a built-in image preprocessing module, an improved YOLOv8 target detection module, an ESPCN subpixel localization module, an SGBM three-dimensional coordinate calculation module and a settlement calculation module, for generating target three-dimensional coordinates and settlement data locally in real time.

[0022] Preferably, the data transmission unit adopts a dual-link redundancy design of 5G wireless communication and optical fiber communication; the power supply unit adopts a hybrid power supply method of solar panels, lithium iron phosphate batteries and mains power backup.

[0023] Preferably, the remote monitoring center includes a server, a time-series database, a monitoring terminal, and a visualization platform. The visualization platform is built based on a three-dimensional digital twin model and is used to integrate and display real-time settlement monitoring data with the three-dimensional model of the dam body, and to display settlement cloud maps, contour maps, settlement process lines, and early warning information.

[0024] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the deep learning-based visual monitoring method for dam settlement.

[0025] The present invention can achieve the following beneficial effects: 1. This invention deploys a binocular industrial camera group in a stable area around the dam, and sets up benchmark targets, monitoring targets and selects virtual targets on the dam surface. It can acquire multi-point image information of the dam surface in a non-contact manner, reducing the dependence of manual leveling on on-site personnel, visibility conditions and measurement cycle. It is suitable for long-term continuous settlement monitoring of dams.

[0026] 2. This invention places the reference target in a stable area unaffected by dam deformation, and establishes a mapping relationship between the image coordinate system, camera coordinate system and dam engineering coordinate system through internal parameter calibration, external parameter calibration and reference target reprojection error verification. This can reduce the impact of camera attitude changes, support micro-movements or external parameter drift on the sedimentation calculation results.

[0027] 3. In the binocular image processing, the present invention sequentially performs denoising, enhancement, distortion correction and illumination normalization. Denoising is used to suppress random noise in the image, enhancement is used to improve the local contrast between the target and the background, distortion correction is used to reduce the influence of lens distortion, and illumination normalization is used to reduce the interference of shadows, strong reflections and sudden changes in illumination on the target identification, thereby improving the usability of image data in complex outdoor environments.

[0028] 4. This invention employs an improved YOLOv8 target detection algorithm to identify baseline targets, monitoring targets, and virtual targets. It also introduces a CBAM attention mechanism module into the backbone of the YOLOv8-s network and uses a BiFPN bidirectional feature pyramid structure in the neck part for multi-scale feature fusion. This enables the detection network to better extract features of distant small-sized targets, low-contrast targets, and partially occluded targets, thereby improving the stability of target recognition under rain, fog, low illumination, and complex dam surface background conditions.

[0029] 5. In addition to monitoring targets, this invention selects natural features of the dam body, such as dam section joints, drainage holes, pre-embedded bolts and permanent markers, as virtual targets. This increases the number of observation points on the dam surface that can participate in settlement calculations, reduces the number of monitoring targets, and improves the data density of local settlement field reconstruction.

[0030] 6. This invention uses the ESPCN subpixel convolutional network for super-resolution reconstruction of the identified target region, and calculates the gray-level weighted centroid based on the reconstructed target region to obtain the subpixel-level coordinates of the target center. Compared with directly using the bounding box center or integer pixel centroid, this method can reduce the impact of long-distance imaging, image blurring, and unclear target edges on the accuracy of center positioning.

[0031] 7. Based on the SGBM stereo matching algorithm, binocular camera focal length, baseline distance, intrinsic parameter matrix, extrinsic parameter matrix, and target center sub-pixel level coordinates, this invention calculates the three-dimensional coordinates of each benchmark target, monitoring target, and virtual target in the dam engineering coordinate system, enabling the image measurement results to be converted into spatial location data in engineering coordinates, providing a coordinate basis for subsequent elevation changes and settlement calculations.

[0032] 8. This invention uses a benchmark target as a stable reference and calculates the settlement based on the elevation changes of the monitored target and the virtual target at different times. It can correct the impact of camera coordinate changes, equipment installation errors and environmental disturbances on the measurement results through benchmark constraints, thereby improving the stability of the settlement calculation results.

[0033] 9. This invention uses ordinary Kriging interpolation to reconstruct the settlement field of the entire dam body based on discrete target settlement data, and combines the dam body structural zoning and geological conditions for constraint correction. It can expand the settlement data of limited measuring points to form a continuous settlement distribution on the dam body surface, which is convenient for identifying local abnormal settlement areas and the overall settlement trend.

[0034] 10. In this invention, the structural constraint weight coefficient is used as the post-processing smoothing correction coefficient of the interpolation result in the seepage barrier wall, core wall or other rigid structure area, and the density of interpolation points is increased in dam sections with poor geological conditions or large historical deformation, so that the settlement field reconstruction process can take into account the differences in dam structure stiffness and the characteristics of local risk areas, and improve the ability of the whole-domain settlement field to express the actual deformation state of different dam sections.

[0035] 11. This invention uses a Transformer-based time series prediction model to analyze the target settlement time series and takes environmental auxiliary features such as temperature, humidity, light and rainfall as inputs. It can predict future settlement trends and provide a data foundation for shifting dam settlement from post-event analysis to trend judgment.

[0036] 12. This invention uses the isolated forest algorithm to detect anomalies in real-time settlement data and determines the warning level based on the settlement amount threshold, settlement rate threshold, and predicted settlement deviation threshold. It can output graded warning information including warning location, warning level, abnormal target number, anomaly type, and handling suggestions when the settlement amount, settlement rate, or predicted deviation is abnormal.

[0037] 13. This invention completes image preprocessing, target detection, subpixel localization, three-dimensional coordinate calculation, and settlement calculation locally through an edge computing unit, which can reduce the bandwidth occupation of the original image full transmission and improve the real-time performance of on-site monitoring data processing.

[0038] 14. This invention enables the system to adapt to the long-term, continuous, and remote monitoring needs of dam sites through the coordinated work of the image acquisition unit, edge computing unit, data transmission unit, remote monitoring center, and power supply unit, combined with the dual-link redundancy design of 5G wireless communication and fiber optic communication, and the hybrid power supply method of solar energy, storage battery and mains power backup.

[0039] 15. This invention integrates and displays real-time settlement data, the entire dam settlement field, settlement contour maps, settlement process lines, and early warning information through a remote monitoring center and a three-dimensional digital twin visualization platform, making it easy for operation and maintenance personnel to intuitively grasp the spatial distribution, temporal changes, and abnormal locations of dam settlement. Attached Figure Description

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the overall structure of the monitoring system of the present invention; Figure 2 This is a flowchart illustrating the monitoring method of the present invention; Figure 3 This is a schematic diagram of the network structure of the improved YOLOv8 target detection algorithm of the present invention; Figure 4 This is a schematic diagram of the settlement time series prediction model based on Transformer of the present invention; Figure 5 This is a schematic diagram of the reconstruction results of the settlement field of the entire dam body according to the present invention.

[0041] In the figure: (a) is a three-dimensional settlement cloud map, (b) is a dam crest settlement contour map, and (c) is a settlement process line of a typical dam section. Detailed Implementation

[0042] like Figures 1 to 5 As shown, a deep learning-based visual monitoring method and system for dam settlement is applicable to concrete gravity dams, arch dams, earth-rock dams, and other hydraulic structures requiring long-term settlement safety monitoring. This method uses a binocular industrial camera system for non-contact image acquisition, with a set of settlement observation points comprised of a baseline target, a monitoring target, and a virtual target. Through image preprocessing, improved YOLOv8 target detection, ESPCN sub-pixel convolutional reconstruction, weighted centroid localization, SGBM stereo matching, 3D coordinate calculation, global settlement field reconstruction, and settlement trend prediction, automated, continuous, and visualized monitoring of dam settlement is achieved.

[0043] like Figure 1 As shown, the monitoring system includes an image acquisition unit, an edge computing unit, a data transmission unit, a remote monitoring center, and a power supply unit. The image acquisition unit collects binocular images of the dam surface and on-site environmental data; the edge computing unit performs image preprocessing, target detection, sub-pixel localization, 3D coordinate calculation, and settlement calculation; the data transmission unit transmits settlement data, processing results, and necessary raw images to the remote monitoring center; the remote monitoring center stores, analyzes, visualizes, and provides early warnings; and the power supply unit ensures continuous operation of on-site equipment. Figure 1 In the diagram, B1, B2, B3, and B4 all represent baseline targets, M represents monitoring targets, and V represents virtual targets.

[0044] S1. System deployment, target placement and coordinate calibration.

[0045] A binocular industrial camera system is deployed in a stable area surrounding the dam. This stable area is preferably located on stable bedrock on both banks, on independent observation piers, or in other locations unaffected by dam deformation. The binocular industrial camera system includes a left camera and a right camera, maintaining a fixed baseline between them. The optical axes of the lenses are pointed towards the target dam section, and the monitoring field of view is adjusted using a motorized pan-tilt unit or fixed support to cover the dam crest, upstream and downstream dam faces, or other settlement-sensitive areas.

[0046] Baseline targets and monitoring targets are deployed on the dam surface, and stable, identifiable natural features of the dam body are selected as virtual targets. Baseline targets are deployed in stable areas on both banks of the dam, with no fewer than three targets, to provide external parameter calibration, coordinate reference constraints, and long-term drift verification. Monitoring targets are deployed in a grid pattern on the dam crest and upstream and downstream faces, with a spacing of 20m to 50m, the specific spacing determined based on the dam length, dam height, important structural parts, and monitoring accuracy requirements. Virtual targets include dam section joints, drainage holes, embedded bolts, permanent markers, or other long-term stable natural features with clear image textures. By simultaneously using monitoring targets and virtual targets, the number of monitoring targets can be reduced, and the density of observation points in local areas of the dam body can be increased.

[0047] The binocular industrial camera system underwent intrinsic and extrinsic parameter calibration. Intrinsic parameter calibration was performed using Zhang's calibration method to obtain the focal length, principal point coordinates, and lens distortion coefficients of the left and right cameras. Binocular calibration was used to obtain the rotation matrix and translation vector between the left and right cameras. Extrinsic parameter calibration employed the PnP algorithm, combined with the known engineering coordinates of the reference target, to solve for the rotation matrix and translation vector from the camera coordinate system to the dam engineering coordinate system, thereby establishing the mapping relationship between the image coordinate system, the camera coordinate system, and the dam engineering coordinate system.

[0048] During long-term operation, the system verifies the external parameter calibration results using the reprojection error of the benchmark target according to a preset cycle. When the reprojection error exceeds a preset threshold, the system generates a calibration verification prompt and performs automatic recalibration or manual verification according to the site conditions to reduce the impact of support micro-motion, temperature deformation, lens offset and foundation settlement on monitoring accuracy.

[0049] S2. Binocular image acquisition and preprocessing.

[0050] Binocular images of the dam surface are acquired according to a preset acquisition frequency. The acquisition frequency can be determined based on the dam's operating status, weather conditions, and warning level; during normal operation, minute-level or hourly acquisition can be used, while the acquisition frequency can be increased during floods, heavy rainfall, seismic disturbances, or abnormal warning phases.

[0051] The acquired left and right images were subjected to denoising, enhancement, distortion correction, and illumination normalization to obtain preprocessed images. Denoising employed Gaussian filtering to suppress sensor noise and local random noise; image enhancement used adaptive histogram equalization to improve the local contrast of the target and natural feature regions; distortion correction was performed based on camera intrinsic parameters and lens distortion coefficients to eliminate the influence of radial and tangential distortion on the localization results; illumination normalization used a multi-scale Retinex algorithm to reduce the impact of shadows, rain, fog, strong reflections, and abrupt changes in illumination on target recognition.

[0052] The edge computing unit performs the above preprocessing locally and stores the preprocessed image, environmental sensor data, and acquisition timestamps together. The environmental sensor data includes at least one of temperature, humidity, light intensity, and rainfall, which can be used to adjust image preprocessing parameters and also serve as auxiliary features for subsequent subsidence trend prediction.

[0053] S3. Improved YOLOv8 target detection and subpixel localization.

[0054] like Figure 3 As shown, an improved YOLOv8 target detection algorithm is used to identify baseline targets, monitoring targets, and virtual targets in preprocessed images. This improved YOLOv8 target detection algorithm is based on the YOLOv8-s network. A CBAM attention mechanism module is introduced after the C2f-3, C2f-4, and C2f-5 modules of the backbone to enhance the network's attention to target regions, low-contrast edges, and local texture features. In the Neck part, a BiFPN bidirectional feature pyramid structure replaces the original PAN structure to achieve bidirectional fusion of features at different scales, thereby improving the detection stability of small-sized targets, distant targets, partially occluded targets, and virtual targets.

[0055] Figure 3 In this context, Conv CBS represents the basic feature extraction module consisting of convolutional layers, batch normalization layers, and SiLU activation functions; C2f-1 to C2f-5 represent five C2f feature extraction modules; SPPF represents the spatial pyramid pooling module; P3, P4, and P5 represent small-scale, medium-scale, and large-scale feature layers, respectively; and Head represents the multi-scale detection output layer, used to output target category, confidence level, and bounding box.

[0056] The YOLOv8 network outputs the category, confidence score, and bounding box for each target. For detection results of the same target in the left and right images, the system correlates them based on category, geometric constraints, epipolar constraints, and historical tracking information to eliminate false positives and low-confidence targets. For targets detected in multiple consecutive frames, target numbers can be maintained based on temporal consistency to ensure a one-to-one correspondence between the sedimentation time series and the actual targets.

[0057] For the detected target region, a 4x super-resolution reconstruction is performed using a sub-pixel convolutional network based on ESPCN. The ESPCN network extracts features in a low-resolution feature space and reconstructs the target region into a high-resolution local image through sub-pixel convolutional layers. The reconstructed target region is then used to perform weighted centroid calculation to obtain the sub-pixel coordinates of the target center.

[0058] The sub-pixel level coordinates of the target center are calculated according to the following formula:

[0059]

[0060] In the formula, The horizontal subpixel coordinates of the target center. The vertical sub-pixel coordinates of the target center. The set of pixels within the target area. The image coordinates of the pixels within the target area. This represents the grayscale value corresponding to the pixel.

[0061] Through the above processing, the target center coordinates are not directly taken from the bounding box center or the centroid of an integer pixel, but are determined based on grayscale weights within the target area after super-resolution reconstruction, thereby reducing the impact of long-distance imaging, motion blur, rain and fog interference, and local contamination on the center localization results.

[0062] S4. Stereo matching and 3D coordinate calculation.

[0063] Based on the stereo matching results of binocular images, camera calibration parameters, and extrinsic parameter matrices, the three-dimensional coordinates of each target in the dam engineering coordinate system are calculated. Specifically, the system uses the SGBM stereo matching algorithm to calculate the disparity map between the left and right images, and combines it with the sub-pixel level coordinates of the target center to obtain the corresponding disparity value. For monitored targets, local disparity near the target center is used preferentially; for virtual targets, disparity is filtered by combining texture matching results of natural feature regions and historical coordinate stability.

[0064] Based on the binocular camera's focal length, baseline distance, intrinsic parameter matrix, and disparity value, the target is first transformed from image coordinates to three-dimensional coordinates in the camera coordinate system, and then transformed to the dam engineering coordinate system using the extrinsic parameter matrix. For each target, the system generates a monitoring record containing the target number, target type, acquisition time, three-dimensional coordinates, location reliability, and environmental data.

[0065] In actual operation, the three-dimensional coordinates of the same target can be filtered across multiple frames. For short-term image jitter, local occlusion, or low-confidence detection results, the system can improve coordinate stability by using sliding window midpoint filtering, confidence-weighted averaging, or outlier removal. This processing does not change the basic logic of settlement calculation; it is only used to reduce the impact of random image errors on the calculation results of a single settlement event.

[0066] S5. Calculation of settlement based on benchmark target.

[0067] Using a baseline target as a stable reference, the system calculates settlement based on the elevation changes of each monitored target and virtual target at different times. During the initial monitoring phase, the system establishes an initial elevation baseline for each target, and at each subsequent monitoring time, it obtains the current elevation of the target based on the three-dimensional coordinate calculation results.

[0068] The settlement amount is calculated according to the following formula:

[0069] In the formula, For the first Each target at time The amount of settlement, For the first The elevation of the target at the initial moment For the first Each target at time The measured elevation, Target number, For monitoring time.

[0070] When the system detects a synchronous offset in the baseline target, it first calculates the coordinate baseline correction using the known engineering coordinates and current solution coordinates of the baseline target. This correction is then applied to the monitoring target and the virtual target to reduce the impact of minor changes in camera attitude, support temperature deformation, or external parameter drift on settlement calculations. For targets with missing data, the system records the missing status and marks the data quality during global settlement field reconstruction and trend prediction.

[0071] S6. Reconstruction and constraint correction of the settlement field throughout the dam body.

[0072] The system reconstructs the global settlement field of the dam body based on discrete target settlement data. The system uses the spatial locations and settlement amounts of monitored and virtual targets as discrete sampling points, and employs ordinary kriging interpolation to generate a continuous settlement field on the dam surface. Figure 5 As shown, the reconstruction results can be displayed through three-dimensional settlement cloud maps, dam crest settlement contour maps, and settlement process lines of typical dam sections.

[0073] Ordinary Kriging interpolation can be expressed as:

[0074] In the formula, Position to be interpolated The initial settlement estimate obtained by ordinary kriging interpolation is given. For the first The sedimentation of a discrete target For the first Kriging weights corresponding to discrete targets The number of discrete targets participating in the interpolation.

[0075] During the settlement field reconstruction process, constraint corrections are made in conjunction with the dam structure zoning and geological conditions. For areas with cutoff walls, core walls, or other rigid structures, the structural constraint weighting coefficients are used as post-processing smoothing correction coefficients for ordinary Kriging interpolation results, rather than simply providing parameter ranges.

[0076] Specifically, when the position to be interpolated Located in the rigid structure region Initially, the estimated settlement value is obtained using the ordinary kriging interpolation formula. And then in Preset neighborhood centered Select pre-generated interpolation points or discrete target points, and weight them according to the inverse distance ratio. Calculate neighborhood smoothing reference value ;in, for With neighboring points Spatial distance between them To avoid positive numbers with a denominator of zero.

[0077] Structural constraint weight coefficients Applying the neighborhood smoothing reference value, we obtain the constrained and corrected settlement. ,in Take a value between 1.2 and 1.5; when the location to be interpolated does not belong to the rigid structure region, Take 0, .

[0078] For the transition zone outside the boundary of the rigid structure region, The interpolation points are gradually reduced in distance from the location to be interpolated to the boundary of the rigid structure region to avoid abrupt changes in the settlement field at the boundary between the rigid structure region and the ordinary dam body region. For dam sections with poor geological conditions, large historical deformation, or high monitoring risks, the density of interpolation points is increased to improve the accuracy of local settlement field representation. For areas with dense virtual targets but few monitoring targets, data confidence weights are set based on the recognition stability of virtual targets and the accuracy of historical repeated observations to avoid mismatches of local natural features affecting the overall settlement field.

[0079] S7. Settlement trend prediction and anomaly early warning.

[0080] Settlement trend prediction and anomaly detection based on target settlement time series. For example... Figure 4 As shown, a Transformer-based time-series prediction model was used to analyze the settlement time series of each target. The time-series prediction model includes an Encoder×6 and a Decoder×6, with the input being settlement data from the past 30 days and environmental auxiliary features, and the output being the predicted settlement value for the next 7 days. The environmental auxiliary features include at least one of temperature, humidity, light intensity, and rainfall. Figure 4In this context, Encoder×6 represents a stacked 6-layer encoder, and Decoder×6 represents a stacked 6-layer decoder.

[0081] During the training phase, the remote monitoring center constructs training samples using historical settlement data, concurrent environmental data, and manual verification results. During the operational phase, the time-series prediction model outputs settlement prediction curves for each monitoring target and virtual target within future time windows. The system can identify continuous settlement trends, phased accelerated settlement trends, and short-term disturbance trends caused by environmental factors based on the prediction results.

[0082] Simultaneously, the isolated forest algorithm is employed to detect anomalies in real-time settlement data. The system determines the warning level based on at least one of the following: settlement amount threshold, settlement rate threshold, and predicted settlement deviation threshold. The tiered warning information includes warning location, warning level, anomaly target number, anomaly type, associated image, settlement process line, and handling recommendations. Warning levels can be set to blue, yellow, orange, and red, corresponding to concern, general anomaly, relatively serious anomaly, and severe anomaly, respectively.

[0083] When the real-time settlement exceeds the threshold but the prediction model does not show a continued trend, the system outputs a local anomaly warning and suggests reviewing the on-site images; when the real-time settlement, settlement rate and predicted settlement deviation are all abnormal at the same time, the system outputs a higher-level warning and highlights the abnormal dam section in the three-dimensional digital twin model.

[0084] A deep learning-based visual monitoring system for dam settlement, employing the aforementioned method, includes an image acquisition unit, an edge computing unit, a data transmission unit, a remote monitoring center, and a power supply unit.

[0085] The image acquisition unit includes two 20-megapixel global shutter industrial cameras, a 16mm fixed-focus lens, a waterproof and dustproof protective cover, a motorized pan-tilt head, and an environmental sensor. The two industrial cameras form a binocular industrial camera group, used to acquire left and right binocular images of the dam surface; the waterproof and dustproof protective cover ensures stable operation of the equipment in rain, fog, high humidity, strong light, and dusty environments; the motorized pan-tilt head is used to adjust the monitoring field of view; and the environmental sensor is used to collect on-site environmental data such as temperature, humidity, light intensity, and rainfall.

[0086] The edge computing unit employs an embedded computing platform, incorporating an image preprocessing module, an improved YOLOv8 target detection module, an ESPCN subpixel localization module, an SGBM 3D coordinate calculation module, and a settlement calculation module. The image preprocessing module performs denoising, enhancement, distortion correction, and illumination normalization; the improved YOLOv8 target detection module identifies baseline targets, monitoring targets, and virtual targets; the ESPCN subpixel localization module outputs subpixel-level coordinates of the target center; the SGBM 3D coordinate calculation module calculates the target's 3D coordinates; and the settlement calculation module generates settlement data based on elevation changes.

[0087] The data transmission unit employs a dual-link redundancy design using both 5G wireless communication and fiber optic communication. When the fiber optic link is functioning normally, real-time settlement data, processing results, and raw images are transmitted via fiber optics first. When the fiber optic link malfunctions, the system automatically switches to the 5G wireless communication link to ensure continuous uploading of critical monitoring data. For bandwidth-constrained scenarios, the system prioritizes uploading settlement data, target coordinates, early warning results, and key images; raw large images can be uploaded as needed or cached locally.

[0088] The remote monitoring center comprises servers, a time-series database, monitoring terminals, and a visualization platform. The servers run programs for settlement field reconstruction, Transformer time-series prediction, and isolated forest anomaly detection. The time-series database stores target coordinates, settlement amounts, environmental data, early warning records, and equipment status. The monitoring terminals are used for manual viewing and handling of early warnings. The visualization platform, built on a 3D digital twin model, integrates real-time settlement monitoring data with the dam's 3D model, displaying settlement cloud maps, contour maps, settlement process lines, and early warning information.

[0089] The power supply unit employs a hybrid power supply method, utilizing solar panels, lithium iron phosphate batteries, and backup mains power. Solar panels generate electricity during the day, batteries provide power at night and during cloudy or rainy weather, and backup mains power is used for power supply during prolonged periods of low light or equipment maintenance. The power supply unit simultaneously monitors battery charge level, charge / discharge status, and power supply anomalies, uploading this information to a remote monitoring center.

[0090] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the deep learning-based visual monitoring method for dam settlement described above. The computer-readable storage medium can be a hard disk, solid-state drive, read-only memory, random access memory, mobile storage, or server storage medium. The computer program can be deployed in an edge computing unit, a remote monitoring center server, or a monitoring platform composed of both.

[0091] Example 1: On-site verification example of a concrete gravity dam This embodiment uses a concrete gravity dam as an example for field verification. The dam is 185m high and 2309m long at the crest, comprising multiple dam sections. The experiment covered typical climatic conditions such as low temperature, heavy rainfall, and high temperature to verify the target identification, sub-pixel localization, three-dimensional coordinate calculation, settlement field reconstruction, and early warning capabilities of the method of this invention in complex outdoor environments.

[0092] First, a binocular industrial camera system was installed on stable bedrock on the left bank of the dam. Both cameras maintained a fixed baseline, and their field of view was adjusted via a motorized pan-tilt unit to cover the dam crest and upstream face of the target dam section. The cameras were 20-megapixel global shutter industrial cameras equipped with 16mm fixed-focus lenses to reduce the impact of motion distortion and viewing angle changes on image acquisition quality.

[0093] Secondly, multiple benchmark targets are deployed in stable areas on both sides of the dam, and monitoring targets are deployed on the dam crest and face of the target dam section. Simultaneously, natural features such as dam section joints, drainage holes, and pre-embedded bolts are identified as virtual targets. The monitoring targets are circular reflective targets, and natural features are established using an initial image database to create feature templates and spatial coordinate associations. The system is configured with an edge computing unit, a communication module, and a hybrid power supply unit. The edge computing unit performs image preprocessing, target detection, sub-pixel localization, and 3D coordinate calculation. The communication module uses redundant 5G and fiber optic links, and the power supply unit uses a combination of solar panels, lithium iron phosphate batteries, and backup mains power.

[0094] Then, a checkerboard calibration board was used to calibrate the intrinsic parameters of the binocular cameras, obtaining the focal length, principal point coordinates, and distortion coefficients of the left and right cameras. The rotation matrix and translation vector between the left and right cameras were also obtained through binocular calibration. Subsequently, using the known engineering coordinates of the benchmark target within the stable bedrock area, the PnP algorithm was used to solve for the camera extrinsic parameters, establishing a mapping relationship between image coordinates and dam engineering coordinates. During system operation, extrinsic parameter verification was performed using the benchmark target at preset intervals. When the reprojection error of the benchmark target exceeded a preset threshold, the system automatically triggered a recalibration or manual verification process.

[0095] Subsequently, the system acquires binocular images at a preset frequency and sequentially performs denoising, enhancement, distortion correction, and illumination normalization. The processed images are input into an improved YOLOv8 network, which outputs the categories and bounding boxes of the baseline target, the monitored target, and the virtual target. For the detected target region, the system uses an ESPCN subpixel convolutional network for 4x super-resolution reconstruction, and then calculates the target center coordinates according to the gray-level weighted centroid formula. Next, the system calculates the disparity between the left and right images based on the SGBM algorithm, combines camera calibration parameters and extrinsic parameter matrices to solve for the target's 3D coordinates, and calculates the settlement based on the initial elevation and the current measured elevation.

[0096] Finally, the system uses discrete target settlement as input and ordinary kriging interpolation to generate a continuous settlement field. Structural constraint weighting coefficients are introduced for rigid structural areas such as the cutoff wall and core wall. These coefficients serve as post-processing smoothing correction coefficients for the ordinary kriging interpolation results, weighting and fusing the initial settlement estimates of the interpolation points within the rigid structural areas with neighborhood smoothing reference values. The density of interpolation points is increased for dam sections with poor geological conditions or historically abnormal settlement. The remote monitoring center uses a Transformer time-series prediction model to predict the trend of settlement sequences for each target. The model input includes settlement data from the past 30 days and environmental auxiliary features such as temperature, humidity, rainfall, and sunlight, outputting predicted settlement values ​​for the next 7 days. Simultaneously, the isolated forest algorithm is used to detect anomalies in real-time settlement data and outputs tiered early warning information based on settlement amount and settlement rate thresholds.

[0097] Under different weather conditions, the target detection performance of the improved YOLOv8 algorithm of this invention was compared with that of the original YOLOv8 and Faster R-CNN algorithms. The results are shown in Table 1.

[0098] Table 1 Comparison of target detection accuracy under different weather conditions

[0099] The results above show that the improved YOLOv8 algorithm maintains a high recognition accuracy under moderate rain and haze conditions, and is suitable for visual monitoring of dams in complex outdoor environments.

[0100] The accuracy comparison of different positioning methods is shown in Table 2.

[0101] Table 2 Comparison of accuracy of different positioning methods

[0102] The results above show that the ESPCN subpixel convolution combined with weighted centroid method can effectively reduce the target center localization error.

[0103] The accuracy comparison of different monitoring methods is shown in Table 3.

[0104] Table 3 Comparison of accuracy of different monitoring methods

[0105] The results above show that the system of the present invention has high settlement measurement accuracy in long-distance non-contact monitoring scenarios.

[0106] Table 4 shows a comparison of the costs and efficiencies of different monitoring methods.

[0107] Table 4. Comparison of Costs and Efficiency of Different Monitoring Methods

[0108] The results above show that the system of the present invention combines the advantages of high frequency, low cost and automated monitoring, and is suitable for long-term safety monitoring of dams.

[0109] The above description is only a preferred embodiment of the present invention. For those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based visual monitoring method for dam settlement, characterized in that, Includes the following steps: S1. Deploy a binocular industrial camera group in a stable area around the dam, set up benchmark targets and monitoring targets on the dam surface, and select stable and identifiable natural features of the dam as virtual targets; perform internal and external parameter calibration on the binocular industrial camera group, and establish the mapping relationship between the image coordinate system, the camera coordinate system and the dam engineering coordinate system. S2. Acquire binocular images of the dam surface according to a preset acquisition frequency, and perform noise reduction, enhancement, distortion correction and illumination normalization on the binocular images to obtain a preprocessed image. S3. An improved YOLOv8 target detection algorithm is used to identify the baseline target, monitoring target and virtual target in the preprocessed image, and super-resolution reconstruction and weighted centroid calculation are performed on the identified target region to obtain the sub-pixel level coordinates of the target center; S4. Based on the stereo matching results of the binocular images, camera calibration parameters, and extrinsic parameter matrices, calculate the three-dimensional coordinates of each target in the dam engineering coordinate system; S5. Using the benchmark target as a stable reference, calculate the settlement based on the elevation changes of each monitoring target and virtual target at different times. S6. Reconstruct the settlement field of the entire dam body based on the discrete target settlement data, and constrain and correct the settlement field in combination with the dam body structural zoning and geological conditions; S7. Based on the target sedimentation time series, perform sedimentation trend prediction and anomaly detection. When the sedimentation amount or sedimentation rate meets the preset early warning conditions, output graded early warning information.

2. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S1, the reference targets are deployed in stable areas on both sides of the dam that are not affected by dam deformation, and the number is no less than 3; the monitoring targets are deployed in a grid pattern on the dam top and upstream and downstream dam faces, with a deployment spacing of 20m to 50m; the virtual targets include at least one of dam section joints, drainage holes, pre-embedded bolts and permanent markers.

3. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S1, the intrinsic parameter calibration of the binocular industrial camera group is completed using Zhang's calibration method to obtain the focal length, principal point coordinates, and lens distortion coefficients of the left and right cameras. The extrinsic parameter calibration is completed using the PnP algorithm combined with the known engineering coordinates of the reference target to obtain the rotation matrix and translation vector from the camera coordinate system to the dam engineering coordinate system. The extrinsic parameter calibration results are then verified using the reprojection error of the reference target according to a preset period.

4. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S2, the denoising is performed using Gaussian filtering, the enhancement is performed using adaptive histogram equalization, the distortion correction is performed based on camera intrinsic parameters and lens distortion coefficients, and the illumination normalization is performed using the multi-scale Retinex algorithm to reduce the impact of noise, shadows and illumination abrupt changes on target recognition.

5. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S3, the improved YOLOv8 target detection algorithm is a detection network formed on the basis of the YOLOv8-s network. The detection network introduces CBAM attention mechanism modules after the C2f-3, C2f-4 and C2f-5 modules of the Backbone, and replaces the original PAN structure with a BiFPN bidirectional feature pyramid structure in the Neck part to perform multi-scale feature fusion detection of the benchmark target, the monitoring target and the virtual target.

6. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S3, the identified target region is reconstructed using a sub-pixel convolutional network based on ESPCN with a 4x super-resolution capability. The sub-pixel coordinates of the target center are then calculated based on the grayscale weights of the reconstructed target region. The calculation formula is as follows: ； In the formula, The horizontal subpixel coordinates of the target center. The vertical sub-pixel coordinates of the target center. The set of pixels within the target area. The image coordinates of the pixels within the target area. For pixels The corresponding grayscale value.

7. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S4, the SGBM stereo matching algorithm is used to calculate the disparity map between the left and right images. Combined with the focal length, baseline distance, intrinsic parameter matrix, extrinsic parameter matrix and target center sub-pixel level coordinates of the binocular camera, the three-dimensional coordinates of each benchmark target, monitoring target and virtual target in the dam engineering coordinate system are calculated.

8. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S5, the settlement amount is calculated according to the following formula: ； In the formula, For the first One target in Settlement at any given time For the first The elevation of the target at the initial moment For the first One target in The measured elevation at any given time. Target number, For monitoring time.

9. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S6, ordinary kriging interpolation is used to generate a global settlement field of the dam body based on the discrete target settlement data. The settlement to be interpolated in ordinary kriging interpolation is the sum of the products of the settlement of each discrete target and the corresponding kriging weight. For areas with impermeable walls, core walls, or other rigid structures, the structural constraint weight coefficient is set to 1.2 to 1.5 during the settlement field reconstruction process, and the structural constraint weight coefficient is used as the weighting coefficient of the neighborhood smoothing term of the interpolation points in the rigid structure area to post-process and correct the ordinary kriging interpolation results. For dam sections with poor geological conditions or large historical deformation, the density of interpolation points is increased to improve the accuracy of local settlement field expression.

10. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S7, a Transformer-based time-series prediction model is used to analyze the settlement time series of each target. The time-series prediction model includes an Encoder×6 and a Decoder×6. The input is the settlement data of the past 30 days and environmental auxiliary features, and the output is the settlement prediction value for the next 7 days. The environmental auxiliary features include at least one of temperature, humidity, light intensity and rainfall.

11. The deep learning-based visual monitoring method for dam settlement according to claim 1, characterized in that, In step S7, the isolated forest algorithm is used to detect anomalies in the real-time settlement data, and the warning level is determined based on at least one of the settlement amount threshold, settlement rate threshold, and predicted settlement deviation threshold. The graded warning information includes at least one of the warning location, warning level, abnormal target number, anomaly type, and handling suggestions.

12. A deep learning-based visual monitoring system for dam settlement, characterized in that, The deep learning-based visual monitoring method for dam settlement according to any one of claims 1-11 is adopted, comprising an image acquisition unit, an edge computing unit, a data transmission unit, a remote monitoring center, and a power supply unit that are sequentially and interactively linked: wherein: The image acquisition unit is used to acquire binocular images of the dam surface and on-site environmental data. An edge computing unit is used to perform image preprocessing, target detection, sub-pixel localization, and three-dimensional coordinate calculation on the binocular image. The data transmission unit is used to transmit settlement data, processing results, and raw images to the remote monitoring center; The remote monitoring center is used to store and manage monitoring data, reconstruct the settlement field of the entire dam body, and perform settlement trend prediction and anomaly early warning. The power supply unit provides continuous power to the image acquisition unit, edge computing unit, data transmission unit, and remote monitoring center.

13. The deep learning-based visual monitoring system for dam settlement according to claim 12, characterized in that, The image acquisition unit includes two 20-megapixel global shutter industrial cameras, a 16mm fixed-focus lens, a waterproof and dustproof protective cover, an electric gimbal, and an environmental sensor; the environmental sensor includes at least one of a temperature sensor, a humidity sensor, a light sensor, and a rain sensor.

14. The deep learning-based visual monitoring system for dam settlement according to claim 12, characterized in that, The edge computing unit adopts an embedded computing platform, which includes an image preprocessing module, an improved YOLOv8 target detection module, an ESPCN subpixel localization module, an SGBM three-dimensional coordinate calculation module, and a settlement calculation module, for generating target three-dimensional coordinates and settlement data locally in real time.

15. The deep learning-based visual monitoring system for dam settlement according to claim 12, characterized in that, The data transmission unit adopts a dual-link redundancy design of 5G wireless communication and optical fiber communication; the power supply unit adopts a hybrid power supply method of solar panels, lithium iron phosphate batteries and mains power backup.

16. The deep learning-based visual monitoring system for dam settlement according to claim 12, characterized in that, The remote monitoring center includes a server, a time-series database, a monitoring terminal, and a visualization platform. The visualization platform is built based on a three-dimensional digital twin model and is used to integrate and display real-time settlement monitoring data with the three-dimensional model of the dam body, as well as settlement cloud maps, contour maps, settlement process lines, and early warning information.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the deep learning-based visual monitoring method for dam settlement as described in any one of claims 1 to 11.

Images