A method and system for real-time detection and measurement of corrosion on the surface of a hydraulic metal structure

Abstract

The application discloses a kind of water conservancy metal structure surface corrosion real-time detection and measurement method and system, it is related to water conservancy metal structure intelligent detection technical field, comprising the following steps: binocular camera calibration and image acquisition;Data annotation and enhancement;Corrosion segmentation network construction;Depth measurement model construction and network training;Corrosion area segmentation, tracking and depth measurement;Corrosion area calculation;Detection result determination and storage.The application adopts above-mentioned a kind of water conservancy metal structure surface corrosion real-time detection and measurement method and system, realizes high-precision real-time segmentation of corrosion area using semantic segmentation network, and the actual area of corrosion area is obtained in combination with binocular vision, and corrosion target is identified and counted across frame by target tracking algorithm.Can output corrosion position and area information in real time, improve water conservancy metal structure corrosion detection efficiency, and support the automatic recording and generation of detection data, can be used for on-site inspection and engineering management.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for hydraulic metal structures, and in particular to a method and system for real-time detection and measurement of surface corrosion of hydraulic metal structures. Background Technology

[0002] Hydraulic metal structures are widely used in critical engineering facilities such as hydropower stations, navigation channel locks, floodgates, maintenance platforms, channel steel linings, and trash racks. During long-term service, these metal structures are inevitably affected by factors such as water erosion, humid and hot environments, chemical corrosion, silt abrasion, and aging of anti-corrosion coatings, leading to the gradual peeling of surface coatings and varying degrees of corrosion. Failure to detect and assess corrosion in a timely manner can result in reduced structural strength, functional failure, and even safety risks. Therefore, conducting regular inspections and corrosion measurements of hydraulic metal structures is of great significance for ensuring the long-term stable operation of engineering facilities.

[0003] Currently, the most common method for rust detection on engineering sites relies on manual inspection. Maintenance personnel visually assess the size and severity of rusted areas based on experience or reference standards. However, this method has significant drawbacks: the results are easily influenced by human experience, making them highly subjective; it's difficult to achieve continuous, quantitative, and traceable measurement of corrosion area; and the complex on-site environment and limited working space also result in low efficiency, high labor intensity, and difficulty in rapidly inspecting large areas of components.

[0004] In recent years, digital image processing-based methods have been attempted for corrosion identification. However, these methods often rely on manually designed color, texture, or threshold features, resulting in poor stability under varying lighting conditions, metallic reflections, or complex backgrounds, making them unsuitable for engineering applications. Deep learning, especially convolutional neural network (CNN) models, has achieved significant results in image segmentation. However, traditional models have a large number of parameters and still have limitations in edge detail recognition and small target corrosion detection, making them unsuitable for deployment on handheld or embedded terminals.

[0005] With the development of embedded computing hardware (such as the Jetson series), lightweight deep learning networks can achieve high real-time performance. However, existing research focuses mainly on image-level recognition of corrosion areas, rarely addressing the quantitative measurement of the actual area of ​​rust. Since monocular images lack scale information, the actual physical size of the rusted area cannot be directly obtained; while depth calculations using binocular vision are easily affected by factors such as metal surface reflection and weak texture, leading to depth jumps and measurement instability, making them difficult to use directly for area calculation. Furthermore, on-site inspection often uses handheld devices for lateral scanning, and the detected target exhibits scale changes in consecutive frames, further increasing the difficulty of area measurement.

[0006] Therefore, developing a method for real-time segmentation, cross-frame tracking, binocular depth stabilization measurement, and quantitative area calculation of rusted areas on an embedded platform has become a critical problem that urgently needs to be solved. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for real-time detection and measurement of surface corrosion of hydraulic metal structures, thereby solving the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a method for real-time detection and measurement of surface corrosion of hydraulic metal structures, comprising the following steps: S1. Binocular camera calibration and image acquisition: The binocular camera is calibrated, and the calibrated binocular camera is used to acquire the left and right eye images of the hydraulic metal structure. S2. Data annotation and enhancement: The annotation tool is used to perform semantic segmentation annotation on the rust region in the left eye image. The annotated image is then enhanced to construct a rust segmentation dataset. S3. Rust Segmentation Network Construction: The YOLO12n-seg semantic network is optimized by using the dual-stream wavelet fusion module DSFM, the lightweight hollow spatial pyramid pooling module ASPP-Lite, and the cross-level attention module Cross-Level Attention to construct the rust segmentation network YOLO12n-RustSeg. S4. Depth Measurement Model Construction and Network Training: A depth back-projection model is constructed based on camera calibration parameters. The Mask-Depth region downsampling method is used for local stereo matching, and robust depth estimation is introduced. YOLO12n-RustSeg is trained using the labeled dataset. S5. Rust Region Segmentation, Tracking, and Depth Measurement: Output the rust region mask using the trained YOLO12n-RustSeg; generate consistent rust target TrackIDs across frames using a multi-target tracking algorithm; perform local stereo matching within the corresponding mask region to generate depth point clouds; S6. Corrosion Area Calculation: Based on the depth value, the actual area of ​​each pixel is calculated and accumulated using the back projection model to obtain a preliminary area result; then, the maximum area of ​​each corrosion region in multiple frames is used for smoothing to obtain the final stable area of ​​each corrosion region. S7. Judgment and storage of test results: Comprehensive judgment on area stability, depth consistency and tracking continuity.

[0009] Preferably, the binocular calibration in S1 includes intrinsic parameter calibration, extrinsic parameter calibration, distortion correction, and binocular alignment.

[0010] Preferably, the specific steps of S2 are as follows: S21. Preprocess the acquired images, including unifying resolution, color normalization, and distortion correction. S22. Mark the rusted areas, and mark the corroded areas pixel by pixel; S23. Perform data augmentation on the labeled image. Data augmentation includes random brightness changes, random contrast changes, random scaling, and random rotation.

[0011] Preferably, the dual-stream wavelet fusion module DSFM in S3 includes a mainstream C3k2 structural feature extraction branch, an auxiliary Haar wavelet high-frequency texture branch, and a dual-stream fusion module based on channel attention.

[0012] Preferably, the cross-scale context-enhanced segmentation head in S3 includes an ASPP-Lite context enhancement module and a cross-level attention module.

[0013] Preferably, the specific steps of S4 are as follows: S41. Limit Mask-Depth by local stereo matching region, and perform stereo matching only within the segmentation mask area; S42. Depth back projection modeling, calculate the depth, and obtain the depth map Z_mask within the Mask region. The formula for calculating the depth is as follows: ; in, Z For depth, f Focal length B Baseline length d This represents the disparity value. S43. Depth Robust Estimation: Automatically selects the optimal depth estimation method based on the number of pixels in the rusted area.

[0014] Preferably, the specific operation of S43 is as follows: For small rusted areas with fewer pixels than the preset threshold T, the median is chosen because it is robust to noise. For large-area rusted regions, where the number of pixels is greater than or equal to a preset threshold T, RANSAC is used to solve for the plane, and the center of the ROI is projected onto the plane to obtain the depth.

[0015] Preferably, step S5 employs a multi-target tracking method based on detection box association to track the rusted area.

[0016] Preferably, the comprehensive determination criterion in S7 is: If the test results are stable, the output will show the rust area, location, and tracking information. If the detection results are unstable or abnormal, the corresponding image frame will be archived as a difficult sample for subsequent model optimization and retraining. Difficult samples include image frames with depth jumps, segmentation errors, tracking loss, and area instability.

[0017] A system for real-time detection and measurement of surface corrosion on hydraulic metal structures includes: Dataset building module: used to construct rust semantic segmentation data; Rust network training module: Train the YOLO12n-RustSeg network using the constructed dataset; Rust segmentation module: The trained YOLO12n-RustSeg network is used to identify rust regions in the left image to obtain an accurate rust region segmentation mask; Target tracking module: used to track rusted areas across consecutive image frames based on the relationship between the target appearance and the detection box, generating a unique TrackID so as to realize the area sequence statistics of the same rusted area; Depth calculation module: used to perform local stereo matching based on segmentation mask and generate depth point cloud of rusted area; then select different depth estimation methods according to the number of pixels in the segmented rusted area to calculate a more robust depth value; The rust area calculation module is used to calculate the real area corresponding to each pixel based on the estimated region depth using a depth back projection model, and accumulate the area to obtain the area of ​​a single frame; and uses a maximum area multi-frame robust statistical algorithm to smooth the cross-frame area sequence to obtain the final stable rust area. Detection result judgment and storage module: used to determine the validity of the measurement results based on depth consistency, area change trend and target tracking continuity; Control and processing module: Used to coordinate the operation process, data transmission and status management between the above modules, and to provide the overall computing and scheduling functions of the system; The dataset construction module, corrosion network training module, corrosion region segmentation module, target tracking module, depth calculation module, corrosion area calculation module, and detection result judgment and storage module are all electrically connected to the control processing module.

[0018] Therefore, the present invention employs the above-mentioned method and system for real-time detection and measurement of surface corrosion of hydraulic metal structures, which has the following beneficial effects: (1) The present invention constructs YOLO12n-RustSeg by using a dual-stream wavelet fusion module and a cross-scale context enhancement segmentation head. Under the premise of ensuring deployability, it significantly improves the segmentation accuracy of the rust region and realizes high-precision real-time rust segmentation on the embedded platform.

[0019] (2) Based on Mask-Depth local matching and robust depth estimation, this invention significantly improves the stability of binocular measurement and effectively alleviates the depth jump problem caused by metallic reflection and weak texture.

[0020] (3) The present invention adopts a target tracking and multi-frame robust area statistics method to realize stable area calculation of the rusted area in the horizontal scanning scene.

[0021] (4) The system of the present invention can be deployed on an embedded platform and has real-time performance, portability and engineering availability. The system can realize automatic detection, continuous tracking and area quantification of rusted areas, providing a more efficient and reliable intelligent inspection method for hydraulic metal structures.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating an embodiment of a method and system for real-time detection and measurement of surface corrosion on hydraulic metal structures according to the present invention. Figure 2 This is a schematic diagram of the overall structure of the corrosion segmentation network in an embodiment of the real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 3 This is a schematic diagram of the dual-stream wavelet fusion module DSFM, representing an embodiment of a real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 4 This is a structural diagram of the ASPP-Lite module of an embodiment of the real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 5 This is a structural diagram of the Cross-Level Attention module, which is an embodiment of a real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 6 This is a schematic diagram of mask-constrained binocular depth calculation in an embodiment of a real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 7 This is a flowchart of the maximum area multi-frame robust statistical algorithm for a real-time detection and measurement method and system for surface corrosion of hydraulic metal structures according to the present invention. Figure 8 This is a schematic diagram of the structure of a real-time detection and measurement system for corrosion of hydraulic metal structures, as described in this invention. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] Example 1 Please see Figures 1-8 This invention provides a method for real-time detection and measurement of surface corrosion on hydraulic metal structures, comprising the following steps: S1. Binocular camera calibration and image acquisition.

[0027] The binocular camera underwent intrinsic and extrinsic parameter calibration and geometric alignment to obtain the camera's intrinsic parameter matrix, distortion parameters, and baseline length. The calibrated binocular camera was then used to acquire left and right eye images of the hydraulic metal structure. The left eye image was used as input for corrosion segmentation, while the binocular images were used for subsequent depth calculations. The specific operations are as follows: Before data acquisition, the binocular cameras were calibrated as follows to obtain the intrinsic parameter matrix and baseline length: intrinsic parameter calibration (focal lengths fx, fy and principal point coordinates cx, cy), extrinsic parameter calibration (rotation and translation matrices of the left and right cameras), distortion correction, and left and right eye alignment. After calibration, left and right eye images of the hydraulic metal structure were acquired. The left eye image was used for corrosion area segmentation, and the binocular images were used for depth measurement.

[0028] Binocular calibration includes intrinsic parameter calibration, extrinsic parameter calibration, distortion correction, and binocular alignment, which are used to ensure the accuracy of depth measurement.

[0029] S2. Data preprocessing and labeling.

[0030] The acquired images underwent format conversion, resolution normalization, image distortion correction, and color enhancement. A pixel-by-pixel annotation of the rust region was performed on the left eye image using an annotation tool to construct a rust semantic segmentation training dataset. Finally, data augmentation was performed. The specific operations are as follows: The acquired corrosion images were preprocessed and labeled to construct a corrosion segmentation training dataset, including: (1) Perform image format processing, including image format unification and resolution standardization; (2) Use annotation tools to perform pixel-by-pixel semantic segmentation annotation of the rusted area, and convert the annotation results into a semantic label format compatible with segmentation network training, so that the pixels of the rusted area are marked as 1 and the background is marked as 0.

[0031] (3) Implement enhancement strategies such as random brightness change, random contrast change, random flipping, and random scaling on the training images to improve the model's adaptability to different lighting conditions and surface characteristics, and enhance the model's robustness.

[0032] After preprocessing, a rust segmentation dataset is obtained that can be used to train the YOLO12n-RustSeg network.

[0033] S3, Corrosion Segmentation Network Construction.

[0034] To address the complex edges and large scale variations in corrosion zones of hydraulic metal structures, the YOLO12n-seg segmentation network was structurally optimized. By constructing a dual-stream wavelet fusion module (DSFM) and a cross-scale context-enhanced segmentation head, YOLO12n-RustSeg was proposed, aiming to improve the segmentation accuracy, edge representation ability, and real-time performance of corrosion zones.

[0035] A dual-stream wavelet fusion module is constructed, with the main stream using the C3k2 structure to extract semantic features; the auxiliary stream uses Haar wavelet decomposition to extract high-frequency features of erosion texture, and the two are fused through channel attention.

[0036] A cross-scale context-enhanced segmentation head is constructed by incorporating the ASPP-Lite module and the Cross-Level Attention module to enhance the expressive power of multi-scale erosion regions.

[0037] The specific steps are as follows: like Figure 2 As shown, this invention employs a dual-stream wavelet fusion module (DSFM), an ASPP-Lite (lightweight hollow spatial pyramid pooling) module, and a cross-level attention module to optimize the YOLO12n-seg semantic network, ultimately constructing the YOLO12n-RustSeg corrosion segmentation network. This enhances the ability to extract details from corrosion edges and improves real-time performance on embedded devices.

[0038] (1) Dual-Stream Fusion Module (DSFM).

[0039] like Figure 3 The diagram shown is a schematic of the structure of the dual-stream wavelet fusion module DSFM proposed in this invention, which includes the mainstream C3k2 feature branch and the auxiliary Haar wavelet high-frequency texture extraction branch.

[0040] The dual-stream wavelet fusion module utilizes depthwise separable convolution and channel attention mechanisms to fuse main and auxiliary features, improving the detection sensitivity of eroded edges. This includes: The mainstream C3k2 structure feature extraction branch adopts the original lightweight residual feature extraction structure C3k2 of YOLO12 (You Only Look Once, version 12) to extract rust semantic features.

[0041] The auxiliary flow Haar wavelet high-frequency texture branch performs Haar wavelet decomposition on the input image to obtain the LL / LH / HL / HH frequency bands, especially enhancing the high-frequency texture response of rust.

[0042] (2) ASPP-Lite module.

[0043] Multi-scale dilated convolutions are used to extract cross-scale contextual information, improving the network's ability to identify and represent edges of rusted regions of different sizes. The ASPP-Lite module is inspired by the classic Atrous Spatial Pyramid Pooling (ASPP), but adopts a lighter structure while maintaining a large receptive field, making it suitable for real-time deployment on embedded devices.

[0044] like Figure 4 As shown, the ASPP-Lite module consists of a four-branch dilated convolutional structure and a feature fusion layer: Four-branch dilated convolution structure: Branch 1: 1×1 convolution, preserving the semantic information of the input features, and maintaining the output size as H×W×C; Branch 2: 3×3 dilated convolution (dilation rate = 1) to extract local neighborhood features; Branch 3: 3×3 dilated convolution (dilation rate = 2) expands the receptive field and obtains mesoscale structural information; Branch 4: 3×3 dilated convolution (dilation rate = 4), capturing a wider range of global structural information.

[0045] All four branches output feature maps (H×W×C) with the same spatial dimensions as the input.

[0046] Feature fusion layer (Concat + 1×1 convolution): The features from the four branches are concatenated along the channel dimension to form a fused feature with dimensions H × W × 4C. The fused feature is then subjected to channel compression via a 1×1 convolution.

[0047] (3) Cross-Level Attention module.

[0048] like Figure 5 As shown, this module aims to enhance the semantic consistency and detail accuracy of rust region segmentation by introducing global information from the high semantic feature layer (network layer 20 P5: 20×20×256) and improving the high-resolution segmentation features (network layer 15 P3 after inputting ASPP-Lite 80×80×64).

[0049] Upsampling of high-level features and construction of query features Query(Query, Q): First, the high-level feature P5 is upsampled to the same resolution as P3 using bilinear interpolation. The upsampled feature is then dimensionality-reduced using a 1×1 convolution to generate the Query feature Q, which represents global semantic guidance information and is used to indicate the main location of the segmented region.

[0050] Key / Value construction of low-level features: P3, as the underlying feature, is fed into two 1×1 convolutions to generate the feature: Key: K and Value: V; The Key is used to compare with the Query to determine which locations need more attention, while the Value is used to express the specific enhanced features.

[0051] Cross-Level Attention is performed as follows: Calculate the similarity between the query and the key to generate attention weights; The Value feature is summed using weights. Obtain attention-enhanced feature maps; The attention-enhanced feature is then added element-wise to the feature map of the original P3 (residual connection) to obtain the final enhanced feature P3_enhance.

[0052] S4. Depth Measurement Model Construction and Network Training.

[0053] Construct a depth back-projection model based on binocular calibration parameters, and adopt the Mask-Depth regional downsampling method to limit the stereo matching range, and only perform local stereo matching within the segmentation mask to improve the speed and efficiency of depth calculation. Mask-Depth is a region selection strategy that spatially constrains the depth calculation process based on semantic segmentation masks. By only performing stereo matching within the target mask region and downsampling the non-concerned regions, the computational complexity is reduced and the depth estimation accuracy of the target region is improved.

[0054] Aiming at the problems that the targets in the rusty areas are sometimes large, sometimes small, and have a lot of noise, introduce depth robust estimation to improve the stability of depth calculation. Finally, use the labeled dataset to train the YOLO12n-RustSeg network. The specific operations are as follows: Use the segmentation dataset constructed in S2 to train the YOLO12n-RustSeg network. After the network training is completed, the rust segmentation results can be output in real time on the left-eye image.

[0055] At the same time, construct a binocular depth calculation model, including: (1)Establish a depth calculation formula according to the camera calibration parameters: ; Among them, Z is the depth, f is the focal length, B is the baseline length, and d is the disparity.

[0056] (2)As Figure 6 shown, it is a schematic diagram of mask-constrained binocular depth calculation (Mask-Depth) proposed by the present invention, which is used to illustrate the execution of local stereo matching within the segmentation mask. Adopt the Mask-Depth region constraint method to only perform local stereo matching within the rust mask, reducing the amount of disparity calculation.

[0057] (3)Automatically select the optimal depth estimation method according to the number of pixels in the rust area: for small-area rust areas (number of pixels < T), select the median value to be robust to noise; for large-area rust areas (number of pixels ≥ T), after using RANSAC to solve the plane, project the center of the ROI onto the plane to obtain the depth.

[0058] (4)Perform grid median downsampling on the matched depth results to reduce the amount of calculation.

[0059] S5. Rust area segmentation, target tracking and depth measurement.

[0060] The trained YOLO12n-RustSeg corrosion segmentation network was applied to the left-eye image to obtain a corrosion region segmentation mask. A multi-object tracking algorithm was used to generate consistent corrosion target TrackIDs across frames. Binocular stereo matching was performed within the corresponding mask region to obtain depth point clouds. The specific operations are as follows: During the reasoning stage: (1) Input the left eye image into the YOLO12n-RustSeg network to obtain the rust area mask.

[0061] (2) Use a multi-target tracking algorithm to generate a unique TrackID for a consistent rust target across frames.

[0062] Rust area tracking employs a multi-target tracking method based on detection box association, including but not limited to ByteTrack, IoU-based association methods, or their equivalent improved algorithms.

[0063] (3) Perform local stereo matching at the pixel position corresponding to the mask area to obtain the depth point cloud.

[0064] This step enables cross-frame recognition and 3D measurement of rusted areas, supporting continuous scanning with handheld devices.

[0065] S6. Calculation of rust area.

[0066] (1) Regional depth Z estimation strategy.

[0067] The actual corrosion area is obtained by back-projecting pixels in the mask region based on depth point cloud and camera intrinsic parameters. Different depth estimation methods are used depending on the size of the corrosion area. When the rusted area is small (number of pixels < preset threshold T), the median depth is used as the area depth; When the rusted area is large (number of pixels ≥ preset threshold), the Random Sampling Consistency Plane Fitting (RANSAC) method is used to estimate the plane depth to eliminate the effects of reflection and noise.

[0068] (2) Area calculation.

[0069] Calculate the true area per pixel using camera intrinsics and region depth Z: ; This represents the actual pixel area. Z This represents the region's depth value. f x For the camera x Equivalent pixel focal length in the horizontal direction. f y For the camera y Equivalent pixel focal length in the direction (vertical).

[0070] The resulting area sequence is smoothed after selecting the largest area from multiple frames to obtain the final area value. For example... Figure 7 The diagram shows the flowchart of the maximum area multi-frame robust statistical algorithm proposed in this invention, used to illustrate the smoothing process of the cross-frame area sequence of the rusted target. The total area of ​​the region is the sum of the areas of all pixels within the mask. Then, the maximum area of ​​each rusted region across multiple frames is used for smoothing to obtain the final stable area of ​​each rusted region.

[0071] S7. Detection and storage of test results.

[0072] The reliability of the test results is determined based on area stability, depth consistency, and tracking continuity.

[0073] If the area change meets the stability threshold, the depth jump is controlled within the range, and the target tracking is continuous, it is judged as a normal result; otherwise, it is judged as an abnormal detection result.

[0074] When the judgment result is normal, output the rust area and the corresponding target number TrackID; If anomalies such as depth jumps, segmentation errors, or tracking loss occur, the image frame will be automatically archived as a "hard sample" and used as a data source for subsequent incremental training of the model, for subsequent model retraining and system optimization.

[0075] Difficult samples, including image frames with depth jumps, segmentation errors, tracking loss, and area instability, are used for system self-learning and retraining.

[0076] Example 2 like Figure 8 As shown, this invention provides a system for real-time detection and measurement of surface corrosion on hydraulic metal structures, comprising: (1) Dataset construction module, used to construct a rust semantic segmentation dataset; (2) Rust network training module: The YOLO12n-RustSeg network is trained using the constructed dataset; (3) Corrosion region segmentation module: The trained YOLO12n-RustSeg network is used to segment the corrosion region of the left eye image of the stereo camera; (4) Target tracking module, used to track the segmented rust targets and assign them a unique TrackID; (5) Depth calculation module: Use the segmentation mask to extract the ROI (the corresponding area of ​​the left and right eyes), perform local stereo matching (only calculate the disparity within the mask) to obtain the depth map; then select different depth estimation methods according to the number of pixels in the segmented rust area to calculate a more robust depth value; (6) The rust area calculation module uses the camera intrinsic parameters to back-project the pixels in the mask into the real area, and then selects the average value of N consecutive frames near the maximum value of each rust area to use as the final rust area. (7) Detection result judgment and storage module: it determines whether the image frame is abnormal by the depth value jump, segmentation failure, tracking loss, etc. If it is normal, it outputs the area result and the corresponding TrackID; if it is abnormal, it saves the image frame.

[0077] (8) Control processing module, used to coordinate the normal operation of various modules of the system.

[0078] The above modules are electrically connected through a control and processing module to achieve cascaded operation of the system.

[0079] Therefore, this invention employs the aforementioned method and system for real-time detection and measurement of surface corrosion on hydraulic metal structures. It solves the problem of how to achieve high-precision automatic segmentation of corrosion areas in corrosion images acquired during on-site inspections of hydraulic metal structures, and how to calculate and statistically analyze the actual area of ​​corrosion areas on an embedded platform without the need for external computing devices. This replaces the problems of strong subjectivity, low efficiency, and inability to perform quantitative measurements inherent in traditional manual inspection. Simultaneously, this invention alleviates the depth jump problem caused by surface reflection and weak texture during binocular measurement, ensuring stable and reliable corrosion detection results. For failed or abnormal results, the system can automatically store samples for subsequent model optimization.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for real-time detection and measurement of surface corrosion on hydraulic metal structures, characterized in that, Includes the following steps: S1. Binocular camera calibration and image acquisition: The binocular camera is calibrated, and the calibrated binocular camera is used to acquire the left and right eye images of the hydraulic metal structure. S2. Data annotation and enhancement: The annotation tool is used to perform semantic segmentation annotation on the rust region in the left eye image. The annotated image is then enhanced to construct a rust segmentation dataset. S3. Rust Segmentation Network Construction: The YOLO12n-seg semantic network is optimized by using the dual-stream wavelet fusion module DSFM, the lightweight hollow spatial pyramid pooling module ASPP-Lite, and the cross-level attention module Cross-Level Attention to construct the rust segmentation network YOLO12n-RustSeg. S4. Depth Measurement Model Construction and Network Training: A depth back-projection model is constructed based on camera calibration parameters. The Mask-Depth region downsampling method is used for local stereo matching, and robust depth estimation is introduced. YOLO12n-RustSeg is trained using the labeled dataset. S5. Rust Region Segmentation, Tracking, and Depth Measurement: Output the rust region mask using the trained YOLO12n-RustSeg; generate consistent rust target TrackIDs across frames using a multi-target tracking algorithm; perform local stereo matching within the corresponding mask region to generate depth point clouds; S6. Corrosion Area Calculation: Based on the depth value, the actual area of ​​each pixel is calculated and accumulated using the back projection model to obtain a preliminary area result; then, the maximum area of ​​each corrosion region in multiple frames is used for smoothing to obtain the final stable area of ​​each corrosion region. S7. Judgment and storage of test results: Comprehensive judgment on area stability, depth consistency and tracking continuity.

2. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that: The binocular calibration in S1 includes intrinsic parameter calibration, extrinsic parameter calibration, distortion correction, and binocular alignment.

3. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that, The specific steps of S2 are as follows: S21. Preprocess the acquired images, including resolution unification, color normalization, and distortion correction. S22. Mark the rusted areas, and mark the corroded areas pixel by pixel; S23. Perform data augmentation on the labeled image. Data augmentation includes random brightness changes, random contrast changes, random scaling, and random rotation.

4. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that: The dual-stream wavelet fusion module DSFM in S3 includes a mainstream C3k2 structural feature extraction branch, an auxiliary Haar wavelet high-frequency texture branch, and a dual-stream fusion module based on channel attention.

5. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that: The cross-scale context enhancement segmentation head in S3 includes the ASPP-Lite context enhancement module and the Cross-Level Attention module.

6. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that, The specific steps of S4 are as follows: S41. Limit Mask-Depth by local stereo matching region, and perform stereo matching only within the segmentation mask area; S42. Depth back projection modeling, calculate the depth, and obtain the depth map Z_mask within the Mask region. The formula for calculating the depth is as follows: ； in, Z For depth, f Focal length B Baseline length d This represents the disparity value. S43. Depth Robust Estimation: Automatically selects the optimal depth estimation method based on the number of pixels in the rusted area.

7. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 6, characterized in that, The specific operation of S43 is as follows: For small rusted areas with fewer pixels than the preset threshold T, the median is chosen because it is robust to noise. For large-area rusted regions, where the number of pixels is greater than or equal to a preset threshold T, RANSAC is used to solve for the plane, and the center of the ROI is projected onto the plane to obtain the depth.

8. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that: The S5 uses a multi-target tracking method based on detection box association to track the rusted area.

9. The method for real-time detection and measurement of surface corrosion of hydraulic metal structures according to claim 1, characterized in that: The criteria for comprehensive judgment in S7 are as follows: If the test results are stable, the output will show the rust area, location, and tracking information. If the detection results are unstable or abnormal, the corresponding image frame will be archived as a difficult sample for subsequent model optimization and retraining. Difficult samples include image frames with depth jumps, segmentation errors, tracking loss, and area instability.

10. A system for real-time detection and measurement of surface corrosion of hydraulic metal structures according to any one of claims 1-9, characterized in that, include: Dataset building module: used to construct rust semantic segmentation data; Rust network training module: Train the YOLO12n-RustSeg network using the constructed dataset; Rust segmentation module: The trained YOLO12n-RustSeg network is used to identify rust regions in the left image to obtain an accurate rust region segmentation mask; Target tracking module: used to track rusted areas across consecutive image frames based on the relationship between the target appearance and the detection box, generating a unique TrackID so as to realize the area sequence statistics of the same rusted area; Depth calculation module: used to perform local stereo matching based on segmentation mask and generate depth point cloud of rusted area; then select different depth estimation methods according to the number of pixels in the segmented rusted area to calculate a more robust depth value; The rust area calculation module is used to calculate the real area corresponding to each pixel based on the estimated region depth using a depth back projection model, and accumulate the area to obtain the area of ​​a single frame; and uses a maximum area multi-frame robust statistical algorithm to smooth the cross-frame area sequence to obtain the final stable rust area. Detection result judgment and storage module: used to determine the validity of the measurement results based on depth consistency, area change trend and target tracking continuity; Control and processing module: Used to coordinate the operation process, data transmission and status management between the above modules, and to provide the overall computing and scheduling functions of the system; The dataset construction module, corrosion network training module, corrosion region segmentation module, target tracking module, depth calculation module, corrosion area calculation module, and detection result judgment and storage module are all electrically connected to the control processing module.

Images