An intelligent crack identification method for water gate structure based on multi-scale feature fusion

Abstract

The application discloses a kind of water gate structure crack intelligent identification method based on multi-scale feature fusion, it is related to water conservancy engineering monitoring field.The present application is based on public concrete crack dataset and water gate exclusive crack sample to construct training set, and collects water gate field monitoring image to construct test set;Progressive layered pre-training method is used to pre-train Deep-Crack crack identification model, and hierarchical fine-tuning is realized in combination with ladder learning rate decay strategy, to adapt to water gate exclusive crack identification;The model is through multi-scale feature fusion deep convolutional neural network, carries out hierarchical feature extraction and fusion to test image, and combines pixel-level prediction identification to output the visual result of black background and white crack;Periodically collect data to construct incremental training set, use transfer learning and error feedback mechanism to iteratively optimize the model, and use joint loss function to guarantee identification accuracy.The present application can accurately output crack related parameters and water gate health early warning information, and has deep application value.

Description

Technical Field

[0001] This invention relates to the field of water conservancy engineering monitoring technology, specifically to a method for intelligent identification of cracks in sluice gate structures based on multi-scale feature fusion. Background Technology

[0002] With the continuous improvement of the intelligence level of water conservancy projects, sluice gates, as core structures of water conservancy projects such as flood control, irrigation, and water supply, are directly related to the structural health of the project and operational stability. Structural cracks in sluice gates are a common form of damage, and timely and accurate identification and detection are crucial to ensuring the safe operation of sluice gates. Currently, sluice gate crack identification technology has been widely used in the daily monitoring of various water conservancy projects. However, existing identification technologies still have significant shortcomings. Manual detection methods are inefficient, highly subjective, and prone to missing minute cracks; conventional automated detection methods have weak anti-interference capabilities and are difficult to adapt to complex interference environments such as shadows, water stains, and stones at the sluice gate site; intelligent identification methods generally struggle to accurately identify cracks of different widths and shapes and cannot adapt to changes in damage during long-term operation of sluice gates.

[0003] In this context, several related technological studies have been conducted within the field, such as:

[0004] Chinese patent (publication number CN221572402U) discloses a water gate crack detection device. The method uses a special detection device with sensing components to scan and detect the surface of the water gate structure, so as to achieve preliminary identification and location of cracks in the water gate and can be adapted to the basic detection needs of the water gate site. However, the method still relies on traditional detection devices, and the detection accuracy is limited. It cannot achieve pixel-level identification of cracks and quantitative analysis of parameters such as width and length. Moreover, the anti-interference ability is weak and it is easily affected by water stains, stones and other interference at the water gate site, which may lead to missed detection or false detection.

[0005] Chinese patent (publication number CN113408327A) discloses a dam crack detection model and method based on an improved Faster-RCNN. This method extracts image features through a convolutional neural network, generates anchor boxes by combining them with a region generation network, and obtains a fused feature map through a multi-scale feature fusion module to achieve the detection and identification of dam cracks. However, this method is mainly designed for dam cracks and does not optimize the model for the special characteristics of cracks in sluice gate structures. It also lacks an effective iterative optimization mechanism and cannot adapt to the complex and ever-changing monitoring environment at sluice gate sites for a long time. The accuracy of identifying minute cracks in sluice gates is insufficient.

[0006] Chinese patent (publication number CN120339290B) discloses a method and system for detecting cracks in concrete structures based on deep learning. This method acquires and fuses images from multiple perspectives, and uses transfer learning to perform semantic segmentation to obtain a crack mask, thereby enabling dynamic analysis of concrete crack changes. However, this method does not focus on cracks in sluice gate structures, the anti-interference design is not adapted to scenarios such as shadows and uneven lighting at the sluice gate site, and the model training accuracy is limited, making it unable to accurately output key parameters such as the opening level of sluice gate cracks.

[0007] In summary, existing technologies for identifying cracks in sluice gate structures still have many shortcomings and cannot meet the actual needs for automated, precise, and long-term stable monitoring of cracks in sluice gate structures. Summary of the Invention

[0008] To address the aforementioned technical problems, this application discloses an intelligent method for identifying cracks in sluice gate structures based on multi-scale feature fusion, specifically including:

[0009] A training set was constructed based on a publicly available dataset of concrete cracks; a test set was constructed by collecting monitoring images of the surface of a sluice gate structure.

[0010] A Deep-Crack crack recognition model is constructed using a multi-scale feature fusion deep convolutional neural network algorithm. The Deep-Crack crack recognition model is pre-trained based on the training set, and the pre-training adopts a progressive hierarchical pre-training method.

[0011] The monitoring images of the test set are input into the pre-trained Deep-Crack crack recognition model. The model performs hierarchical feature extraction and multi-scale feature fusion processing on the monitoring images in sequence to obtain a fused feature map. Based on the fused feature map, the model performs pixel-level prediction and recognition of sluice gate cracks to obtain the recognition result.

[0012] Regularly collect monitoring images, predict and identify results, and iteratively optimize the Deep-Crack crack identification model.

[0013] Preferably, the progressive layered pre-training method specifically involves: performing initial pre-training on the backbone network of the Deep-Crack crack recognition model, freezing the shallow parameters of the network during the process, training the deep feature extraction layer and output layer of the network, and obtaining the initial pre-trained model.

[0014] A subset of sluice gate structure crack samples is extracted from the training set, and the initial pre-trained model is fine-tuned in a second layer. Shallow parameters are gradually unfrozen, and a step-wise learning rate decay strategy is adopted to adapt the model from general crack recognition to sluice gate-specific crack recognition, thus obtaining the target pre-trained model.

[0015] Preferably, the stepped learning rate decay strategy specifically involves: based on the phased training requirements of secondary layered fine-tuning, and combining the loss feedback adjustment term, the round decay term, and the layered adaptation coefficient, adaptively adjusting the learning rate, as shown in the formula:

[0016]

[0017] in, For the first Wheel, First Learning rate during phase training The initial learning rate, Based on the attenuation coefficient, It is a natural constant. For round decay term, To fine-tune the adaptation coefficients in a layered manner, It is the hyperbolic tangent function. For the first The loss of the wheel, This is the average loss. The standard deviation of the training loss. This is a loss feedback adjustment term.

[0018] Preferably, the progressive hierarchical pre-training and iterative optimization process employs a joint loss function of Dice loss and cross-entropy loss, as shown in the following formula:

[0019]

[0020]

[0021]

[0022] in, For joint losses, For cross-entropy loss, For Dice's loss, The coefficients of the Dice loss are... Total number of pixels For pixels The true label, For pixels The prediction confidence level This represents the actual crack area. To predict the crack area.

[0023] Preferably, the Deep-Crack crack recognition model is constructed using a multi-scale feature fusion deep convolutional neural network algorithm; the monitoring image data of the input test set is used to extract hierarchical features from the monitoring images to obtain low-level detail features and high-level semantic features respectively; the low-level detail features and high-level semantic features are fused at multiple scales to generate a fused feature map; pixel-level crack prediction is performed based on the fused feature map, and a visually accurate crack recognition image is output.

[0024] Preferably, the multi-scale feature fusion deep convolutional neural network algorithm specifically includes an input layer, a shallow convolutional layer, a middle convolutional layer, a deep convolutional layer, a feature fusion layer, and an output layer. The algorithm is based on a deep convolutional neural network framework, integrating a hierarchical feature extraction module, channel attention weighting, multi-scale fusion, and pixel-level prediction. It differentially extracts features at different levels and performs adaptive fusion to perform end-to-end sluice gate crack identification.

[0025] Preferably, the hierarchical feature extraction specifically involves: performing hierarchical feature extraction collaboratively through shallow convolutional layers, mid-level convolutional layers, and deep convolutional layers. The shallow convolutional layers extract low-level detail features of the monitoring image, including edges, gradients, textures, and fine crack contours. The mid-level convolutional layers connect the shallow and deep layers to adapt the sluice gate-specific crack features, achieving a transition from general features to specific features. The deep convolutional layers extract high-level semantic features of the monitoring image, including the global direction of cracks, continuous structure, and regional semantic information. During the extraction process, a batch normalization algorithm is used to optimize feature extraction accuracy and model stability. The formula is:

[0026]

[0027] in, For batch normalized output feature values, , The scaling and offset parameters are determined during pre-training. The output features of the convolutional layer This is the batch average. For batch variance, It is a very small constant.

[0028] Preferably, the multi-scale feature fusion processing specifically involves: using a fully connected network to calculate channel attention weights through a feature fusion layer; weighting low-level detail features and high-level semantic features based on the channel attention weights; and performing multi-scale feature fusion processing through a cross-scale concatenation algorithm to generate a fused feature map.

[0029] Preferably, the pixel-level prediction and recognition of sluice gate cracks specifically involves: performing bilinear interpolation upsampling on the fused feature map through the output layer to restore it to the input image size; then, using the Sigmoid activation function to output a crack probability map, as shown in the formula:

[0030]

[0031] in, For pixels The confidence level of the crack. For fusion of feature maps Local characteristics;

[0032] based on Binarization segmentation with adaptive thresholding is performed using the following formula:

[0033]

[0034] in, Pixel labels for the binarized segmentation results of the sluice gate cracks. The adaptive threshold is determined during pre-training, and the formula is:

[0035]

[0036] in, This represents the number of samples in the training set. For the first The set consisting of the crack confidence scores of all pixels in the training sample image. For the first The median of all confidence levels for the sample images. This is the threshold adjustment coefficient. These represent the maximum confidence and minimum confidence of a pixel, respectively.

[0037] Based on pixel tags, the location, shape, and extension direction of cracks are accurately depicted on a black background image, achieving a precise visual representation of cracks.

[0038] Preferably, the iterative optimization specifically involves: collecting newly added sluice gate structure monitoring images and corresponding prediction and recognition results at fixed intervals; filtering samples with recognition errors greater than a preset threshold and constructing an incremental training set together with the newly added samples; performing data annotation and enhancement on the incremental training set; adopting a transfer learning strategy; freezing the core parameters of the Deep-Crack model backbone network and updating only the parameters of the feature fusion layer and output layer; calculating the recognition accuracy and recall of the model before and after iteration; and constructing an error feedback function. If the error If the error exceeds the preset threshold, repeat the iterative optimization until the error E is below the threshold, thus completing the model iterative optimization.

[0039] Compared with the prior art, the technical solution of this application has the following technical effects:

[0040] This invention effectively suppresses background interference by using a progressive hierarchical pre-training method, combined with a multi-scale feature fusion algorithm and batch normalization optimization. This solves the problems of weak anti-interference ability and easy omission and misjudgment in existing technologies, and is suitable for complex monitoring scenarios at sluice gates.

[0041] This invention uses a hierarchical feature extraction method based on the Deep-Crack model to take into account both the detailed features of minute cracks and the semantic features of global cracks. Combined with an adaptive threshold segmentation algorithm, it outputs a visualization result of a black background and white cracks, which can accurately quantify key parameters such as crack length, width, and opening level.

[0042] This invention employs a progressive hierarchical pre-training method, combined with a step-by-step nonlinear learning rate decay strategy, to achieve precise adaptation of the model from general crack recognition to sluice gate-specific crack recognition, significantly improving adaptability and enabling it to meet the needs of sluice gate crack recognition with different widths and shapes.

[0043] This invention employs an end-to-end deep convolutional neural network algorithm throughout the entire process, eliminating the need for manual preprocessing, significantly improving detection efficiency, reducing labor costs, and enabling automated and intelligent detection of cracks in sluice gates.

[0044] This invention constructs an error feedback-driven iterative optimization mechanism, periodically collects new samples to build an incremental training set, and combines transfer learning and joint loss function to achieve dynamic model updates, ensuring the stability and accuracy of long-term monitoring.

[0045] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0046] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0048] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0049] Figure 1 This is a flowchart of an intelligent method for identifying cracks in sluice gate structures based on multi-scale feature fusion.

[0050] Figure 2 This is an architecture diagram of an intelligent crack identification method for sluice gate structures based on multi-scale feature fusion;

[0051] Figure 3 This is an architecture diagram of the Deep-Crack crack recognition model in this application;

[0052] Figure 4 This is a schematic diagram of the installation of the monitoring equipment in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram illustrating the crack recognition effect in an embodiment of this application;

[0054] Figure 6 This is a data diagram illustrating the model training process for each method in this application;

[0055] Figure 7 This is a comparison chart of the identification precision and recall data of each method in this application over 10 cycles;

[0056] Figure 8 This is a comparison chart of the overall performance data of the various methods in this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0058] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0059] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0060] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0061] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0062] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0063] Example 1 mainly describes a method for intelligent identification of cracks in sluice gate structures based on multi-scale feature fusion, such as... Figure 1 As shown, it specifically includes:

[0064] A training set was constructed based on a publicly available dataset of concrete cracks; a test set was constructed by collecting monitoring images of the surface of a sluice gate structure.

[0065] A Deep-Crack crack recognition model is constructed using a multi-scale feature fusion deep convolutional neural network algorithm. The Deep-Crack crack recognition model is pre-trained based on the training set, and the pre-training adopts a progressive hierarchical pre-training method.

[0066] The monitoring images of the test set are input into the pre-trained Deep-Crack crack recognition model. The model performs hierarchical feature extraction and multi-scale feature fusion processing on the monitoring images in sequence to obtain a fused feature map. Based on the fused feature map, the model performs pixel-level prediction and recognition of sluice gate cracks to obtain the recognition result.

[0067] Regularly collect monitoring images, predict and identify results, and iteratively optimize the Deep-Crack crack identification model.

[0068] Furthermore, the training set is based on a publicly available concrete crack dataset, which includes concrete crack images under different working conditions and environments, covering different crack widths (micro-cracks, fine cracks, medium cracks, wide cracks), different crack morphologies (continuous cracks, discontinuous cracks), and different background environments (no interference, slight noise, shadow interference). At the same time, it is supplemented with crack samples specifically for sluice gate structures (crack images collected from the sluice gate site). All samples have been annotated at the pixel level, and the annotation content includes crack area, crack type, and interference area (stone particles, black spots, etc.).

[0069] Furthermore, the test set consists entirely of on-site monitoring images of sluice gate structures, covering key structural parts of sluice gates such as the Kangshan Flood Diversion Gate. The images include real interference factors on-site (shadows, pebbles, black spots, water stains, uneven lighting), covering sluice gate crack images of different crack types and different degrees of deterioration. There is no artificial intervention or preprocessing, preserving the original image characteristics on-site.

[0070] Furthermore, the progressive layered pre-training method is as follows: the backbone network of the Deep-Crack crack recognition model is initially pre-trained, the shallow parameters of the network are frozen, and the deep feature extraction layer and output layer of the network are trained to obtain the initial pre-trained model.

[0071] A subset of sluice gate structure crack samples is extracted from the training set, and the initial pre-trained model is fine-tuned in a second layer. Shallow parameters are gradually unfrozen, and a step-wise learning rate decay strategy is adopted to adapt the model from general crack recognition to sluice gate-specific crack recognition, thus obtaining the target pre-trained model.

[0072] Furthermore, the second-stage layer fine-tuning is as follows: First, unfreeze the parameters of the middle layer of the network and train them together with the deep feature extraction layer and the output layer for 10 to 20 rounds, using a step-wise learning rate decay strategy; then unfreeze the parameters of the shallow layer of the network and train the shallow, middle, deep, and output layer parameters together for 10 to 20 rounds, maintaining a step-wise learning rate decay during the process.

[0073] Furthermore, the step-wise learning rate decay strategy is as follows: based on the phased training requirements of secondary layer fine-tuning, combined with the loss feedback adjustment term, the round decay term, and the layer adaptation coefficient, the learning rate is adaptively adjusted, and the formula is:

[0074]

[0075] in, For the first Wheel, First The learning rate for the two phases of training (two phases of secondary layered fine-tuning). The initial learning rate, Based on the attenuation coefficient, It is a natural constant. For round decay term, To fine-tune the adaptation coefficients in a layered manner, It is the hyperbolic tangent function. For the first The loss of the wheel, This is the average loss. The standard deviation of the training loss. This is a loss feedback adjustment term.

[0076] Furthermore, the loss feedback adjustment term is specifically: when When the value is small, the decay rate is gradual, ensuring initial parameter adaptation; when... When the value is large, the decay rate gradually increases, driving the model to converge quickly;

[0077] The specific layer adaptation coefficient is as follows: when s=1, it corresponds to the unfreezing training phase of the middle layer parameters. =1.0, at which point the learning rate decays slowly, ensuring the adaptability of the mid-layer parameters to the deep layer parameters and the output layer parameters; when s=2, it corresponds to the shallow layer parameter unfreezing training phase. =0.1, the learning rate is reduced and decayed overall to avoid drastic fluctuations in shallow parameters, protect the general crack features of the initial pre-training, and at the same time adapt to the fine-tuning requirements of the crack features specific to sluice gates.

[0078] The round decay term specifically aims to: achieve abrupt adjustment of the learning rate and quickly suppress oscillations; when the deviation is small, The output approaches 0. Approaching 1, the learning rate is kept to decay steadily, achieving adaptive regulation.

[0079] Furthermore, the progressive hierarchical pre-training process employs a joint loss function combining Dice loss and cross-entropy loss, as shown in the following formula:

[0080]

[0081]

[0082]

[0083] in, For joint losses, For cross-entropy loss, For Dice's loss, The coefficients of the Dice loss are... Total number of pixels For pixels The true label, For pixels The prediction confidence level This represents the actual crack area. To predict the crack area.

[0084] Furthermore, such as Figure 3 The diagram shown illustrates the architecture of the Deep-Crack crack recognition model. Specifically, the Deep-Crack crack recognition model is constructed using a multi-scale feature fusion deep convolutional neural network algorithm. It inputs monitoring image data from the test set, performs hierarchical feature extraction on the monitoring images, obtaining low-level detail features and high-level semantic features respectively. It then fuses the low-level detail features and high-level semantic features at multiple scales to generate a fused feature map. Based on the fused feature map, it performs pixel-level crack prediction and outputs a visually accurate crack recognition image.

[0085] Furthermore, the multi-scale feature fusion deep convolutional neural network algorithm specifically includes an input layer, a shallow convolutional layer, a middle convolutional layer, a deep convolutional layer, a feature fusion layer, and an output layer. The algorithm is based on a deep convolutional neural network framework, integrating a hierarchical feature extraction module, channel attention weighting, multi-scale fusion, and pixel-level prediction. It differentially extracts features from different levels and performs adaptive fusion to perform end-to-end sluice gate crack identification.

[0086] Furthermore, it has 1-3 shallow convolutional layers and 4 or more deep convolutional layers, and each convolutional layer is equipped with a batch normalization (BN) module;

[0087] The shallow convolutional layers have a kernel size of 3×3, a stride of 1, and a padding method of "same". Each convolutional layer is followed by a BN module and a ReLU activation function, and the output feature map has 64-128 channels.

[0088] The kernel size of the middle convolutional layer is 3×3, the stride is 1, the padding method is same, the BN module and the ReLU activation function are configured, the number of output feature map channels is 128-256, and the number of layers is 1-2.

[0089] The deep convolutional layers have a kernel size of 3×3, a stride of 1-2, and use the same padding method. Each convolutional layer is followed by a BN module and a ReLU activation function. The output feature map has 256-512 channels and 4 or more layers, including 1-2 pooling layers (pooling kernel size 2×2, stride 2).

[0090] The feature fusion layer adopts a structure combining channel attention weighting and cross-scale concatenation, including a global average pooling layer (the pooling kernel size is the same as the input feature map size), a 2-layer fully connected network (the number of neurons in the first fully connected layer is 1 / 4 of the number of input channels, and the number of neurons in the second fully connected layer is the same as the number of input channels), configured with ReLU activation function and Sigmoid activation function, and the fusion method is channel-dimensional concatenation;

[0091] The output layer uses a bilinear interpolation upsampling module (the upsampling factor is the same as the total downsampling factor of the deep convolutional layers), the output layer convolutional kernel size is 1×1, the stride is 1, the padding method is same, the sigmoid activation function is configured, the output feature map size is the same as the input monitoring image size, and the number of output channels is 1.

[0092] The global average pooling of channel attention outputs a channel feature vector with the same dimension as the number of channels in the input feature map. Typically, a two-layer fully connected network is used to calculate the attention weights, corresponding to the weight matrix of the first fully connected layer. The dimension is (number of input channels, number of input channels / 4), and it is a second-layer fully connected network. The dimension is (number of input channels / 4, number of input channels).

[0093] Furthermore, hierarchical feature extraction is performed through the collaborative use of shallow, mid, and deep convolutional layers. The shallow convolutional layers extract low-level detail features from the monitoring image, including edges, gradients, textures, and subtle crack contours. The mid-level convolutional layers connect the shallow and deep layers to adapt the sluice gate-specific crack features, achieving a transition from general features to specific features. The deep convolutional layers extract high-level semantic features from the monitoring image, including the global direction of cracks, continuous structure, and regional semantic information. During the extraction process, batch normalization (BN) algorithm is used to optimize feature extraction accuracy and model stability. The formula is:

[0094]

[0095] in, For batch normalized output feature values, , The scaling and offset parameters are determined during pre-training. The output features of the convolutional layer This is the batch average. For batch variance, It is a very small constant.

[0096] Furthermore, the multi-scale feature fusion processing is specifically as follows: Through the feature fusion layer, a fully connected network is used to calculate channel attention weights. Based on these channel attention weights, low-level detail features and high-level semantic features are weighted. A cross-scale concatenation algorithm is then used to perform multi-scale feature fusion processing, generating a fused feature map. The fusion formula is:

[0097]

[0098] in, To fuse feature maps, This is a low-level detail feature map. For high-level semantic feature maps, , These are the feature weights for adaptive learning in the channel attention mechanism. For channel-by-channel weighted calculation, This involves splicing the data along the channel dimension.

[0099] Furthermore, pixel-level prediction and recognition of sluice gate cracks are achieved as follows: The fused feature map is upsampled using bilinear interpolation through the output layer to restore it to the input image size. Then, a Sigmoid activation function is used to output the crack probability map, as shown in the formula:

[0100]

[0101] in, For pixels The confidence level of the crack. For fusion of feature maps Local characteristics;

[0102] based on Binarization segmentation with adaptive thresholding is performed using the following formula:

[0103]

[0104] in, Pixel labels for the binarized segmentation results of the sluice gate cracks. The adaptive threshold is determined during pre-training, and the formula is:

[0105]

[0106] in, This represents the number of samples in the training set. For the first The set consisting of the crack confidence scores of all pixels in the training sample image. For the first The median of all confidence levels for the sample images. This is the threshold adjustment coefficient. These represent the maximum confidence and minimum confidence of a pixel, respectively.

[0107] Based on pixel tags, the location, shape, and extension direction of cracks are accurately depicted on a black background image, achieving a precise visual representation of cracks.

[0108] Furthermore, the precise visualization includes: pixel-level segmentation of cracks, crack location coordinates (two-dimensional coordinates with the sluice gate structure reference point as the origin), crack length (calculated using Euclidean distance), average crack width (converted using pixel spacing), and crack opening level (divided into micro cracks ≤0.2mm, fine cracks 0.2~0.5mm, medium cracks 0.5~1.0mm, and wide cracks >1.0mm based on width).

[0109] Further iterative optimization involves: collecting monitoring images of newly added sluice gate structures and corresponding prediction and recognition results at fixed intervals; filtering samples with recognition errors greater than a preset threshold (5%~10%) and constructing an incremental training set together with the newly added samples; labeling and augmenting the incremental training set; employing a transfer learning strategy; freezing the core parameters of the Deep-Crack model backbone network; and updating only the parameters of the feature fusion layer and output layer; calculating the recognition accuracy and recall of the model before and after iteration; and constructing an error feedback function. The formula is:

[0110]

[0111] in, For feedback error, This is a true positive (correctly identified crack). This is a true negative (correctly identifying a non-crack). This is a false positive (a non-crack was mistakenly identified as a crack). The result was a false negative (the crack was not detected).

[0112] If error If the error exceeds the preset threshold, repeat the iterative optimization until the error E is below the threshold, thus completing the model iterative optimization.

[0113] Furthermore, the incremental training set is enhanced by random rotation (0°~360°), random cropping (cropping ratio 0.7~1.0), brightness perturbation (perturbation range ±15%), contrast perturbation (perturbation range ±20%), Gaussian noise superposition (variance 0.01~0.05), and shadow simulation. This enhances the data in the incremental training set, simulates the complex lighting and environmental interference at the sluice gate site, and improves the robustness of the model after iteration.

[0114] This embodiment details an intelligent crack recognition method for sluice gate structures based on multi-scale feature fusion. A training set is constructed using publicly available concrete crack datasets and sluice gate-specific crack samples, while a test set is built using on-site monitoring images of the sluice gate. A progressive hierarchical pre-training method is employed to pre-train the Deep-Crack crack recognition model, combined with a stepped learning rate decay strategy to achieve hierarchical fine-tuning, adapting it to the specific crack recognition requirements of sluice gates. The model uses a multi-scale feature fusion deep convolutional neural network to perform hierarchical feature extraction and fusion on the test images, combining pixel-level prediction and recognition to output a visual result of a black background and white cracks. Data is periodically collected to construct an incremental training set, and transfer learning and error feedback mechanisms are used to iteratively optimize the model, while a joint loss function ensures recognition accuracy.

[0115] Example 2, based on Example 1, describes in detail the implementation process of intelligent identification of cracks in a flood diversion gate structure using the method of the present invention, taking a flood diversion gate as the implementation object. The specific details are as follows:

[0116] like Figure 4 The diagram showing the installation of monitoring equipment indicates that 8 image AI crack gauges and 6 machine vision intelligent measuring instruments were deployed at the flood diversion gate site to collect original monitoring images of the sluice gate structure surface, covering key parts such as the gate chamber bottom plate, side walls, and gate. The images contain interference such as on-site shadows, water stains, stones, black spots, and uneven lighting. A test set was constructed using 800 unprocessed original monitoring images from the flood diversion gate site.

[0117] The training set is based on the DeepCrack public crack segmentation dataset from the GitHub open-source repository and the CrackTree260 crack dataset from OneDrive. It is supplemented with crack samples from the flood diversion gate site, and pixel-level annotations are completed. It covers micro cracks, fine cracks, medium cracks, wide cracks, as well as continuous cracks and discontinuous cracks, with a total of 3200 samples.

[0118] A multi-scale feature fusion deep convolutional neural network is constructed. The shallow convolutional layers consist of 3 layers with 3×3 kernels and 64, 64, and 128 channels respectively; the middle convolutional layers consist of 1 layer with 3×3 kernels and 128 channels; and the deep convolutional layers consist of 4 layers with 3×3 kernels and 256, 256, 512, and 512 channels respectively. The feature fusion layers are configured with channel attention weighting and cross-scale concatenation. The output layer is configured with 1×1 convolution and sigmoid, outputting a single-channel binary segmentation map. Each convolutional layer is configured with a batch normalization (BN) module and a ReLU activation function.

[0119] The Deep-Crack crack recognition model was pre-trained using a cross-domain progressive hierarchical pre-training method. The process was as follows: shallow layer parameters were frozen, and the deep and output layers were trained for 35 epochs; middle layer parameters were unfrozen, and the model was jointly trained with the deep and output layers for 35 epochs; shallow layer parameters were unfrozen, and the entire network was collaboratively trained for 30 epochs. The pre-training process employed a non-linear stepwise learning rate decay strategy, with an initial learning rate of... The basic attenuation coefficient is 0.9, and the layered adaptation coefficient is... .

[0120] The test set is input into the pre-trained model. The shallow convolutional layer outputs a low-level detail feature map with 128 channels, which extracts edge, texture, and fine crack contour features. The deep convolutional layer outputs a high-level semantic feature map with 512 channels, which extracts the global direction of cracks and continuous structural features.

[0121] The feature fusion layer adopts a channel attention weighting combined with a cross-scale concatenation structure. The output channel vectors of the global average pooling layer have dimensions of 128 and 512, respectively. The first fully connected layer has 32 and 128 neurons, with weight matrices W1 having dimensions of (128, 32) and (512, 128), and the activation function is ReLU. The second fully connected layer has 128 and 512 neurons, with weight matrices W2 having dimensions of (32, 128) and (128, 512), and the activation function is Sigmoid. After channel-wise weighting, the layers are concatenated along the channel dimensions to obtain a fused feature map with 640 channels.

[0122] The fused feature map is upsampled to the original size of the input image using bilinear interpolation. A sigmoid activation function is used to output pixel-level crack confidence scores. Adaptive thresholding is employed for binarization segmentation. The threshold formula uses 3200 samples, a threshold adjustment coefficient of 0.2, and threshold values ​​between 0.5. The output is as follows: Figure 5 The image shown is a pixel-level segmented image with a black background and white crack lines.

[0123] The model collects newly added monitoring images of the Kangshan Flood Diversion Gate every 15 days. Samples with identification errors greater than 5% are selected to construct an incremental training set. Data augmentation is performed using random rotation from 0° to 360°, random cropping ratio of 0.8, brightness perturbation of ±15%, contrast perturbation of ±20%, Gaussian noise variance of 0.03, and shadow simulation. The parameters of the backbone network are frozen and only the parameters of the feature fusion layer and the output layer are updated. The cross-entropy and Dice joint loss function is adopted with a coefficient of 0.5. Iterative optimization is completed with the error feedback function less than 5% as the convergence condition.

[0124] Based on on-site measurements at the flood diversion gate, the model achieved an accuracy rate of 96.2% over a 150-day implementation period. Under shadow interference, the accuracy rate was 93.7%, with a false positive rate of only 1.2% for stones and black spots. It could identify cracks as small as 0.15mm wide, with a crack width measurement error of no more than 0.05mm and a length measurement relative error of no more than 2.1%. The inference time for a single image was no more than 68ms. This model enables fully automated, high-precision pixel-level identification and parameter quantification of sluice gate cracks in complex on-site environments, meeting the engineering requirements for intelligent structural health diagnosis and risk warning.

[0125] This embodiment details the implementation process of intelligent identification of structural cracks in a flood diversion gate using the method of this invention. It includes the deployment of on-site monitoring equipment, the construction of training and testing sets, the specific structural parameters of the Deep-Crack neural network model, the cross-domain progressive hierarchical pre-training process, the specific implementation of hierarchical feature extraction and channel attention-weighted multi-scale feature fusion, pixel-level prediction and adaptive threshold segmentation, and model iterative optimization. Data measured on-site at the Kangshan flood diversion gate is provided, verifying the technical feasibility, identification accuracy, anti-interference ability, and engineering practicality of this patented method under complex flood diversion gate conditions.

[0126] Example 3, based on Example 1 or 2, describes in detail a comparative experiment conducted simultaneously at the flood diversion gate site described in Example 1, using existing methods GCD-Sensor (gate crack sensing detection method), Dam-RCNN (dam crack deep learning detection method), and Concrete-Seg (concrete crack image segmentation method). The details are as follows:

[0127] In the comparison scheme, GCD-Sensor (gate crack sensing detection method) uses sensing components to scan and collect data on the surface of the sluice gate structure, completes the positioning of measuring points through mechanical drive, records the crack location signal and generates detection marks, determines the existence state of the crack based on the sensing data, and outputs the detection results and location information.

[0128] Dam-RCNN (Deep Learning Method for Dam Crack Detection) constructs a convolutional neural network to extract image features, generates detection anchor boxes through a region generation network, and completes feature processing using a multi-scale feature fusion module to locate and classify crack regions in the image, outputting crack detection results.

[0129] Concrete-Seg (concrete crack image segmentation method) acquires multi-view images of concrete structures, performs image stitching and feature enhancement processing, uses transfer learning to complete network training and semantic segmentation, generates crack segmentation masks, and extracts crack regions and morphological parameters based on the segmentation results.

[0130] The comparative experiment lasted 150 days, with each model having an update iteration cycle of 15 days. Data from all comparative methods were collected throughout the experiment, and combined with the implementation data of our method, resulting in the data shown in Tables 1-4 below:

[0131] Table 1. Model training process data for each method

[0132] Comparison indicators GCD-Sensor Dam-RCNN Concrete-Seg This patented method Convergence iteration rounds 78 85 89 79 Convergence accuracy 92.9% 93.4% 94.8% 98.6%

[0133] According to Table 1 and Figure 6 As shown in the model training process data graphs of each method, the proposed method only requires 79 iterations to achieve convergence, which is significantly faster than Dam-RCNN (85 iterations) and Concrete-Seg (89 iterations), and close to the convergence speed of GCD-Sensor (78 iterations). In terms of final detection accuracy, the proposed method achieves 98.6%, which is significantly higher than GCD-Sensor (92.9%), Dam-RCNN (93.4%) and Concrete-Seg (94.8%). This indicates that the proposed method has the dual advantages of fast convergence speed and high fitting accuracy, and its iterative training efficiency and model performance are both superior to existing comparative algorithms.

[0134] Table 2. Identification accuracy data for each method over 10 periods.

[0135] Optimization cycle GCD-Sensor Dam-RCNN Concrete-Seg This method 1 75.80% 87.84% 86.19% 91.87% 2 76.81% 89.30% 87.83% 93.05% 3 77.56% 90.36% 88.97% 93.90% 4 78.17% 91.18% 89.87% 94.59% 5 78.68% 91.84% 90.61% 95.17% 6 79.12% 92.39% 91.24% 95.66% 7 79.50% 92.86% 91.79% 96.09% 8 79.84% 93.27% 92.28% 96.47% 9 80.15% 93.64% 92.72% 96.81% 10 80.43% 93.97% 93.13% 97.12%

[0136] In Table 2, the identification accuracy is the proportion of pixels identified as cracks that have been verified to actually be cracks.

[0137] Table 3. Recall data for each method over 10 periods.

[0138] Optimization cycle GCD-Sensor Dam-RCNN Concrete-Seg This method 1 88.40% 82.34% 85.79% 90.47% 2 89.41% 83.80% 87.43% 91.65% 3 90.16% 84.86% 88.57% 92.50% 4 90.77% 85.68% 89.47% 93.19% 5 91.28% 86.34% 90.21% 93.77% 6 91.72% 86.89% 90.84% 94.26% 7 92.10% 87.36% 91.39% 94.69% 8 92.44% 87.77% 91.88% 95.07% 9 92.75% 88.14% 92.32% 95.41% 10 93.03% 88.47% 92.73% 95.72%

[0139] In Table 3, the recall rate is the proportion of actual crack pixels that are correctly identified as cracks.

[0140] According to Tables 2 and 3, and Figure 7 The comparison chart of the recognition precision and recall data of each method over 10 periods shows that each detection method achieves a certain performance improvement through optimization in the early stage. However, the average values ​​of the two core indicators, recognition precision and recall, of this method are the best. It has a faster convergence speed in the early stage and stronger steady-state performance in the later stage. It can maintain a better and more stable comprehensive recognition ability in continuous iterative optimization. Its overall robustness and engineering adaptability are better than existing conventional detection methods.

[0141] Table 4. Combined data for each method

[0142] Comparison indicators GCD-Sensor Dam-RCNN Concrete-Seg This patented method Recognition accuracy 79.5% 92.6% 91.1% 96.2% Recall rate 92.1% 87.1% 90.7% 94.8% F1 score 85.3% 89.7% 90.9% 95.5% Shadow interference state accuracy 68.2% 88.4% 84.6% 93.7% False positive rate for gravel / black spots 12.6% 6.8% 4.3% 1.2% Minimum recognition width 0.24mm 0.16mm 0.13mm 0.08mm

[0143] In Table 4, the F1 score is the harmonic mean of precision and recall, calculated as 2 × (precision × recall) / (precision + recall).

[0144] According to Table 4 and Figure 8 As shown in the comprehensive performance data comparison chart of the various methods, this method outperforms the three comparison methods, GCD-Sensor, Dam-RCNN, and Concrete-Seg, in core indicators such as recognition accuracy, recall, F1 score, and accuracy under shadow interference. At the same time, it has the lowest false positive rate for stones / black spots and has higher recognition accuracy and anti-interference ability under complex working conditions.

[0145] This embodiment describes in detail a comparative experiment conducted at the flood diversion gate site described in Embodiment 1, simultaneously using existing methods and combining them with this method. The results verify that this method, through a progressive hierarchical pre-training method, hierarchical feature extraction of the Deep-Crack model, end-to-end deep convolutional neural network algorithm, and error feedback-driven iterative optimization mechanism, achieves significant advantages in accuracy, recall, F1 score, accuracy under shadow interference, and recognition width. It can adapt to the on-site working conditions of hydraulic structures with multiple interferences and complex lighting, and has outstanding engineering application advantages.

[0146] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for intelligent identification of cracks in sluice gate structures based on multi-scale feature fusion, characterized in that, include: A training set was constructed based on a publicly available dataset of concrete cracks; a test set was constructed by collecting monitoring images of the surface of a sluice gate structure. A Deep-Crack crack recognition model is constructed using a multi-scale feature fusion deep convolutional neural network algorithm. The Deep-Crack crack recognition model is pre-trained based on the training set, and the pre-training adopts a progressive hierarchical pre-training method. The monitoring images of the test set are input into the pre-trained Deep-Crack crack recognition model. The model performs hierarchical feature extraction and multi-scale feature fusion processing on the monitoring images in sequence to obtain a fused feature map. Based on the fused feature map, the model performs pixel-level prediction and recognition of sluice gate cracks to obtain the recognition result. Regularly collect monitoring images, predict and identify results, and iteratively optimize the Deep-Crack crack identification model.

2. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 1, characterized in that, The progressive layered pre-training method specifically involves: performing initial pre-training on the backbone network of the Deep-Crack crack recognition model, freezing the shallow parameters of the network during the process, training the deep feature extraction layer and output layer of the network, and obtaining the initial pre-trained model. A subset of sluice gate structure crack samples is extracted from the training set, and the initial pre-trained model is fine-tuned in a second layer. Shallow parameters are gradually unfrozen, and a step-wise learning rate decay strategy is adopted to adapt the model from general crack recognition to sluice gate-specific crack recognition, thus obtaining the target pre-trained model.

3. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 2, characterized in that, The stepped learning rate decay strategy is as follows: based on the phased training requirements of secondary layer fine-tuning, and combined with the loss feedback adjustment term, round decay term, and layer adaptation coefficient, the learning rate is adaptively adjusted, and the formula is: in, For the first Wheel, First Learning rate during phase training The initial learning rate, Based on the attenuation coefficient, It is a natural constant. For round decay term, To fine-tune the adaptation coefficients in a layered manner, It is the hyperbolic tangent function. For the first The loss of the wheel, This is the average loss. The standard deviation of the training loss. This is a loss feedback adjustment term.

4. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 3, characterized in that, The progressive hierarchical pre-training uses a joint loss function of Dice loss and cross-entropy loss, as shown in the formula: in, For joint losses, For cross-entropy loss, For Dice's loss, The coefficients of the Dice loss are... Total number of pixels For pixels The true label, For pixels The prediction confidence level This represents the actual crack area. To predict the crack area.

5. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 1, characterized in that, The Deep-Crack crack recognition model is specifically constructed using a multi-scale feature fusion deep convolutional neural network algorithm. It inputs monitoring image data from a test set, performs hierarchical feature extraction on the monitoring images to obtain low-level detail features and high-level semantic features, respectively. It then fuses the low-level detail features and high-level semantic features at multiple scales to generate a fused feature map. Based on the fused feature map, it performs pixel-level crack prediction and outputs a visually accurate crack recognition image.

6. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 5, characterized in that, The multi-scale feature fusion deep convolutional neural network algorithm specifically includes an input layer, a shallow convolutional layer, a middle convolutional layer, a deep convolutional layer, a feature fusion layer, and an output layer. The algorithm is based on a deep convolutional neural network framework, integrating a hierarchical feature extraction module, channel attention weighting, multi-scale fusion, and pixel-level prediction. It differentially extracts features at different levels and performs adaptive fusion to identify water gate cracks end-to-end.

7. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 6, characterized in that, The hierarchical feature extraction specifically involves the collaborative use of shallow, mid, and deep convolutional layers. The shallow convolutional layers extract low-level detail features from the monitoring image, including edges, gradients, textures, and subtle crack contours. The mid-level convolutional layers connect the shallow and deep layers to adapt the sluice gate-specific crack features, achieving a transition from general features to specific features. The deep convolutional layers extract high-level semantic features from the monitoring image, including the global crack orientation, continuous structure, and regional semantic information. During the extraction process, a batch normalization algorithm is used to optimize feature extraction accuracy and model stability. The formula is: in, For batch normalized output feature values, , The scaling and offset parameters are determined during pre-training. The output features of the convolutional layer This is the batch average. For batch variance, It is a very small constant.

8. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 7, characterized in that, The multi-scale feature fusion processing is specifically as follows: through the feature fusion layer, a fully connected network is used to calculate the channel attention weights, and the low-level detail features and high-level semantic features are weighted based on the channel attention weights. Multi-scale feature fusion processing is performed through a cross-scale concatenation algorithm to generate a fused feature map.

9. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 8, characterized in that, The pixel-level prediction and identification of sluice gate cracks specifically involves: Upsampling the fused feature map using bilinear interpolation through the output layer to restore it to the input image size; then, using the Sigmoid activation function, outputting a crack probability map, as shown in the formula: in, For pixels The confidence level of the crack. For fusion of feature maps Local characteristics; based on Binarization segmentation with adaptive thresholding is performed using the following formula: in, Pixel labels for the binarized segmentation results of the sluice gate cracks. The adaptive threshold is determined during pre-training, and the formula is: in, This represents the number of samples in the training set. For the first The set consisting of the crack confidence scores of all pixels in the training sample image. For the first The median of all confidence levels for the sample images. This is the threshold adjustment coefficient. These represent the maximum confidence and minimum confidence of a pixel, respectively. Based on pixel tags, the location, shape, and extension direction of cracks are accurately depicted on a black background image, achieving a precise visual representation of cracks.

10. The intelligent identification method for cracks in sluice gate structures based on multi-scale feature fusion according to claim 1, characterized in that, The iterative optimization specifically involves: collecting monitoring images of newly added sluice gate structures and corresponding prediction and recognition results at fixed intervals; filtering samples with recognition errors greater than a preset threshold and constructing an incremental training set together with the newly added samples; performing data annotation and enhancement on the incremental training set; adopting a transfer learning strategy; freezing the core parameters of the Deep-Crack model backbone network and updating only the parameters of the feature fusion layer and output layer; calculating the recognition accuracy and recall of the model before and after the iteration; and constructing an error feedback function. If the error If the error exceeds the preset threshold, repeat the iterative optimization until the error E is below the threshold, thus completing the model iterative optimization.

Images