An underwater bridge pier crack detection method, system, device, medium and product

Abstract

The application discloses an underwater bridge pier crack detection method, system, device, medium and product, and relates to the field of image processing. The method comprises the following steps: acquiring underwater bridge pier images shot by an underwater robot in real time; performing data enhancement on the underwater bridge pier images by using an improved CycleGAN network trained, to obtain enhanced images; the improved CycleGAN network comprises an encoder, a CBAM attention mechanism module and a decoder; a dimension increasing and reducing module is arranged between the encoder and the decoder; performing segmentation on the enhanced images by using an improved Deeplabv3+ network trained, to identify cracks on the surface of the underwater bridge pier; the improved Deeplabv3+ network comprises a double attention mechanism module, and the double attention mechanism module is used for performing feature enhancement on multi-scale feature maps. The application can improve the precision of underwater bridge pier crack detection.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, system, device, medium, and product for detecting cracks in underwater bridge piers. Background Technology

[0002] Bridge piers, as a crucial component of cross-river and cross-sea bridges, are prone to cracking under extreme loads and the combined effects of multiple factors. Underwater crack detection faces numerous challenges, including complex environments, diverse crack types, and difficulties in identification. Traditional manual inspection is time-consuming, inefficient, and has a high rate of missed detections. Sensor monitoring is costly, and sonar and radar have limited detection depth, cannot comprehensively inspect deep-water sections, and suffer from large positioning errors and poor adaptability. Utilizing underwater robots for crack detection on bridge pier surfaces represents a future trend in the intelligent development of emergency rescue equipment. Underwater robots play a vital role in underwater engineering construction and maintenance fields such as dam and tunnel foundation inspection, submarine cable and pipeline maintenance, and offshore platform structural monitoring and repair.

[0003] Underwater crack detection technology is of great significance in promoting the intelligent application of robots. However, the relative scarcity of real-world datasets of underwater cracks limits the training and generalization capabilities of advanced technology models such as deep learning, thus affecting the further application and development of the technology. Summary of the Invention

[0004] The purpose of this application is to provide a method, system, equipment, medium, and product for detecting cracks in underwater bridge piers. Based on the improved CycleGAN network and the improved Deeplabv3+ network, the accuracy of underwater bridge pier crack detection can be improved.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a method for detecting cracks in underwater bridge piers, including:

[0007] Real-time acquisition of underwater bridge pier images captured by underwater robots;

[0008] The underwater bridge pier image is augmented using a trained and improved CycleGAN network to obtain an enhanced image. The improved CycleGAN network includes an encoder, a CBAM attention mechanism module, and a decoder. A dimensionality increase-decrease module is set between the encoder and the decoder.

[0009] The enhanced image is segmented using a trained and improved Deeplabv3+ network to identify cracks on the surface of underwater bridge piers. The improved Deeplabv3+ network includes a dual attention mechanism module, which is used to enhance features of multi-scale feature maps.

[0010] Secondly, this application provides an underwater bridge pier crack detection system, comprising:

[0011] The image acquisition module is used to acquire underwater bridge pier images captured by the underwater robot in real time;

[0012] The data augmentation module is used to augment the underwater bridge pier image using a trained improved CycleGAN network to obtain an enhanced image. The improved CycleGAN network includes an encoder, a CBAM attention mechanism module, and a decoder. A dimensionality increase-decrease module is provided between the encoder and the decoder.

[0013] A crack identification module is used to segment the enhanced image using a trained and improved Deeplabv3+ network to identify cracks on the surface of underwater bridge piers; the improved Deeplabv3+ network includes a dual attention mechanism module, which is used to enhance features of multi-scale feature maps.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described underwater bridge pier crack detection method.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described underwater bridge pier crack detection method.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described underwater bridge pier crack detection method.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects:

[0018] This application combines an improved CycleGAN network and an improved Deeplabv3+ network, fully leveraging the domain adaptation capabilities of the CycleGAN network in few-shot learning and the lightweight and high-precision advantages of the Deeplabv3+ network. It effectively solves the interference caused by the few-shot problem in underwater bridge pier crack detection, thus improving the accuracy of underwater bridge pier crack detection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of an underwater bridge pier crack detection method according to an embodiment of this application;

[0021] Figure 2 A flowchart illustrating an underwater bridge pier crack detection method provided in one embodiment of this application;

[0022] Figure 3 A schematic diagram of the structure of the improved CycleGAN network;

[0023] Figure 4 This is a schematic diagram of the CBAM attention mechanism module.

[0024] Figure 5 A schematic diagram of the structure of the improved Deeplabv3+ network;

[0025] Figure 6 This is a schematic diagram of the structure of the high-efficiency channel attention module;

[0026] Figure 7 A schematic diagram of the loss function curve during training of the improved Deeplabv3+ network;

[0027] Figure 8 This is a schematic diagram showing the results of image segmentation using different algorithms;

[0028] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] The underwater bridge pier crack detection method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send underwater bridge pier images captured by an underwater robot to server 104. After receiving the underwater bridge pier images, server 104 uses a trained and improved CycleGAN network to perform data augmentation on the underwater bridge pier images, obtaining an enhanced image. Then, it uses a trained and improved Deeplabv3+ network to segment the enhanced image and identify surface cracks on the underwater bridge piers. Server 104 can then feed back the identified underwater bridge pier surface cracks to terminal 102. In addition, in some embodiments, the underwater bridge pier crack detection method can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly perform crack detection on the underwater bridge pier images taken by the underwater robot, or the server 104 can obtain the underwater bridge pier images taken by the underwater robot from the data storage system and perform crack detection on the underwater bridge pier images taken by the underwater robot.

[0032] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0033] In one exemplary embodiment, such as Figure 2 As shown, an underwater bridge pier crack detection method is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S3. Wherein:

[0034] S1: Real-time acquisition of underwater bridge pier images captured by underwater robots.

[0035] In this embodiment, an operational underwater robot is used to capture images of underwater bridge piers. The operational underwater robot mainly consists of a main frame, an electronic sealing chamber, a propulsion device, a deep learning image processing module, and a lighting system.

[0036] The electronic sealing chamber, propulsion system, lighting system, and deep learning image processing module are all mounted on the main frame, which is bolted to it. Considering the underwater robot's need to meet rigidity and strength requirements, as well as possess certain corrosion resistance and ductility, the main frame is made of high-polymer nylon material. Additionally, four buoyancy materials for balancing are located on the upper part of the main frame. These buoyancy materials also possess certain corrosion resistance and ductility.

[0037] The electronic sealing chamber primarily houses the various drive control boards and testing equipment, ensuring these electronic components are protected from damage caused by seawater pressure and corrosion. Simultaneously, the electronic sealing chamber accounts for a portion of the underwater robot's mass. In this embodiment, the underwater robot will adopt a cylindrical design with a hemispherical head. The hemispherical head is connected to the chamber body using an O-ring seal. The other end of the chamber houses waterproof connectors for various components and waterproof connectors for communication cables. The cylindrical body of the electronic sealing chamber is made of aluminum alloy, while the hemispherical head uses an acrylic hemispherical cover. The electronic sealing chamber houses numerous critical electronic components, such as the underlying hardware drive board, power management system board, and high-performance lithium batteries.

[0038] In addition, to enable a wider observation range, a 1080P pitch underwater camera was placed in the hemispherical observation chamber of the sealed cabin, with enough space to set the camera's pitch and other degrees of freedom; in order to make it easier to observe the environment inside the water park, and to expand the observation range, an external 1080P underwater camera was also installed to observe the surface of underwater bridge piers and piles.

[0039] The propulsion system mainly consists of six propeller thrusters and a bottom-level hardware drive board. The underwater robot achieves multi-attitude control by coordinating the operation of the six propeller thrusters to meet the requirements of stable and flexible image acquisition. Two propeller thrusters are used for vertical levitation control, while the other four are arranged horizontally for propulsion in various directions. In this embodiment, the propeller thrusters utilize airfoil-shaped propellers with good fluid dynamics, which contributes to optimal propulsion. The propeller thrusters are driven by the bottom-level hardware drive board, with each propeller drive signal line connected to this board. Each propeller drive power line is connected to the power management system board, which not only effectively identifies the battery level but also provides functions such as balanced charging and discharging, temperature protection, MOSFET switching, and hardware voltage regulation.

[0040] The deep learning image processing module mainly consists of a camera acquisition device and a deep learning image processing control board. To mitigate the impact of low underwater lighting on image recognition, this embodiment selects a starlight-level low-light camera as the main body of the acquisition device and equips it with an illumination system. The camera data cable is connected to the signal input terminal of the deep learning image processing control board. To expand the robot's detection field of view, the underwater robot will use a two-degree-of-freedom camera gimbal for wide-range searching. The camera and the two-degree-of-freedom gimbal are connected by bolts, enabling the camera module to perform pitch and yaw movements. The camera will capture visual images in front of the underwater robot and transmit this visual image information to the deep learning image processing control board for image processing. The deep learning image processing control board integrates an improved CycleGAN network and an improved Deeplabv3+ network to process the captured image information in real time and transmit the processed image information back for display through the output terminal of the deep learning image processing control board.

[0041] The lighting system mainly consists of two high-power underwater lights. Good lighting conditions are crucial for underwater observation, as water has poor light transmittance and light attenuates rapidly. To meet the functional requirements of observation in low-light underwater environments, the underwater robot is equipped with an auxiliary lighting system. The main body of the lights uses P70HI-LED bulbs and features a 30° angled, pressure-resistant focusing glass lampshade. The overall shell is made of 6061 aluminum alloy and CNC machined, possessing both corrosion resistance and pressure resistance. The lights are evenly distributed on both sides of the main frame.

[0042] S2: The underwater bridge pier image is augmented using a trained, improved CycleGAN network to obtain an enhanced image. For example... Figure 3 As shown, the improved CycleGAN network includes an encoder, a CBAM (Convolutional Block Attention Module) attention mechanism module, and a decoder. A dimensionality increase / decrease module is provided between the encoder and the decoder. The dimensionality increase / decrease module includes a residual connection module and a dimensionality reduction convolution kernel. The CBAM attention mechanism module includes a channel attention module and a spatial attention module.

[0043] This embodiment leverages the feature alignment capabilities of the improved CycleGAN network's encoder to effectively incorporate the style of the real underwater environment used for training into the crack image data to be detected. This process aligns the features of the ground concrete crack images used for training and the underwater crack images to be detected in the feature space, thereby minimizing the potential interference of background differences on crack feature extraction.

[0044] The specific working process of the improved CycleGAN network is as follows:

[0045] like Figure 3 As shown, after the underwater bridge pier image undergoes feature extraction by an encoder composed of convolutions, a residual connection mechanism (implemented by a residual connection module) is used to directly transmit some feature information to the decoder output. This mechanism effectively reduces information loss caused by the increase in network depth. After the convolution operation, batch normalization is uniformly applied to improve training stability, followed by a ReLU activation function to introduce nonlinear characteristics.

[0046] Before the input features enter the decoding stage, the features of the encoder output and the residual connection are fused in the channel dimension. Part of the feature fusion uses 1×1 convolution for channel compression and information mapping. Figure 3 The middle arrows indicate the direction of feature flow and the interlayer connectivity. This enhances information interaction between different features and improves feature representation capabilities. Through the above optimizations, this embodiment effectively reduces feature loss during the transmission of input feature information, improves the generation accuracy of the improved CycleGAN network, and is suitable for image style transfer, medical image enhancement, and other image generation tasks that require preservation of detailed features.

[0047] To enable the encoder of the CycleGAN network to focus more on key feature information, this embodiment introduces a CBAM attention mechanism module in the feature extraction residual block part. The structure of the enhanced residual block is as follows: Figure 4 As shown.

[0048] To enhance feature extraction capabilities, this application designs an improved residual block. Input features are processed through three different paths and then concatenated along the channel dimension to form rich output features. The first path sequentially passes through a channel attention module and a spatial attention module to highlight key channels and spatial location information. The second path performs padding, convolution, and batch normalization operations to extract fine-grained local features and enhance feature representation. The third path directly passes the input features to the output, preserving original information to ensure gradient stability. Finally, the features from the three paths are concatenated, integrating information at different levels to improve the model's ability to perceive and represent crack detail features.

[0049] The CBAM attention mechanism module, based on a feature optimization method combining channel and spatial attention mechanisms, aims to enhance the CycleGAN network's attention to important features, thereby improving its expressive power and recognition accuracy. The CBAM attention mechanism module achieves dynamic weighted processing of features through the combined action of the channel and spatial attention modules.

[0050] After the input features enter the channel attention module, global average pooling and global max pooling are first performed on the feature maps of each channel to extract the overall information of the channel. Subsequently, the pooling results are processed through a shared multilayer perceptron (MLP) to learn the weight relationships between channels, and channel attention weights are generated using a sigmoid activation function. These weights are used to weight the input channel features to highlight key feature information.

[0051] After channel weighting, the input features enter the spatial attention module. In this process, global max pooling and global average pooling are first performed channel-wise on the input features, and the results are concatenated along the channel dimension. Then, convolution operations are used to reduce the dimensionality to single-channel features, and spatial attention weights are generated using the sigmoid function. Finally, these weights are multiplied element-wise with the input features to enhance the expressive power of important regions in the spatial dimension.

[0052] In summary, to further optimize feature propagation, this embodiment introduces a CBAM attention mechanism module, enabling effective superposition of input information as it passes through convolutional layers, residual connections, and the attention module. This design not only maintains the characteristics of global connectivity but also reduces information loss through residual connections, avoiding network performance degradation and thus improving the model's stability and generalization ability.

[0053] S3: The enhanced image is segmented using a trained, improved Deeplabv3+ network to identify surface cracks on underwater bridge piers. The improved Deeplabv3+ network includes a dual attention mechanism module for feature enhancement of multi-scale feature maps. The dual attention mechanism module includes a CBAM attention mechanism module and an Efficient Channel Attention (ECA) module. The CBAM attention mechanism module in the improved Deeplabv3+ network has the same structure as the CBAM attention mechanism module in the improved CycleGAN network.

[0054] This embodiment utilizes a trained, improved Deeplabv3+ network to accurately segment underwater bridge pier images enhanced by an improved CycleGAN network. In traditional neural networks, continuous convolutional processing of images often leads to the loss of image information. Especially in images, crack information typically occupies only a relatively small portion, making it particularly difficult to preserve and extract crack features. To address this issue, this embodiment designs a feature enhancement structure that integrates multiple attention mechanisms based on the original Deeplabv3+ network. The structure of the improved Deeplabv3+ network is as follows: Figure 5 As shown

[0055] The specific working process of the improved Deeplabv3+ network is as follows:

[0056] In this embodiment, the enhanced image is a 512-pixel color crack image. After processing by the MobilenetV2 backbone network, feature maps of different scales are obtained. The 64-pixel feature map undergoes feature enhancement through the CBAM attention mechanism module to improve the focus on key information. Simultaneously, the 32-pixel feature map is introduced through the ECA module to strengthen the feature representation capability between channels. In the feature fusion stage, firstly, an upsampling convolution operation is used to expand the 32-pixel feature map, converting it to a 64-pixel feature map. This 64-pixel feature map is then concatenated with the CBAM attention mechanism-processed 64-pixel feature map along the channel dimension to enhance information representation capability. Subsequently, the fused feature map is further upsampled to expand it to a size of 128 for integration with the feature map output from another branch. Finally, the 128-pixel feature map is concatenated with the upsampled 128-pixel feature map after hollow spatial pyramid pooling processing along the channel dimension, achieving efficient fusion of multi-scale features.

[0057] like Figure 6 As shown, the ECA module balances the capture of local and global information by adaptively selecting the size of the convolutional kernel. The kernel size is dynamically adjusted based on the number of channels in the input feature map. First, the input feature map is fed into a global average pooling layer, where spatial average pooling is performed on the features within each channel, resulting in a channel description vector with the same dimension as the number of channels in the input feature map. Then, this channel description vector is fed into a one-dimensional convolutional layer for processing. Local interaction relationships between channels are modeled through convolutional operations on the local receptive field, thereby generating weight coefficients for each channel. The one-dimensional convolutional operation uses a fixed-size kernel to reduce computational complexity and avoid the large parameter overhead of traditional fully connected networks. Furthermore, the output vector processed by the one-dimensional convolution is fed into a sigmoid activation function to normalize the weight coefficients of each channel, ensuring that the weight values ​​of each channel fall between 0 and 1. Finally, the normalized channel weight coefficients are multiplied one-by-one with the original input feature map according to the channel dimension to achieve channel recalibration of the original feature map, so as to enhance the feature channels with higher weights and suppress the feature channels with lower weights, and finally output the attention-weighted feature map.

[0058] In this embodiment, the training samples used during the training of the improved CycleGAN network and the improved Deeplabv3+ network include ground crack images, underwater crack images generated by the algorithm, and real underwater crack images.

[0059] The training samples consist of 9954 images and 9954 labels, divided into training, validation, and test sets. 4932 ground crack images (448×448 pixels) were divided in an 8:2 ratio. The training set contained 3945 images, and the validation set contained 987 images. Furthermore, an improved CycleGAN network was used to perform style transfer on the 4932 ground crack images, resulting in 4932 underwater crack images, which were also divided proportionally. The remaining 90 images (512×512 pixels, real underwater crack images) served as the test set.

[0060] The training set guides the optimization of model parameters, allowing the model to adjust its internal weights. The validation set is used for evaluation during or after training. The test set is used to evaluate the model's final performance, focusing on verifying its generalization ability and simulating unknown data that the model might encounter in real-world scenarios.

[0061] The improved Deeplabv3+ network uses two loss functions during training: cross-entropy loss and Dice loss.

[0062] Cross-entropy loss function CE loss It is a standard method for evaluating the difference between the predicted probability distribution and the true label distribution in classification tasks.

[0063]

[0064] y i p(y) represents the true label of the current category. i ) represents the probability of the current category, and N represents the number of categories.

[0065] In image segmentation tasks, the cross-entropy loss function directly measures the difference between the probability distribution of the model's output and the true labels, providing a clear objective for model optimization. Since this function directly incorporates the classification result of each pixel into the loss calculation, it effectively promotes the optimization of the model in terms of improving pixel-level prediction accuracy. However, cross-entropy loss calculates the average loss across all classes. When the number of samples for some classes in the dataset is significantly higher than for others, the model tends to optimize those classes that appear more frequently. This phenomenon occurs because, from a probabilistic perspective, improving the prediction accuracy of common classes leads to a more significant reduction in the total loss, making the model less sensitive to classes with fewer samples. In our dataset, since the crack region occupies a small proportion of the entire image, relying solely on cross-entropy loss may lead to a severe class imbalance problem.

[0066] The Dice loss function is a metric that measures the similarity between two sets, and is often used to compare predicted results with true results in image segmentation tasks.

[0067]

[0068] X represents the pixel label of the ground truth segmented image, while Y represents the pixel class predicted by the model for the segmented image. The numerator is approximately the sum of the dot products between the predicted image pixels and the ground truth label image pixels, i.e., the sum of the corresponding pixel products. The denominator represents the sum of the pixels in their respective images. Therefore, the Dice loss focuses on the overlapping region between the prediction and the ground truth label, rather than pure pixel accuracy. This allows the model to naturally focus on fewer positive samples (cracks) during training, thus reducing the excessive influence of the majority class (background). Therefore, the Dice loss is less affected by class imbalance. However, the Dice loss may not be sensitive enough to small target regions or discontinuous target boundaries. When the target region is very small or highly dispersed, even small prediction biases can lead to significant changes in the Dice coefficients. This may cause the model to underlearn these small regions, thus affecting the overall segmentation quality.

[0069] In summary, the loss function Loss of the Deeplabv3+ network during training is:

[0070] Loss = CE loss +Dice loss

[0071] The combination of Dice loss and cross-entropy loss in the Loss function simultaneously optimizes global accuracy and local overlap in image segmentation tasks, improving training stability and overcoming the challenge of class imbalance. This design not only stabilizes the training process but also enhances the model's overall performance in terms of precision, recall, and specificity through the advantages of each loss function. It is particularly suitable for crack image segmentation tasks that require precise contours. Figure 7 As can be seen, the model's loss function eventually converges, ranging from 0.1 to 0.15. This result validates the effectiveness of the generated dataset and the training effectiveness of the improved Deeplabv3+ network.

[0072] To verify the detection performance of the improved Deeplabv3+ network, comparative experiments were conducted on the Deeplabv3+ network before and after the improvement. The experiments consisted of three main parts: first, the training loss curves were compared to verify the improved feature extraction capabilities of the Deeplabv3+ network during training; second, experiments were designed and validated using evaluation metrics for the areas where the model was improved; and finally, the detection results of underwater bridge pier surface defects were presented, visually demonstrating the effectiveness of the improved Deeplabv3+ network.

[0073] First, to verify the effectiveness of the improved loss function, the experimental results of the improved Deeplabv3+ network's loss function are compared with those of the original Deeplabv3+ network trained using only the cross-entropy loss function. This confirms that combining cross-entropy loss and Dice loss can effectively reduce the impact of class imbalance. Next, a dual attention mechanism module is introduced to the structure of the original Deeplabv3+ network, and the results are compared with the original model. Finally, based on this, further comparative experiments are conducted using the improved CycleGAN and the Deeplabv3+ network.

[0074] The ablation test results of the model on the validation set are shown in Table 1. Comparing Experiment 1 and Experiment 2, the results show that combining the cross-entropy loss with the Dice loss function (i.e., CE-dice loss) improves recall and cross-union ratio by 2.35% and 1.48%, respectively. This is because the traditional cross-entropy loss function provides stability during training but has limitations in handling class imbalance. The Dice loss function, by assigning higher weights to minority class targets and lowering the weights to majority class targets, can mitigate the impact of class imbalance to some extent. Therefore, the design of the CE-Dice loss function helps improve the detection performance of crack targets.

[0075] In comparison experiments 1 and 3, the structure of the original Deeplabv3+ network was optimized, and an attention mechanism was introduced to further enhance the feature fusion structure. This optimization improved the model's recall and intersection-union ratio by 0.82% and 1.48%, respectively. The optimized network structure not only preserves the original information of small targets in the image but also enhances the model's ability to extract information about small targets.

[0076] Comparing Experiments 1 and 4, the recall and intersection-over-union ratio (IoU) improved by 2.83% and 1.82%, respectively, after combining loss function optimization and model structure improvement. Finally, comparing Experiments 1 and 5, by combining loss function optimization, network structure improvement, and the introduction of the Cycle-Deeplabv3+ architecture, the recall and IoU were improved by 3.93% and 2.22%, respectively, achieving the best results. This is because the Cycle-Deeplabv3+ architecture can unify the background style of the dataset while preserving the crack features, thereby reducing the interference of the background on crack features. The final results demonstrate that this application significantly improves detection performance by unifying the dataset style while enhancing small object detection, effectively validating the effectiveness of the model improvement.

[0077] Table 1

[0078]

[0079] The model's results on the test set are shown in Table 2. Comparing Experiments 1 and 2, optimizing the loss function improved recall and intersection-over-union (IoU) by 0.32% and 0.31%, respectively. Comparing Experiments 1 and 3, the improved model structure improved recall and IoU by 5.73% and 6.21%, respectively. Comparing Experiments 1 and 4, the combined results of optimized loss function and improved model structure improved recall and IoU by 11.18% and 11.89%, respectively.

[0080] Table 2

[0081]

[0082] However, the first four experiments performed significantly worse on the test set than on the validation set. This is mainly due to the significant difference between the style of real crack images and the style of training images. Although crack features have some similarity, the model failed to effectively identify crack features in real underwater crack images due to limitations imposed by the dataset and the model's own performance, as well as interference from complex backgrounds.

[0083] To further verify the generalization performance of the model, it was compared with traditional segmentation algorithms. The experimental results are shown in Table 3. The results in Table 3 show that this application achieves a good balance between detection performance and lightweight design.

[0084] Table 3

[0085]

[0086] After the model training is completed, a set of images will be randomly selected to visualize the detection results of this application and the aforementioned traditional segmentation algorithm, such as... Figure 8 As shown. Figure 8 The results show that Unet and Segformer, as the algorithms with the most complex network structures, significantly outperform Pspnet and Hrnet in detection performance. However, for more complex underwater crack image recognition, these algorithms perform poorly in crack feature extraction. In contrast, the visualization results of the proposed GAN-Deeplab algorithm demonstrate its advantage in processing such images, showing that it can more accurately extract crack features from underwater crack images with complex backgrounds.

[0087] Based on the same inventive concept, this application also provides an underwater bridge pier crack detection system. The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more underwater bridge pier crack detection system embodiments provided below can be found in the limitations of the underwater bridge pier crack detection method described above, and will not be repeated here.

[0088] In one exemplary embodiment, an underwater bridge pier crack detection system is provided, comprising:

[0089] The image acquisition module is used to acquire underwater bridge pier images taken by the underwater robot in real time.

[0090] The data augmentation module is used to augment the underwater bridge pier image using a trained improved CycleGAN network to obtain an enhanced image. The improved CycleGAN network includes an encoder, a CBAM attention mechanism module, and a decoder. A dimensionality increase-decrease module is provided between the encoder and the decoder.

[0091] A crack identification module is used to segment the enhanced image using a trained and improved Deeplabv3+ network to identify cracks on the surface of underwater bridge piers; the improved Deeplabv3+ network includes a dual attention mechanism module, which is used to enhance features of multi-scale feature maps.

[0092] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device may be a server or a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data to be processed. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a method for detecting cracks in underwater bridge piers.

[0093] Those skilled in the art will understand that Figure 9The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0094] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0095] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0098] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting cracks in underwater bridge piers, characterized in that, include: Real-time acquisition of underwater bridge pier images captured by underwater robots; An improved CycleGAN network, after training, is used to augment the underwater bridge pier image to obtain an enhanced image. The improved CycleGAN network includes an encoder, a CBAM attention mechanism module, and a decoder. A dimensionality increase-decrease module is set between the encoder and the decoder. The dimensionality increase-decrease module includes a residual connection module and a dimensionality reduction convolution kernel. The CBAM attention mechanism module includes a channel attention module and a spatial attention module. After underwater bridge pier images are processed by an encoder consisting of convolutions to extract features, a residual connection mechanism is used to directly transmit some feature information to the decoder output. After the convolution operation, batch normalization is uniformly applied, followed by a ReLU activation function to introduce nonlinear characteristics. Before the input features enter the decoding stage, the encoder output and the residual connection features are fused in the channel dimension. Some feature fusion uses 1×1 convolution for channel compression and information mapping. The enhanced image is segmented using a trained and improved Deeplabv3+ network to identify cracks on the surface of underwater bridge piers; the improved Deeplabv3+ network includes a dual attention mechanism module, which is used to enhance features of multi-scale feature maps; The dual attention mechanism module includes a CBAM attention mechanism module and an efficient channel attention module. After the enhanced image is processed by the MobilenetV2 backbone network, feature maps of different scales are obtained. Among them, the 64-size feature map is enhanced by the CBAM attention mechanism module, and the 32-size feature map is enhanced by the ECA module. In the feature fusion stage, firstly, the 32-size feature map is expanded by upsampling convolution operation to convert it into a 64-size feature map, and then concatenated with the 64-size feature map processed by the CBAM attention mechanism module in the channel dimension. Subsequently, the fused feature map is further upsampled and integrated with the feature map output by another branch. Finally, the feature map expanded to a size of 128 is concatenated with the 128-size feature map after being processed by dilated spatial pyramid pooling and upsampled in the channel dimension.

2. The underwater bridge pier crack detection method according to claim 1, characterized in that, The kernel size of the efficient channel attention module is dynamically adjusted according to the number of channels in the input feature map.

3. The underwater bridge pier crack detection method according to claim 1, characterized in that, The loss function of the improved Deeplabv3+ network during training. for: in, Let cross-entropy be the loss function. The Dice loss function, The true label for the current category. X represents the probability of the current category, N represents the pixel label of the real segmented image, Y represents the number of categories, and Y represents the pixel category predicted by the improved Deeplabv3+ network for the segmented image.

4. An underwater bridge pier crack detection system, characterized in that, include: The image acquisition module is used to acquire underwater bridge pier images captured by the underwater robot in real time; The data augmentation module is used to augment the underwater bridge pier image using a trained improved CycleGAN network to obtain an enhanced image. The improved CycleGAN network includes an encoder, a CBAM attention mechanism module, and a decoder. A dimensionality increase-decrease module is provided between the encoder and the decoder. The dimensionality increase-decrease module includes a residual connection module and a dimensionality reduction convolution kernel. The CBAM attention mechanism module includes a channel attention module and a spatial attention module. After underwater bridge pier images are processed by an encoder consisting of convolutions to extract features, a residual connection mechanism is used to directly transmit some feature information to the decoder output. After the convolution operation, batch normalization is uniformly applied, followed by a ReLU activation function to introduce nonlinear characteristics. Before the input features enter the decoding stage, the encoder output and the residual connection features are fused in the channel dimension. Some feature fusion uses 1×1 convolution for channel compression and information mapping. A crack identification module is used to segment the enhanced image using a trained and improved Deeplabv3+ network to identify cracks on the surface of underwater bridge piers; the improved Deeplabv3+ network includes a dual attention mechanism module, which is used to enhance features of multi-scale feature maps; The dual attention mechanism module includes a CBAM attention mechanism module and an efficient channel attention module. After the enhanced image is processed by the MobilenetV2 backbone network, feature maps of different scales are obtained. Among them, the 64-size feature map is enhanced by the CBAM attention mechanism module, and the 32-size feature map is enhanced by the ECA module. In the feature fusion stage, firstly, the 32-size feature map is expanded by upsampling convolution operation to convert it into a 64-size feature map, and then concatenated with the 64-size feature map processed by the CBAM attention mechanism module in the channel dimension. Subsequently, the fused feature map is further upsampled and integrated with the feature map output by another branch. Finally, the feature map expanded to a size of 128 is concatenated with the 128-size feature map after being processed by dilated spatial pyramid pooling and upsampled in the channel dimension.

5. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the underwater bridge pier crack detection method according to any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the underwater bridge pier crack detection method according to any one of claims 1-3.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the underwater bridge pier crack detection method according to any one of claims 1-3.

Images