Intelligent detection method and system for multiple types of underwater diseases of hydraulic concrete structure

Abstract

The invention discloses an intelligent detection method and system for multiple types of underwater diseases of a hydraulic concrete structure, and the method comprises the steps: taking a single-stage neural network YOLOv8s as a basic model, adding a GAM attention mechanism behind three feature output layers of a backbone network, and strengthening the attention of the model to target features from two dimensions of channels and space; cIoU is replaced by WIoUv3 to serve as a positioning loss function, the bounding box regression balance problem between samples of different qualities is solved, and therefore an accurate and efficient hydraulic concrete structure underwater disease target detection model is constructed. According to the method, accurate identification of various underwater diseases such as cracks, exposed ribs, holes and spalling can be realized only by using optical image data as input, and especially the condition of overlapping of various underwater diseases can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater target detection, and in particular to an intelligent detection method and system for multiple categories of underwater diseases of hydraulic concrete structures. Background Art

[0002] The underwater part of the hydraulic concrete structure is subjected to long-term water erosion and internal stress during its service, and various underwater defects such as cracks, spalling, exposed reinforcement and holes will inevitably occur, affecting the safety and durability of the hydraulic concrete structure. Therefore, it is necessary to strengthen the underwater defect detection and hazard removal and reinforcement work of the hydraulic concrete structure.

[0003] Traditional underwater detection methods mainly rely on visual inspection and exploration by professional divers. Engineering practice shows that this method not only has low detection efficiency, but also easily creates safety hazards and is greatly affected by subjective factors of the inspectors. However, with the continuous improvement of remote control technology and computer performance, many scholars have achieved efficient and contactless underwater disease detection by using underwater robots equipped with high-definition optical cameras to obtain underwater disease images and using deep learning-based target detection algorithms to extract underwater disease information end-to-end.

[0004] There are two types of deep learning-based target detection algorithms: two-stage detection and single-stage detection. The former is a process of roughly generating candidate boxes and then finely extracting information, while the latter is a one-step process of direct classification and regression, which can significantly improve detection efficiency with a small loss of accuracy. YOLOv8s is the most advanced single-stage target detection algorithm in existing general scenarios. For a relatively single target category, it can improve detection efficiency while ensuring high accuracy. However, hydraulic concrete structures often have a variety of underwater diseases such as cracks, spalling, exposed steel bars and holes, and there are occlusions and overlaps between different diseases, resulting in high false detection and missed detection rates when used directly. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and system for intelligent detection of multiple categories of underwater diseases of hydraulic concrete structures. A single-stage neural network YOLOv8s is used as the basic model, a GAM attention mechanism is added after the three feature output layers of its backbone network, and CIoU is replaced by WIoUv3 as the positioning loss function. A target detection model for underwater diseases of hydraulic concrete structures is constructed to achieve accurate identification of multiple underwater diseases such as cracks, exposed reinforcement, holes and spalling, especially when multiple underwater diseases overlap.

[0006] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0007] In a first aspect, the present invention provides a method for intelligently detecting multiple types of underwater diseases of hydraulic concrete structures, comprising the following steps:

[0008] S1: Underwater disease videos of hydraulic concrete structures are obtained at the construction site. Disease images are extracted and screened frame by frame from the underwater disease videos to construct a multi-category underwater disease dataset of hydraulic concrete structures.

[0009] S2: Label each underwater disease image with information about the disease category and the diseased area, and divide the labeled data set into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set and the validation set are used to evaluate the training process of the target detection model, and the test set is used to test the recognition performance of the target detection model.

[0010] S3: Based on the single-stage neural network YOLOv8s as the basic model, a multi-category underwater disease target detection model for hydraulic concrete structures is constructed. Mosaic enhancement is performed on the data set during the training stage to obtain the trained multi-category underwater disease target detection model for hydraulic concrete structures.

[0011] S4: Reacquire a portion of unlabeled underwater disease images of hydraulic concrete structures, use the target detection model obtained in S3 to perform detection, and obtain the category and location information of the underwater disease.

[0012] As a further technical solution of the present invention, in step S1, an underwater robot equipped with a high-definition camera is used to detect underwater defects. The original sampling data is an avi format video with a sampling frame rate of 30 frames per second. An underwater defect data set of hydraulic concrete structures is constructed by capturing one picture every five frames, covering four types of underwater defects, namely cracks, exposed reinforcement, holes and spalling, as well as the overlapping of multiple underwater defects.

[0013] As a further technical solution of the present invention, in step S2, the visual annotation software LabelImg is used to annotate each image in the data set to generate an xml file, and the position of the underwater disease area in the image is determined by the annotated rectangular box, wherein the annotation information includes the underwater disease category, the coordinates of the center point of the rectangular box, and the length and width of the rectangular box, which is converted into the YOLO label format after normalization and saved as a txt file.

[0014] As a further technical solution of the present invention, the specific process of constructing a multi-category underwater disease target detection model for hydraulic concrete structures in step S3 is as follows:

[0015] S3-1: Add a GAM attention mechanism that fuses spatial and channel dimension features after the three feature output layers (C1, C2, C3) of the YOLOv8s backbone network.

[0016] S3-2: The loss function of YOLOv8s consists of two parts: bounding box loss and classification loss, and their respective multiplication by the corresponding weight coefficients. The bounding box loss is calculated by the CIoU+DFL function, and the classification loss is calculated by the binary cross entropy (BCE) loss function. In order to improve the accuracy of bounding box regression, the WIoUv3 function (2023) based on distance attention and dynamic non-monotonic focusing mechanism is used to replace the CIoU function as the loss function of coordinate regression. The calculation method is:

[0017] L WIoUv3 =rL WIoUv1

[0018]

[0019] L WIoUv1 =R WIoU L IoU

[0020]

[0021] Where: L WIoUv3 is the loss function of WIoUv3; r is the dynamic non-monotonic focusing coefficient; β is the abnormality of the anchor box; δ and α are the hyperparameters for controlling r; L WIoUv1 is the loss function of WIoUv1; R WIoU is the regression weighted IoU, which is used to significantly increase the IoU loss of the normal quality anchor box; L IoU is the intersection-over-union loss, which is used to significantly reduce the regression weighted intersection-over-union of high-quality anchor boxes; x and y are the coordinates of the center point of the anchor box; x gt and gt is the center point coordinate of the target frame; W g and H g is the width and height of the minimum bounding rectangle that contains both the anchor box and the target box.

[0022] S3-3: Use transfer learning to apply the pre-trained weights of YOLOv8s in the COCO dataset to the underwater disease target detection model of hydraulic concrete structures.

[0023] S3-4: The specific process of the Mosaic data enhancement method is to first randomly extract four pictures and corresponding labels from the data set each time, then randomly select the splicing reference point, flip, scale and transform the color gamut of the four pictures respectively, and then place them in the upper left, upper right, lower left and lower right positions of the reference point in turn, and splice them into a new combined image. Finally, the coordinates of the detection box that exceeds the boundary are processed to generate the label of the combined image to increase the diversity of the data and improve the robustness of the model.

[0024] In a second aspect, the present invention provides an intelligent detection system for multiple types of underwater diseases of hydraulic concrete structures, comprising:

[0025] Data acquisition module, used to obtain underwater disease videos of hydraulic concrete structures and construct data sets;

[0026] A data processing module, used to annotate the data set with information about the disease category and the diseased area, and divide the annotated data set into a training set, a validation set, and a test set;

[0027] The model building module is used to build a multi-category underwater disease target detection model for hydraulic concrete structures based on the single-stage neural network YOLOv8s as the basic model, add the GAM attention mechanism after the three feature output layers of its backbone network, and replace the CIoU in the bounding box loss function with WIoUv3 as the positioning loss function;

[0028] Model training module, which uses training set and validation set to train and evaluate the multi-category underwater disease target detection model of hydraulic concrete structure, and uses the test set for verification;

[0029] The disease detection module is used to detect unlabeled underwater disease images of hydraulic concrete structures using a multi-category underwater disease target detection model for hydraulic concrete structures, and obtain the category and location information of the underwater diseases.

[0030] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures.

[0031] In a fourth aspect, the present invention provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method for intelligent detection of multiple categories of underwater diseases of hydraulic concrete structures is implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention takes the single-stage neural network YOLOv8s as the basic model, adds the GAM attention mechanism after the three feature output layers of its backbone network, and strengthens the model's attention to target features from two dimensions: channel and space. The CIoU is replaced by WIoUv3 as the positioning loss function to solve the bounding box regression balance problem between samples of different qualities, thereby constructing a target detection model for underwater defects of hydraulic concrete structures, and realizing accurate recognition of various underwater defects such as cracks, exposed reinforcement, holes and spalling, especially when multiple underwater defects overlap. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the work flow chart of the present invention.

[0035] Figure 2 It is a schematic diagram of the target detection model of multiple categories of underwater diseases of hydraulic concrete structures of the present invention.

[0036] Figure 3 This is the structural diagram of the GAM attention mechanism of the present invention.

[0037] Figure 4 This is an example of an image enhanced by Mosaic data.

[0038] Figure 5 The figure is a comparison chart of the detection effect of the method of the present invention and the prior art on multiple types of underwater diseases of hydraulic concrete structures. DETAILED DESCRIPTION

[0039] The specific implementation of the present invention is described below in conjunction with the accompanying drawings to facilitate the understanding of the present invention by those skilled in the art. Obviously, the present invention is not limited to the scope of the specific implementation. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are non-creative work, and all inventions and creations using the invention concept of this law are protected.

[0040] Example 1

[0041] like Figure 1 As shown, this embodiment provides a method for intelligently detecting multiple types of underwater diseases of hydraulic concrete structures, and the specific steps are as follows:

[0042] S1: Underwater disease videos of hydraulic concrete structures are obtained at the construction site. Disease images are extracted and screened frame by frame from the underwater disease videos to construct a multi-category underwater disease dataset of hydraulic concrete structures.

[0043] As a further technical solution of this embodiment, in step S1, the detection surface of the hydraulic concrete structure is selected, and the underwater disease detection work is carried out using an underwater robot equipped with a high-definition camera. Considering the detection environment requirements of the underwater robot, the underwater detection platform generally adopts the full-process video recording mode for detection. The original sampling data is an avi format video with a sampling frame rate of 30 frames per second. The video stream is converted into JPG format image data by intercepting one picture every 5 frames, and a multi-category underwater disease data set of hydraulic concrete structures is constructed, covering four categories of underwater diseases: cracks, exposed reinforcement, holes and spalling, as well as the overlap of multiple underwater diseases.

[0044] S2: Label each underwater disease image with information about the disease category and the diseased area, and divide the labeled data set into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set and the validation set are used to evaluate the training process of the target detection model, and the test set is used to test the recognition performance of the target detection model.

[0045] As a further technical solution of this embodiment, in step S2, the underwater disease image is labeled using visual labeling software LabelImg to distinguish underwater disease units from background units. The position of the underwater disease area in the image is determined by the labeled rectangular frame, wherein the generated xml file includes the underwater disease category, the coordinates of the center point of the rectangular frame, and the length and width information of the rectangular frame, so as to check the correctness of the labeled information. In order to facilitate the subsequent target detection model training, the generated xml file needs to be normalized, converted into the YOLO label format and saved as a txt file.

[0046] S3: Based on the single-stage neural network YOLOv8s as the basic model, a multi-category underwater disease target detection model for hydraulic concrete structures is constructed. Mosaic enhancement is performed on the data set during the training stage to obtain the trained multi-category underwater disease target detection model for hydraulic concrete structures.

[0047] As a further technical solution of this embodiment, the specific process of constructing a multi-category underwater disease target detection model for hydraulic concrete structures in step S3 is as follows:

[0048] S3-1: Add the GAM attention mechanism that integrates spatial and channel dimension features after the three feature output layers (C1, C2, C3) of the YOLOv8s backbone network, such as Figure 2 shown.

[0049] The purpose of the GAM attention mechanism is to reduce information scaling and amplify global dimension interaction features. By adjusting the weights to distinguish the importance of different feature information, the detection performance of the target detection model in multi-category, small target images can be improved. The specific structure is as follows: Figure 3 As shown. Assume that the input feature map F 1 ∈R C*H*W , then the intermediate state F 2 and output F 3 for:

[0050]

[0051] Where: M c is the feature map of channel attention; M s is the feature map of spatial attention; is element-wise multiplication.

[0052] S3-2: The loss function of YOLOv8s consists of two parts: bounding box loss and classification loss, and their respective multiplication by the corresponding weight coefficients. The bounding box loss is calculated by the CIoU+DFL function, and the classification loss is calculated by the binary cross entropy (BCE) loss function. In order to improve the accuracy of bounding box regression, the WIoUv3 function (2023) based on distance attention and dynamic non-monotonic focusing mechanism is used to replace the CIoU function as the loss function of coordinate regression. The calculation method is:

[0053] L WIoUv3 =rL WIoUv1

[0054]

[0055] L WIoUv1 =R WIoU L IoU

[0056]

[0057] Where: L WIoUv3 is the loss function of WIoUv3; r is the dynamic non-monotonic focusing coefficient; β is the abnormality of the anchor box; δ and α are the hyperparameters for controlling r; L WIoUv1 is the loss function of WIoUv1; R WIoU is the regression weighted IoU, which is used to significantly increase the IoU loss of the normal quality anchor box; L IoU is the intersection-over-union loss, which is used to significantly reduce the regression weighted intersection-over-union of high-quality anchor boxes; x and y are the coordinates of the center point of the anchor box; x gt and gt is the center point coordinate of the target frame; W g and H g is the width and height of the minimum bounding rectangle that contains both the anchor box and the target box.

[0058] S3-3: Use transfer learning to apply the pre-trained weights of YOLOv8s in the COCO dataset to the underwater disease target detection model of hydraulic concrete structures.

[0059] S3-4: The specific process of the Mosaic data enhancement method is to first randomly extract four pictures and corresponding labels from the data set each time, then randomly select the splicing reference point, flip, scale and color gamut transform the four pictures respectively, and then place them in the upper left, upper right, lower left and lower right positions of the reference point in turn, splice them into a new combined picture, and finally process the coordinates of the detection box that exceeds the boundary, generate the label of the combined picture, so as to increase the diversity of the data and improve the robustness of the model. The picture example after Mosaic data enhancement is as follows: Figure 4 shown.

[0060] S4: Reacquire a portion of unlabeled underwater disease images of hydraulic concrete structures, use the target detection model obtained in S3 to perform detection, and obtain the category and location information of the underwater disease.

[0061] For four common underwater diseases, namely cracks, exposed reinforcement, holes and spalling, as well as the overlapping of multiple underwater diseases, YOLOv8s, YOLOv7 and YOLOv5s are used as benchmark methods to verify that the method of this embodiment has strong generalization ability and robustness in detecting multiple categories of underwater disease targets. The specific detection effect comparison is shown in the figure below. Figure 5 shown.

[0062] Example 2

[0063] This embodiment provides a multi-category underwater disease intelligent detection system for hydraulic concrete structures, including:

[0064] Data acquisition module, used to obtain underwater disease videos of hydraulic concrete structures and construct data sets;

[0065] A data processing module, used to annotate the data set with information about the disease category and the diseased area, and divide the annotated data set into a training set, a validation set, and a test set;

[0066] The model building module is used to build a multi-category underwater disease target detection model for hydraulic concrete structures based on the single-stage neural network YOLOv8s as the basic model, add the GAM attention mechanism after the three feature output layers of its backbone network, and replace the CIoU in the bounding box loss function with WIoUv3 as the positioning loss function;

[0067] Model training module, which uses training set and validation set to train and evaluate the multi-category underwater disease target detection model of hydraulic concrete structure, and uses the test set for verification;

[0068] The disease detection module is used to detect unlabeled underwater disease images of hydraulic concrete structures using a multi-category underwater disease target detection model for hydraulic concrete structures, and obtain the category and location information of the underwater diseases.

[0069] Example 3

[0070] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for intelligent detection of multiple categories of underwater diseases of hydraulic concrete structures described in Embodiment 1 is implemented.

[0071] Example 4

[0072] This embodiment provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures described in Embodiment 1 is implemented.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0074] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. An intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures, characterized in that: include: S1: Obtain underwater disease videos of hydraulic concrete structures at the project site, extract and filter disease images frame by frame from the underwater disease videos, and construct a multi-category underwater disease dataset for hydraulic concrete structures; S2: Label each image in the underwater disease dataset with information about the disease category and the diseased area, and divide the labeled dataset into a training set, a validation set, and a test set according to a preset ratio. The training set and the validation set are used to evaluate the training process of the target detection model, and the test set is used to test the recognition performance of the target detection model. S3: Based on the single-stage neural network YOLOv8s as the basic model, a multi-category underwater disease target detection model for hydraulic concrete structures is constructed. Mosaic enhancement is performed on the data set during the training phase to obtain a trained multi-category underwater disease target detection model for hydraulic concrete structures. S4: Reacquire a portion of unlabeled underwater disease images of hydraulic concrete structures, perform detection using the target detection model acquired in step S3, and obtain underwater disease categories and location information.

2. The intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures according to claim 1 is characterized in that: In step S1, an underwater robot equipped with a high-definition camera is used to detect underwater defects. The original sampling data is an avi format video with a sampling frame rate of 30 frames per second. An underwater defect dataset of hydraulic concrete structures is constructed by capturing one picture every five frames, covering four types of underwater defects: cracks, exposed reinforcement, holes, and spalling, as well as the overlap of multiple underwater defects.

3. The intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures according to claim 1 is characterized in that: In step S2, the visual annotation software LabelImg is used to annotate each image in the data set to generate an xml file, and the position of the underwater disease area in the image is determined by the annotated rectangular box. The annotation information includes the underwater disease category, the coordinates of the center point of the rectangular box, and the length and width of the rectangular box. After normalization, it is converted into the YOLO label format and saved as a txt file.

4. The intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures according to claim 1 is characterized in that: In step S2, the labeled data set is divided into a training set, a validation set, and a test set according to a preset ratio of 8:1:

1.

5. The intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures according to claim 1 is characterized in that: In step S3, the specific process of constructing a multi-category underwater disease target detection model for hydraulic concrete structures is as follows: S3-1: Add the GAM attention mechanism that integrates spatial and channel dimension features after the three feature output layers of the YOLOv8s backbone network; S3-2: The loss function of YOLOv8s consists of two parts: bounding box loss and classification loss, and their respective multiplication by the corresponding weight coefficients. The bounding box loss is calculated by the CIoU+DFL function, and the classification loss is calculated by the binary cross entropy loss function. In order to improve the accuracy of bounding box regression, the WIoUv3 function based on distance attention and dynamic non-monotonic focusing mechanism is used to replace the CIoU function as the loss function of coordinate regression. The calculation method is: L WIoUv3 =rL WIoUv1 L WIoUv1 =R WIoU L IoU Where: L WIoUv3 is the loss function of WIoUv3; r is the dynamic non-monotonic focusing coefficient; β is the abnormality of the anchor box; δ and α are both hyperparameters for controlling r; L WIoUv1 is the loss function of WIoUv1; R WIoU is the regression weighted IoU, which is used to significantly increase the IoU loss of the normal quality anchor box; L IoU is the intersection-over-union loss, which is used to significantly reduce the regression weighted intersection-over-union of high-quality anchor boxes; x and y are the coordinates of the center point of the anchor box; x gt and gt is the center point coordinate of the target frame; W g and H g is the width and height of the minimum bounding rectangle that contains both the anchor box and the target box; S3-3: Use transfer learning to apply the pre-trained weights of YOLOv8s in the COCO dataset to the underwater disease target detection model of hydraulic concrete structures; S3-4: The specific process of the Mosaic enhancement method is to first randomly extract four pictures and corresponding labels from the data set each time, then randomly select the splicing reference point, flip, scale and transform the color gamut of the four pictures respectively, and then place them in the upper left, upper right, lower left and lower right positions of the reference point in turn, and splice them into a new combined image. Finally, the coordinates of the detection box that exceeds the boundary are processed to generate the label of the combined image to increase the diversity of the data and improve the robustness of the model.

6. An intelligent detection system for multiple types of underwater diseases of hydraulic concrete structures, characterized in that: include: Data acquisition module, used to obtain underwater disease videos of hydraulic concrete structures and construct data sets; A data processing module, used to annotate the data set with information about the disease category and the diseased area, and divide the annotated data set into a training set, a validation set, and a test set; The model building module is used to build a multi-category underwater disease target detection model for hydraulic concrete structures based on the single-stage neural network YOLOv8s as the basic model, add the GAM attention mechanism after the three feature output layers of its backbone network, and replace the CIoU in the bounding box loss function with WIoUv3 as the positioning loss function; Model training module, which uses training set and validation set to train and evaluate the multi-category underwater disease target detection model of hydraulic concrete structure, and uses the test set for verification; The disease detection module is used to detect unlabeled underwater disease images of hydraulic concrete structures using a multi-category underwater disease target detection model for hydraulic concrete structures, and obtain the category and location information of the underwater diseases.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent detection method for multiple categories of underwater diseases of hydraulic concrete structures as claimed in any one of claims 1 to 5.

8. A computer device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method for intelligent detection of multiple categories of underwater diseases of hydraulic concrete structures according to any one of claims 1 to 5 is implemented.

Images