Dam underwater structure defect real-time detection method and device under weak computing power condition

Abstract

The invention relates to the technical field of dam underwater structure defect detection, in particular to a dam underwater structure defect real-time detection method and device under a weak computing power condition, and the method comprises the steps: obtaining dam underwater structure defect video data, and extracting defect images to construct a dam underwater structure defect data set; performing defect category and defect area information labeling on the extracted defect image, and dividing a labeled data set into a training set, a verification set and a test set; constructing a dam underwater structure defect detection model by taking YOLOv8n as a basic model, training and evaluating the dam underwater structure defect detection model by utilizing the training set and the verification set, and checking by utilizing the test set; and analyzing the dam underwater structure defect real-time image by using the dam underwater structure defect detection model to obtain the category and position information of the underwater structure defect. According to the method, the defect image of the underwater structure of the dam can be rapidly detected under the condition of weak computing power, and the cost and the safety risk of manual detection are greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater dam structure defect detection, and in particular to a real-time dam underwater structure defect detection method and device under weak computing power conditions. Background Art

[0002] As time goes by, under the long-term erosion of water flow, various defects will inevitably appear in the underwater structure of the dam, affecting the safe operation and benefit of the project. Therefore, it is necessary to further eliminate the safety hazards of dangerous reservoirs and standardize the management of reservoir dam risk removal, reinforcement and safe operation.

[0003] Since emptying the reservoir for inspection will bring huge environmental impact and economic costs, and most projects do not have emptying facilities or conditions, the underwater structural defect detection of dams often needs to be carried out underwater. The detection of existing underwater structural defects of dams can be carried out visually through manual diving, visual inspection or exploration, and underwater robot photography. However, these methods mainly rely on manual judgment of the test results. The large detection range and complex image data will easily cause fatigue to the inspectors, and are greatly affected by subjective factors, resulting in high detection costs but low quality.

[0004] Compared with traditional manual inspection, the automated underwater dam structure defect detection method based on machine vision can produce accurate and stable underwater dam structure defect detection results by automatically analyzing the collected images. Generally speaking, the automated underwater dam structure defect detection system needs to be deployed on an underwater robot platform, which has low hardware configuration, weak computing power and small storage capacity. In order to realize real-time detection of underwater dam structure defects, it is necessary to propose a lightweight detection algorithm with three characteristics: small resource usage, fast reasoning speed and high detection accuracy. Summary of the invention

[0005] The purpose of the present invention is to address the defects of the prior art and provide a real-time detection method and device for underwater structural defects of dams under weak computing power conditions, which can realize rapid detection of underwater structural defect images of dams under weak computing power conditions and greatly reduce the cost and safety risks of manual detection.

[0006] In order to solve the above technical problems, in a first aspect, the present invention provides a real-time detection method for underwater structural defects of a dam under weak computing power conditions, comprising:

[0007] Obtain video data of underwater structural defects of dams, extract defect images from underwater structural defect videos to construct a dataset of underwater structural defects of dams;

[0008] Annotate the extracted defect images with information about defect categories and defect areas, and divide the annotated data set into a training set, a validation set, and a test set;

[0009] Based on the single-stage neural network YOLOv8n as the basic model, a dam underwater structure defect detection model is constructed. The dam underwater structure defect detection model is trained and evaluated using the training set and the validation set, and tested using the test set to obtain a dam underwater structure defect detection model that meets the recognition accuracy requirements.

[0010] The dam underwater structure defect detection model is used to analyze the real-time image of the dam underwater structure defects. The feature information of the defects in the real-time image is extracted to identify and classify the underwater structure defects.

[0011] In some embodiments, in the dam underwater structure defect detection model, the lightweight network MobileNetV3 is used to replace the YOLOv8n backbone feature extraction network.

[0012] Preferably, in the dam underwater structure defect detection model, the MobileNetV3-Small version of the lightweight network MobileNetV3 is used to replace the YOLOv8n backbone feature extraction network.

[0013] In some embodiments, in the underwater structural defect detection model of the dam, the SIoU function is used to replace the original CIoU of YOLOv8n as the loss function of the bounding box regression, and its calculation formula is:

[0014]

[0015] Where: L SIoU is the regression loss function, IoU is the intersection over union ratio between the real box and the predicted box, Δ is the distance cost, and Ω is the shape cost.

[0016] In some embodiments, a method for labeling the extracted defect image with information about defect categories and defect areas includes: using visual labeling software LabelImg to label each image in the dam underwater structure defect data set to generate an xml file, wherein the labeling information includes the defect category, the coordinates of the center point of the rectangular box, and the length and width of the rectangular box, which is converted into the YOLO label format after normalization and saved as a txt file.

[0017] Furthermore, the defect categories include cracks, spalling, exposed reinforcement and holes.

[0018] In some embodiments, an underwater robot equipped with a high-definition camera is used to explore the underwater structure of the dam. The original sampling data is an avi format video with a sampling frame rate of 30 frames per second. The underwater structure defect dataset of the dam is constructed by capturing one picture every five frames.

[0019] In some embodiments, the dam underwater structure defect detection model is used to analyze the real-time image of the dam underwater structure defect to obtain the category and location information of the underwater structure defect, including:

[0020] The underwater structural defect detection model is deployed on an underwater robot platform. The underwater robot platform obtains real-time images of the underwater structural defects of the dam. The underwater structural defect detection model analyzes the real-time images and simultaneously displays the categories and location information of the underwater structural defects.

[0021] In a second aspect, the present invention provides a real-time detection device for underwater structural defects of a dam under weak computing power conditions, comprising:

[0022] Data acquisition module: used to collect video data of underwater structural defects of dams, and extract defect images from underwater structural defect videos to construct underwater structural defect dataset of dams;

[0023] Data processing module: used to annotate the extracted defect images with information about defect categories and defect areas, and divide the annotated data set into training set, validation set and test set;

[0024] Model training module: used to build a dam underwater structure defect detection model based on YOLOv8n, train and evaluate the dam underwater structure defect detection model using the training set and validation set, and test it using the test set to obtain a dam underwater structure defect detection model that meets the recognition accuracy requirements;

[0025] Defect detection module: It is used to analyze the real-time image of the dam underwater structure defects using the dam underwater structure defect detection model, and to identify and classify the underwater structure defects by extracting the feature information of the defects in the real-time image.

[0026] In a third aspect, the present invention provides a non-transitory computer-readable storage medium for storing computer programs or instructions. When the computer program or instructions are executed by a computer, the real-time detection method for underwater structural defects of dams under weak computing power conditions is implemented.

[0027] In a fourth aspect, the present invention provides a computer program product, which includes computer instructions; when part or all of the computer instructions are run on a computer, the real-time detection method for underwater structural defects of dams under weak computing power conditions is executed.

[0028] The beneficial effects of the present invention are:

[0029] 1. The present invention improves the single-stage neural network YOLOv8n to construct a dam underwater structure defect detection model, including constructing a new lightweight backbone network MobileNetV3-Small and incorporating a new loss function SIoU, which can realize rapid detection of dam underwater structure defect images under weak computing power conditions, greatly reducing the cost and safety risks of manual detection.

[0030] 2. Compared with the classic YOLOv5s, YOLOv7 and YOLOv8n, the method of the present invention has the least parameters and the model size is less than 50% of YOLOv8n, which is only 2.53MB. However, the recognition accuracy can reach 98.5% of YOLOv8n (the highest mAP50 value is 0.468), and the reasoning time for a single image is only 6.7ms, and the corresponding FPS value is as high as 149.3 frames / second.

[0031] 3. The method of the present invention has significant advantages such as small computing resource occupation, fast reasoning speed and high detection accuracy. It can be carried on an underwater robot platform to achieve accurate, real-time and automated detection of underwater structural defects of dams. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a work flow chart of the present invention;

[0033] Figure 2 It is a structural diagram of the single-stage neural network YOLOv8n of the present invention. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] Example 1

[0036] like Figure 1 As shown, this embodiment provides a real-time detection method for underwater structural defects of a dam under weak computing power conditions, including:

[0037] S1. Obtain video data of underwater structural defects of dams, extract defect images from the underwater structural defect videos to construct a dataset of underwater structural defects of dams;

[0038] The original sampling data is in avi format with a sampling frame rate of 30 frames per second. The underwater structure defect dataset of the dam is constructed by capturing one picture every 5 frames.

[0039] S2, annotating the extracted defect images with information about defect categories and defect areas, and dividing the annotated data set into a training set, a validation set, and a test set;

[0040] The method for labeling the extracted defect image with information about defect category and defect area includes: using visual labeling software LabelImg to label each image in the dam underwater structure defect data set to generate an xml file, wherein the labeling information includes the defect category, the coordinates of the center point of the rectangular box, and the length and width of the rectangular box, which is converted into the YOLO label format after normalization and saved as a txt file.

[0041] Among them, the defect categories include cracks, spalling, exposed reinforcement and holes. The defect detection model can be established in advance for the identification of defect categories. The defect detection model is used to identify each image in the dam underwater structure defect data set. The method for establishing the defect detection model is as follows: collect the dam underwater structure defect detection video data, extract defect images from the underwater structure defect video, and then manually classify the defect images according to the specific characteristics of different defects, and mark them as categories such as concrete cracks, spalling, exposed reinforcement and holes. Then, the defect image data sets of different categories are divided into training sets, validation sets and test sets. The defect detection model is trained and evaluated using the training sets and validation sets, and tested using the test sets to obtain a defect detection model that meets the recognition accuracy requirements. In the actual detection process, the defect detection model is used to analyze the real-time image of the dam underwater structure defect, and the underwater structure defects are identified and classified by extracting the feature information of the defects in the real-time image.

[0042] S3. Taking the single-stage neural network YOLOv8n as the basic model, the MobileNetV3-Small version of the lightweight network MobileNetV3 is used to replace the backbone feature extraction network of YOLOv8n, and the SIoU function is used to replace the original CIoU of YOLOv8n as the loss function of bounding box regression to build a dam underwater structure defect detection model.

[0043] It should be noted that the single-stage neural network YOLOv8n consists of three modules: backbone network, neck network, and head network. The detailed structure of each module is as follows Figure 2 As shown in Table 1. YOLOv8 is the latest development of the YOLO series. Compared with previous versions, it has significantly improved small target detection and real-time reasoning, and can better balance the interaction between detection speed and accuracy. In addition, YOLOv8 has five scale models: n, s, m, l, and x. The difference between these five models lies in the depth and width of the convolution channel. As the depth and width continue to increase, the corresponding number of parameters and resource calculations will increase significantly, as shown in Table 1. Considering the problems of low hardware configuration, weak computing power, and small storage capacity of the underwater robot platform, YOLOv8n is selected as the basic model.

[0044] Table 1 YOLOv8 models of different scales

[0045]

[0046] MobileNetV3 network generally has two versions: MobileNetV3-Large and MobileNetV3-Small. Considering the limited computing resources on the project site, the MobileNetV3-Small version suitable for weak computing resources is used to replace the YOLOv8n backbone feature extraction network. The specific network structure configuration is as follows:

[0047] Table 2 Structural configuration of MobileNetV3-Small network

[0048]

[0049] Note: conv2d is a two-dimensional convolution operation; bneck is an inverted residual block; SE is an attention mechanism; HS is an h-swish activation function; RE is a ReLU activation function; NBN is no normalization.

[0050] The loss function of YOLOv8n consists of two parts: the bounding box regression loss and the category classification loss, and their respective multiplication by the corresponding weight coefficients. The bounding box regression loss is calculated by the CIoU+DFL function, and the category classification loss is calculated by the binary cross entropy (BCE) loss function. In order to further improve the speed of model training and the accuracy of reasoning, the SIoU function that considers angle cost, distance cost, and shape cost is selected to replace the original CIoU of YOLOv8n as the loss function of bounding box regression. The calculation formula is:

[0051]

[0052] Where: L SIoU is the regression loss function, IoU is the intersection over union ratio of the real box and the predicted box, Δ is the distance cost, and Ω is the shape cost. This loss function restricts the free movement of the shape by optimizing the aspect ratio of the shape, making the convergence faster and the accuracy higher.

[0053] After the construction of the dam underwater structure defect detection model is completed, the training set and validation set are used to train and evaluate the dam underwater structure defect detection model, and the test set is used to test it, so as to obtain the dam underwater structure defect detection model that meets the recognition accuracy requirements.

[0054] In this embodiment, the dam underwater structure defect detection model is trained with the classic YOLOv5s, YOLOv7 and YOLOv8n models on the same data set. The final performance comparison results are shown in Table 3.

[0055] Table 3 Performance comparison results of different detection models

[0056]

[0057] Note: GFLPOPs is the number of floating-point operations per second. The larger the value, the more inference times the model requires and the more complex the model is. mAP50 is the average precision at the intersection-over-union threshold of 50%. The higher the value, the more accurate the model detection is.

[0058] S4. Deploy the underwater structural defect detection model onto an underwater robot platform. The underwater robot platform obtains real-time images of underwater structural defects of the dam. The underwater structural defect detection model analyzes the real-time images and simultaneously displays the categories and location information of the underwater structural defects.

[0059] Example 2

[0060] This embodiment provides a real-time detection device for underwater structural defects of a dam under weak computing power conditions, including:

[0061] Data acquisition module: used to collect video data of underwater structural defects of dams, and extract defect images from underwater structural defect videos to construct underwater structural defect dataset of dams;

[0062] Data processing module: used to annotate the extracted defect images with information about defect categories and defect areas, and divide the annotated data set into training set, validation set and test set;

[0063] Model training module: used to build a dam underwater structure defect detection model based on YOLOv8n, train and evaluate the dam underwater structure defect detection model using the training set and validation set, and test it using the test set to obtain a dam underwater structure defect detection model that meets the recognition accuracy requirements;

[0064] Defect detection module: It is used to analyze the real-time image of the dam underwater structure defects using the dam underwater structure defect detection model, and to identify and classify the underwater structure defects by extracting the feature information of the defects in the real-time image.

[0065] Example 3

[0066] This embodiment provides a non-transitory computer-readable storage medium for storing computer programs or instructions. When the computer program or instructions are executed by a computer, the real-time detection method for underwater structural defects of a dam under weak computing power conditions described in Embodiment 1 is implemented.

[0067] Example 4

[0068] This embodiment provides a computer program product, which includes computer instructions. When part or all of the computer instructions are run on a computer, the real-time detection method for underwater structural defects of a dam under weak computing power conditions described in Embodiment 1 is executed.

[0069] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A real-time detection method for underwater structural defects of dams under weak computing power conditions, characterized in that: include: Obtain video data of underwater structural defects of dams, extract defect images from underwater structural defect videos to construct a dataset of underwater structural defects of dams; Annotate the extracted defect images with information about defect categories and defect areas, and divide the annotated data set into a training set, a validation set, and a test set; Based on the single-stage neural network YOLOv8n as the basic model, a dam underwater structure defect detection model is constructed. The dam underwater structure defect detection model is trained and evaluated using the training set and the validation set, and tested using the test set to obtain a dam underwater structure defect detection model that meets the recognition accuracy requirements. The dam underwater structure defect detection model is used to analyze the real-time image of the dam underwater structure defects. The feature information of the defects in the real-time image is extracted to identify and classify the underwater structure defects.

2. The real-time detection method for underwater structural defects of dams under weak computing power conditions according to claim 1 is characterized in that: In the dam underwater structure defect detection model, the lightweight network MobileNetV3 is used to replace the YOLOv8n backbone feature extraction network.

3. The real-time detection method for underwater structural defects of dams under weak computing power conditions according to claim 2 is characterized in that: In the dam underwater structure defect detection model, the MobileNetV3-Small version of the lightweight network MobileNetV3 is used to replace the YOLOv8n backbone feature extraction network.

4. The real-time detection method for underwater structural defects of dams under weak computing power conditions according to any one of claims 1 to 3 is characterized in that: In the underwater structural defect detection model of dams, the SIoU function is used to replace the original CIoU of YOLOv8n as the loss function of bounding box regression, and its calculation formula is: Where: L SIoU is the regression loss function, IoU is the intersection over union ratio between the real box and the predicted box, Δ is the distance cost, and Ω is the shape cost.

5. The real-time detection method for underwater structural defects of dams under weak computing power conditions according to any one of claims 1 to 3, characterized in that: The method for labeling the extracted defect image with information about defect category and defect area includes: using visual labeling software LabelImg to label each image in the dam underwater structure defect data set to generate an xml file, wherein the labeling information includes the defect category, the coordinates of the center point of the rectangular box, and the length and width of the rectangular box, which is converted into the YOLO label format after normalization and saved as a txt file.

6. The real-time detection method for underwater dam structural defects under weak computing power conditions according to claim 5 is characterized in that: The defect categories include cracks, spalling, exposed reinforcement and holes.

7. The real-time detection method for underwater structural defects of dams under weak computing power conditions according to any one of claims 1 to 3 is characterized in that: An underwater robot equipped with a high-definition camera was used to explore the underwater structure of the dam. The original sampling data was in the avi format with a sampling frame rate of 30 frames per second. The underwater structure defect dataset of the dam was constructed by capturing one picture every five frames.

8. A real-time detection device for underwater structural defects of dams under weak computing power conditions, characterized in that: include: Data acquisition module: used to collect video data of underwater structural defects of dams, and extract defect images from underwater structural defect videos to construct underwater structural defect dataset of dams; Data processing module: used to annotate the extracted defect images with information about defect categories and defect areas, and divide the annotated data set into training set, validation set and test set; Model training module: used to build a dam underwater structure defect detection model based on YOLOv8n, train and evaluate the dam underwater structure defect detection model using the training set and validation set, and test it using the test set to obtain a dam underwater structure defect detection model that meets the recognition accuracy requirements; Defect detection module: It is used to analyze the real-time image of the dam underwater structure defects using the dam underwater structure defect detection model, and to identify and classify the underwater structure defects by extracting the feature information of the defects in the real-time image.

9. A non-transitory computer-readable storage medium, characterized in that: Used to store computer programs or instructions. When the computer programs or instructions are executed by a computer, the real-time detection method for underwater structural defects of a dam under weak computing power conditions as described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The computer program product includes computer instructions; when part or all of the computer instructions are executed on a computer, the real-time detection method for underwater structural defects of a dam under weak computing power conditions as described in any one of claims 1-7 is executed.

Images