Air-ground collaborative reservoir dam face crack detection model optimization method

Abstract

The invention discloses an air-ground collaborative reservoir dam surface crack detection model optimization method, which adopts a dam surface crack detection model W facing an unmanned aerial vehicle to carry out dam surface crack detection, and carries out secondary confirmation through an identification result of a dam surface crack detection model G facing a high-point camera, thereby improving the identification accuracy. Meanwhile, corresponding high-resolution images are marked to form high-precision dam face crack marking samples, the samples serve as training samples, optimization training is continuously conducted on the dam face crack detection model W, and therefore the recognition accuracy of the dam face crack detection model W is improved on the premise that a small number of samples are trained.

Description

Technical Field

[0001] The invention belongs to the technical field of dam surface crack detection, and in particular relates to an air-ground coordinated reservoir dam surface crack detection model optimization method. Background Art

[0002] Reservoir dams are an important part of my country's water conservancy infrastructure and play an important role in supporting the development of the national economy. Cracks are one of the most important diseases of dams. Most cracks originate from the surface of the dam body (i.e., the dam face), which will reduce the stability of the dam (such as concrete dams and earth-rock dams). Stress concentration at the cracks may accelerate structural damage and even cause collapse, seriously affecting the durability of the dam structure. Therefore, regular and systematic detection of cracks on the dam surface, timely detection of small cracks and treatment of them can effectively prevent cracks from causing greater damage to the dam structure, which is of great significance to ensuring the safe and stable operation of reservoir dams.

[0003] At present, there are mainly the following methods for detecting cracks on dam surface: 1. Manually obtain images of the dam surface for inspection regularly, identify cracks on the dam surface through images and record the location, length and width of the cracks. This solution is labor-intensive and inconvenient to observe tiny cracks, and is no longer used as the main means of inspection.

[0004] The main research directions of dam surface detection based on images are: building a better detection model (for example, invention patent application CN117994206A), fusing image features of different types of images (for example, invention patent application CN114511788A), optimizing training algorithms (for example, invention patent application CN118608850A), and optimizing preliminary image processing (for example, invention patent application CN116091483A).

[0005] 2. Using satellite images to inspect the dam surface has the advantages of wide coverage and convenience for regular inspection. However, the resolution of satellite images is limited, making it difficult to detect tiny cracks. There are also disadvantages such as large weather influences and complex data processing.

[0006] 3. Use drones to cruise regularly, take aerial photos of the dam surface to obtain several high-definition images, and then use the detection model to identify the high-definition images to determine whether there are cracks on the dam surface and locate them. This solution has the advantages of flexibility and convenience in capturing fine cracks on the dam surface, but the high-definition image data volume is large, the processing is complex, and there are too many texture features, which makes it difficult for the model to identify crack features and is not convenient for quickly and accurately marking cracks. Improving the recognition accuracy requires a large number of samples for training.

[0007] Patent documents for dam surface inspection based on drones include invention patent CN111751380B, invention patent application CN116433580A, invention patent CN113848209B, and invention patent application CN118072193A. These public patent documents all use drones to take high-definition images of the dam surface, which cannot overcome the above technical defects. Summary of the invention

[0008] In view of the technical defects of the existing dam surface crack detection, the present invention proposes an air-ground coordinated reservoir dam surface crack detection model optimization method, constructs dual detection models for drones and high-point cameras respectively, and uses the recognition results of the detection model for high-point cameras to optimize the detection model for drones, thereby improving the recognition accuracy of the detection model for drones.

[0009] An air-ground collaborative reservoir dam surface crack detection model optimization method is proposed to construct a dam surface crack detection model G for high-point cameras and a dam surface crack detection model W for drones. The dam surface crack detection model W is injected into the multispectral drone by the drone ground station, and a shooting task is assigned to it; the multispectral drone inspects the reservoir dam to obtain high-resolution images, and then detects cracks through the dam surface crack detection model W, generates line coordinates of suspected dam surface cracks, and sends them to the drone ground station together with the high-resolution images, where the line coordinates include the crack length, the longitude and latitude of the crack starting point, and the longitude and latitude of the crack end point; According to the line coordinates received by the UAV ground station, the high-point camera is used to collect images of the suspected dam surface crack area, and the dam surface crack detection model G is used to detect the dam surface cracks. The recognition results are used as crack marks of high-resolution images to form a high-precision dam surface crack mark sample; The high-precision dam surface crack marked samples are used as training samples to optimize the dam surface crack detection model W, and then the optimized dam surface crack detection model W is injected into the multispectral UAV using the UAV ground station.

[0010] Furthermore, in the initial stage of crack detection using the dam surface crack detection model W, the classification threshold of cracks is relaxed, and then continuously optimized and gradually adjusted.

[0011] Furthermore, samples that are identified as cracks by the dam surface crack detection model W but not as cracks by the dam surface crack detection model G are manually verified. If cracks are found, they are marked and included in the training samples.

[0012] Furthermore, the dam surface crack detection model W for UAVs adopts the YOLOv5 target detection model.

[0013] Furthermore, the dam surface crack detection model G for high-point cameras adopts the Faster-RCNN model, the diamond convolution Unet model or the MobileNet-SSD model.

[0014] Furthermore, high-point cameras are deployed on both sides of the reservoir dam, and the azimuth of the mobile camera covers the surface of the reservoir dam.

[0015] Furthermore, the sensor of the multispectral drone uses a combination of one or more of the 450nm, 555nm, 660nm, 750nm, and 840nm bands.

[0016] Furthermore, the high-resolution images are centimeter-level high-definition images or sub-meter-level multispectral image fragments.

[0017] The present invention adopts a dam surface crack detection model W for drones to detect dam surface cracks, and performs secondary confirmation through the recognition results of the dam surface crack detection model G for high-point cameras, thereby improving the recognition accuracy; at the same time, the corresponding high-resolution images are marked to form high-precision dam surface crack marked samples, and used as training samples to continuously optimize the dam surface crack detection model W, thereby improving the recognition accuracy of the dam surface crack detection model W under the premise of training with a small number of samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of the air-ground collaborative reservoir dam surface crack detection model optimization method proposed in the present invention; Figure 2 This is a simulated display of the visual range of a camera at a certain high point; Figure 3 This is a high-definition drone image of cracks on the dam surface; Figure 4 for Figure 3 The corresponding high point camera image; Figure 5 for Figure 4 Marks obtained by dam surface crack detection model G. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for the purpose of illustration and description, and are not intended to be exhaustive or to limit the present invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention, and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

[0020] A reservoir dam surface crack detection model optimization method with air-ground collaboration, the main process is as follows Figure 1 As shown in the figure, a dam surface crack detection model G for high-point cameras and a dam surface crack detection model W for drones are constructed.

[0021] The visual range parameters of the camera are shown in Table 1. The visual range of a single camera is Figure 2 As shown, through the combination of multiple cameras, full coverage of the reservoir dam surface can be achieved.

[0022] Table 1 parameter Numeric Horizontal Angle 120° Vertical Angle 50° high 3m The sensors of multispectral drones use a combination of one or more of the 450nm, 555nm, 660nm, 750nm, and 840nm bands.

[0023] The UAV ground control station (GCS) is the core control platform of the UAV system, responsible for key functions such as flight mission planning, real-time monitoring, data reception and processing, etc. In this embodiment, the UAV ground station is used to inject the dam surface crack detection model W onto the multispectral UAV and issue a shooting task to it.

[0024] Multispectral drones can simultaneously output centimeter-level high-definition images and sub-meter-level multispectral image fragments to meet multi-level needs from rapid inspection to detailed analysis. Compared with high-definition images, multispectral image fragments have higher resolution and can observe more subtle cracks; and choosing multispectral image fragments with higher resolution also means that a larger amount of data needs to be processed, so the inspection party can choose one for crack detection according to the needs.

[0025] Multispectral drones inspect reservoir dams and obtain high-definition images and multispectral image clips at the same time. Figure 3 , and then crack detection is performed through the dam surface crack detection model W to generate the line coordinates of suspected dam surface cracks. The line coordinates are shown in Table 2.

[0026] Table 2 content Numeric Crack length (m) 1.2 Starting point longitude 115.879464 Starting point latitude 31.672046 End longitude 115.879651 End point latitude 31.672023 The multispectral UAV sends the line coordinates of suspected dam surface cracks together with high-definition images or multispectral image fragments to the UAV ground station.

[0027] According to the line coordinates received by the UAV ground station, adjust the azimuth of the high point camera to ensure that it is aimed at the suspected crack and collect images. Figure 4 , the dam surface crack detection model G is used to detect dam surface cracks, and the recognition results are used as crack marks of high-definition image / multispectral image fragments to form a high-precision dam surface crack mark sample, see Figure 5The dam surface crack detection model G here has been trained by default, and the recognition accuracy has reached the expected level.

[0028] The high-precision dam surface crack marked samples are used as training samples to optimize the dam surface crack detection model W, and then the optimized dam surface crack detection model W is injected into the multispectral UAV using the UAV ground station. This process can adopt an incremental update method.

[0029] Multispectral drones simultaneously output centimeter-level high-definition images and sub-meter-level multispectral image fragments. Although they are clear to the naked eye, the amount of data is too large. From the perspective of model recognition, either a large number of samples are required for training, or it is difficult to accurately capture the required features. The dam surface crack detection model W in the present invention is initially trained with only a small number of samples, and then put into crack detection. At the same time, the recognition results of the detection model facing the high-point camera are used to assist the dam surface crack detection model W in crack judgment. In the early stage, the possibility of false negatives can be reduced by relaxing the classification threshold.

[0030] According to the line coordinates received by the UAV ground station, the azimuth of the high-point camera is adjusted to ensure that it is aimed at the suspected crack (compared to searching in the entire visual field, it also greatly reduces the workload), and image acquisition is performed, and dam surface crack detection is performed through the dam surface crack detection model G. This is both a secondary confirmation of the recognition result to ensure the recognition accuracy; it is also a reinforcement of the recognition result of the dam surface crack detection model W. With the continuous optimization of the dam surface crack detection model W, the classification threshold can be gradually adjusted.

[0031] Of course, from Figure 3 In the high-definition image, relatively subtle cracks can be seen, but the high-point camera did not capture them, which would lead to the problem that there are actually cracks, but they are confirmed as non-cracks. If such subtle cracks need to be checked, samples that are identified as cracks by the dam surface crack detection model W and non-cracks by the dam surface crack detection model G can be marked separately for manual verification. If there are indeed cracks, they will be marked and included in the training samples.

[0032] When the recognition accuracy of the dam surface crack detection model W has reached the expected level, the optimization process can be stopped, and subsequent regular patrol and identification by drones can be performed directly.

[0033] As for the dam surface crack detection model W for drones and the dam surface crack detection model G for high-point cameras, they can be implemented using existing technologies and are not the core content of the present invention.

[0034] For example, invention patent application CN116433580A adopts the YOLOv5 detection model for dam crack detection, and invention patent application CN117830826A adopts the improved YOLOv5 detection model for dam crack detection.

[0035] For example, invention patent CN113408327B uses Faster-RCNN model for dam crack detection, invention patent CN110544251B uses MobileNet-SSD model for dam crack detection, and invention patent application CN116703856A uses diamond convolution Unet model for dam crack detection.

[0036] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without creative work should fall within the scope of protection of the present invention.

Claims

1. An air-ground collaborative reservoir dam surface crack detection model optimization method, characterized in that: Construct a dam surface crack detection model G for high-point cameras and a dam surface crack detection model W for drones; The dam surface crack detection model W is injected into the multispectral drone by the drone ground station, and a shooting task is assigned to it; the multispectral drone inspects the reservoir dam to obtain high-resolution images, and then detects cracks through the dam surface crack detection model W, generates line coordinates of suspected dam surface cracks, and sends them to the drone ground station together with the high-resolution images, where the line coordinates include the crack length, the longitude and latitude of the crack starting point, and the longitude and latitude of the crack end point; According to the line coordinates received by the UAV ground station, the high-point camera is used to collect images of the suspected dam surface crack area, and the dam surface crack detection model G is used to detect the dam surface cracks. The recognition results are used as crack marks of high-resolution images to form a high-precision dam surface crack mark sample; The high-precision dam surface crack marked samples are used as training samples to optimize the dam surface crack detection model W, and then the optimized dam surface crack detection model W is injected into the multispectral UAV using the UAV ground station.

2. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: In the initial stage of crack detection using the dam surface crack detection model W, the classification threshold of cracks is relaxed, and then it is continuously optimized and gradually adjusted.

3. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 or 2 is characterized in that: For samples that are identified as cracks by the dam surface crack detection model W but not as cracks by the dam surface crack detection model G, they are manually verified. If cracks are indeed present, they are marked and included in the training samples.

4. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The dam surface crack detection model W is uploaded to the multispectral UAV using an incremental update method.

5. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The dam surface crack detection model W for drones adopts the YOLOv5 target detection model.

6. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The dam surface crack detection model G facing the high-point camera adopts the Faster-RCNN model.

7. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The dam surface crack detection model G for high-point cameras adopts the MobileNet-SSD model.

8. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The dam surface crack detection model G facing the high-point camera adopts the diamond convolution Unet model.

9. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: The sensors of multispectral drones use a combination of one or more of the 450nm, 555nm, 660nm, 750nm, and 840nm bands.

10. The air-ground coordinated reservoir dam surface crack detection model optimization method according to claim 1 is characterized in that: High-resolution images are centimeter-level high-definition images or sub-meter-level multispectral image fragments.

Images