Unmanned aerial vehicle-based water conservancy facility patrol and early warning method, system, device and medium

Abstract

The application relates to a water conservancy facility patrol early warning method, system, device and medium based on a UAV, wherein the method comprises the following steps: acquiring an image data set of a water conservancy facility dam based on a UAV platform; constructing a YOLOv5 model, and performing dam detection model training based on the image data set to obtain a first target detection model of the water conservancy facility dam in daytime and a second target detection model of the water conservancy facility dam at night; acquiring a to-be-tested image data set, and inputting the to-be-tested image data set into the first target detection model and the second target detection model to output a target screenshot; cutting the target screenshot into a nine-square grid, and performing dam breach analysis on the water conservancy facility dam based on the area ratio of the dam region in the nine-square grid to obtain an analysis result; and performing patrol early warning on the water conservancy facility based on the analysis result. The application is beneficial to improving the patrol early warning efficiency of the water conservancy facility, and can give an early warning for dam breach.

Description

Technical Field

[0001] This application relates to the field of water conservancy monitoring technology, and in particular to a method, system, equipment and medium for water conservancy facility inspection and early warning based on unmanned aerial vehicles (UAVs). Background Technology

[0002] River embankments are a crucial component of flood control projects, playing a vital role in flood prevention and disaster reduction. However, due to long-term operation and lack of maintenance, embankments are susceptible to seepage and erosion, leading to varying degrees of danger and seriously impacting national property and the safety of people's lives. Therefore, the safety monitoring of the nation's water conservancy facilities is an ongoing process.

[0003] Currently, most water conservancy facilities rely on fixed monitoring, lacking aerial inspection capabilities and infrared inspection methods. The inspection methods are limited and cannot achieve all-weather inspection. They are subject to many limitations and cannot provide early warnings for some areas with continuous damage, resulting in low efficiency in the inspection and early warning of water conservancy facilities. Summary of the Invention

[0004] The purpose of this application is to propose a method, system, equipment and medium for water conservancy facility inspection and early warning based on unmanned aerial vehicles (UAVs), so as to improve the efficiency of water conservancy facility inspection and early warning, realize all-weather inspection and issue early warnings in advance.

[0005] To address the aforementioned technical problems, this application provides a method for unmanned aerial vehicle (UAV)-based inspection and early warning of water conservancy facilities, including:

[0006] Image datasets of water conservancy facilities and dams are acquired based on UAV platforms. The image datasets include optical images of water conservancy facilities and dams during the day and fused images of water conservancy facilities and dams at night. The fused images are formed by fusing infrared images and visible light images.

[0007] A YOLOv5 model was constructed, and a dam detection model was trained based on the optical image to obtain the first target detection model of the water conservancy facility dam during the daytime.

[0008] Based on the fused image, a dam detection model is trained to obtain a second target detection model for water conservancy dams at night.

[0009] The image dataset to be tested of the water conservancy facility dam is obtained based on the UAV platform, and the image dataset to be tested is input into the first target detection model and the second target detection model to output target screenshots;

[0010] The target screenshot is divided into a nine-square grid, and based on the occupancy ratio of the dam area in the nine-square grid, a dam failure analysis is performed on the dam of the water conservancy facility to obtain the analysis results.

[0011] Based on the analysis result, the water conservancy facility is patrolled and warned.

[0012] To solve the above technical problems, the embodiment of the application provides a water conservancy facility patrol and warning system based on a unmanned aerial vehicle, which comprises:

[0013] An image data set acquisition unit is configured to acquire an image data set of a water conservancy facility dam based on a unmanned aerial vehicle platform, wherein the image data set comprises an optical image corresponding to the water conservancy facility dam in daytime and a fusion image corresponding to the water conservancy facility dam at night, and the fusion image is an image formed by fusing an infrared image and a visible light image;

[0014] A first target detection model construction unit is configured to construct a YOLOv5 model and perform dam detection model training based on the optical image to obtain a first target detection model of the water conservancy facility dam in daytime.

[0015] A second target detection model construction unit is configured to perform dam detection model training based on the fusion image to obtain a second target detection model of the water conservancy facility dam at night.

[0016] A target screenshot output unit is configured to acquire a to-be-tested image data set of the water conservancy facility dam based on the unmanned aerial vehicle platform and input the to-be-tested image data set into the first target detection model and the second target detection model to output a target screenshot.

[0017] An analysis result generation unit is configured to cut the target screenshot into a nine-square grid and perform dam breach analysis on the water conservancy facility dam based on a duty cycle of a dam region in the nine-square grid to obtain an analysis result.

[0018] A water conservancy facility patrol and warning unit is configured to perform patrol and warning on the water conservancy facility based on the analysis result.

[0019] To solve the above technical problems, the application adopts a technical scheme, which is to provide a computer device, comprising one or more processors, and a memory configured to store one or more programs, so that the one or more processors implement the water conservancy facility patrol and warning method based on a unmanned aerial vehicle.

[0020] To solve the above technical problems, the application adopts a technical scheme, which is a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the water conservancy facility patrol and warning method based on a unmanned aerial vehicle.

[0021] This invention provides a method, system, device, and medium for inspecting and issuing early warnings for water conservancy facilities based on unmanned aerial vehicles (UAVs). The invention acquires an image dataset of water conservancy facility dams using a UAV platform. This image dataset includes optical images of the dams during the day and fused images of the dams at night. A YOLOv5 model is constructed, and a dam detection model is trained based on the optical images to obtain a first target detection model for the water conservancy facility dams during the day. A second target detection model for the water conservancy facility dams at night is trained based on the fused images. A dataset of images of the water conservancy facility dams to be tested is acquired using the UAV platform and input into the first and second target detection models to output target screenshots. The target screenshots are divided into a nine-grid layout, and based on the dam area's occupancy ratio within the nine-grid layout, a dam failure analysis is performed on the water conservancy facility dams to obtain analysis results. Based on the analysis results, inspections and early warnings are issued for the water conservancy facilities. This application embodiment uses a drone platform to patrol and acquire image datasets, and constructs daytime and nighttime target detection models to achieve all-weather patrols. It can also divide target screenshots into a nine-square grid, and perform dam failure analysis on the dam area of ​​the water conservancy facility based on the occupancy ratio of the dam area in the nine-square grid. This enables continuous patrols of water conservancy facilities, which is beneficial to improving the patrol and early warning efficiency of water conservancy facilities, and can issue early warnings of dam failures. Attached Figure Description

[0022] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of an implementation of a UAV-based water conservancy facility inspection and early warning method provided in an embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating an implementation of a sub-process in the UAV-based water conservancy facility inspection and early warning method provided in this application embodiment;

[0025] Figure 3 This is another implementation flowchart of a sub-process in the UAV-based water conservancy facility inspection and early warning method provided in the embodiments of this application;

[0026] Figure 4 This is another implementation flowchart of a sub-process in the UAV-based water conservancy facility inspection and early warning method provided in the embodiments of this application;

[0027] Figure 5is another implementation flowchart of the sub-process in the water conservancy facility patrol and early warning method based on a UAV provided in an embodiment of the present application;

[0028] Figure 6 is another implementation flowchart of the sub-process in the water conservancy facility patrol and early warning method based on a UAV provided in an embodiment of the present application;

[0029] Figure 7 is a schematic diagram of the water conservancy facility patrol and early warning system based on a UAV provided in an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application; the description and the drawings are to be regarded as illustrative in nature and are not intended to limit the application; the terminology used in the description of the application herein including the abstract is not intended to be limiting of the application and is only used for the purpose of providing constructive reduction to practice of the application. The terms "comprising," "having," "including," and "containing" used in the detailed description and the claims herein are used in their broadest sense and are not intended to limit the application. The terms "first," "second," and the like used in the description and the claims herein are used for distinguishing between similar objects and are not necessarily used in a sequential or chronological sense.

[0032] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely examples from a

[0033] In order to make the technical personnel in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings.

[0034] It should be noted that the water conservancy facility patrol and early warning method based on a UAV provided in the embodiments of the present application is generally executed by a server, and accordingly, the water conservancy facility patrol and early warning system based on a UAV is generally configured in the server.

[0035] Please refer to Figure 1 , Figure 1 shows one specific implementation of the water conservancy facility patrol and early warning method based on a UAV.

[0036] It should be noted that the method of the present application does not necessarily have Figure 1The flow sequence shown is limited, and the method comprises the following steps:

[0037] S1. Obtain an image data set of the water conservancy facility dam based on the unmanned aerial vehicle platform, wherein the image data set comprises an optical image corresponding to the water conservancy facility dam during the day and a fusion image corresponding to the water conservancy facility dam at night, and the fusion image is an image formed by fusing an infrared image and a visible light image.

[0038] The embodiment of the present application is a water conservancy facility patrol early warning system based on an unmanned aerial vehicle platform, which is composed of an unmanned aerial vehicle platform and a background server system. The unmanned aerial vehicle platform involves route design of water conservancy facilities, full-automatic data collection, and data push streaming back to the background server. The background server can realize a water conservancy facility patrol early warning method based on an unmanned aerial vehicle.

[0039] Specifically, the unmanned aerial vehicle autonomously collects video data of the dam along the river bank along the fixed route, and the background server synchronously receives the video data pushed back by the unmanned aerial vehicle in the rtmp format, so as to perform real-time integrity detection of the water conservancy facility dam. Among them, during the day, the unmanned aerial vehicle obtains an optical image corresponding to the water conservancy facility dam through an optical camera; at night, the unmanned aerial vehicle obtains a fusion image corresponding to the water conservancy facility dam through an infrared optical camera, and the fusion image is an image formed by fusing an infrared image and a visible light image. The image data set can be a video image set of the water conservancy facility, or an image set of the water conservancy facility.

[0040] S2. Construct a YOLOv5 model, and perform dam detection model training based on the optical image to obtain a first target detection model of the water conservancy facility dam during the day.

[0041] Please refer to Figure 2 , Figure 2 A specific implementation of step S2 is shown, which is described as follows:

[0042] S21. Return the optical image to the development end, so that the development end makes a Yolo format label for the optical image to obtain a training optical image, wherein the training optical image has a Yolo format label.

[0043] S22. Construct a YOLOv5 model through a Pytorch framework.

[0044] S23. Perform dam detection model training on the YOLOv5 model through the training optical image to obtain a training result.

[0045] S24. Compare the label result in the training result with the Yolo format label to obtain a comparison result.

[0046] S25. Based on the comparison result, iteratively optimize the weight, retrain the YOLOv5 model dam detection model, and obtain the first target detection model of the water conservancy facility dam in the daytime.

[0047] Specifically, in the embodiment of the present application, the optical image is returned to the development end, and the development end makes Yolo format labels through labelimg software. When the label making is completed, the optical image to be trained is obtained and returned to the background server. Then, the YOLOv5 model is built through the Pytorch framework, and the YOLOv5 model dam detection model is trained through the optical image to be trained. The training set and the test set are trained in a ratio of 8:2 during the training process. Then, the label result in the training result is compared with the Yolo format label to obtain a comparison result. Based on the comparison result, the weight is iteratively optimized, the YOLOv5 model dam detection model is retrained, the training number epochs is set to 300, and finally the first target detection model of the water conservancy facility dam in the daytime is obtained after the training is completed.

[0048] S3. Based on the fusion image, the dam detection model is trained to obtain the second target detection model of the water conservancy facility dam at night.

[0049] Specifically, in the embodiment of the present application, the dam detection model is trained through the fusion image to obtain the second target detection model of the water conservancy facility dam at night. The specific training process adopts the same process as steps S21-S25 described above. To avoid repetition, details are not described here.

[0050] Specifically, after the first target detection model of the water conservancy facility dam in the daytime and the second target detection model of the water conservancy facility dam at night are trained, all-weather patrol and early warning of the water conservancy facility dam are realized.

[0051] S4. Based on the unmanned aerial vehicle platform, obtain the image data set to be tested of the water conservancy facility dam, and input the image data set to be tested into the first target detection model and the second target detection model to output the target screenshot.

[0052] Please refer to Figure 3 , Figure 3 A specific implementation of step S4 is shown as follows:

[0053] S41. Based on the unmanned aerial vehicle platform, obtain the image data set to be tested of the water conservancy facility dam.

[0054] S42. Input the optical image in the image data set to be tested into the first target detection model, and output the target screenshot through the first target detection model.

[0055] S43. Input the fusion image in the image data set to be tested into the second target detection model, and output the target screenshot through the second target detection model.

[0056] Specifically, after the training is completed, in the actual patrol and early warning process of the water conservancy facilities, the to-be-tested image dataset of the water conservancy facility dam is obtained through the unmanned aerial vehicle platform. The to-be-tested image dataset includes the optical image corresponding to the water conservancy facility dam during the day and the fusion image corresponding to the water conservancy facility dam at night. Then the optical image in the to-be-tested image dataset is input into the first target detection model, and the target screenshot is output through the first target detection model; the fusion image in the to-be-tested image dataset is input into the second target detection model, and the target screenshot is output through the second target detection model.

[0057] S5. The target screenshot is cut into a nine-square grid, and based on the duty cycle of the dam region in the nine-square grid, the dam of the water conservancy facility is analyzed to obtain an analysis result.

[0058] Please refer to Figure 4 , Figure 4 A specific implementation of step S5 is shown as follows:

[0059] S51. The target screenshot is analyzed for a gray value to obtain a gray value of the target screenshot.

[0060] S52. The target screenshot is cut into a nine-square grid, and based on the gray value, the duty cycle of the dam region in the nine-square grid is calculated.

[0061] S53. Based on the duty cycle of the dam region, the target dam region is screened out, and the dam analysis of the target dam region is performed again to obtain an analysis result.

[0062] Specifically, in the embodiment of the present application, the target region screenshot recognized through the Yolov5 algorithm parameter setting of the target detection model can be saved, that is, the target screenshot is retained. Then the gray value analysis is performed on the region of the target screenshot, and the dam contour is displayed through the gray value. Then the target screenshot is cut into a nine-square grid, and each region in the nine-square grid has a corresponding gray value. Then the duty cycle of each nine-square grid region is calculated. According to the duty cycle, different water conservancy facility regions are reflected, for example, the region with a high duty cycle is basically a complete dam region, the region with a low duty cycle is basically a water flow region, and the intermediate value of the duty cycle is a reserved region. Therefore, the corresponding reserved region is screened out as the target dam region in the embodiment of the present application, and the dam analysis of the target dam region is performed again to obtain an analysis result.

[0063] Please refer to Figure 5 , Figure 5 A specific implementation of step S53 is shown as follows:

[0064] S531. A preset proportion of duty cycles is screened out, and a dam region corresponding to the preset proportion of duty cycles is obtained, to obtain a target dam region.

[0065] S532. The target dam region is divided into a nine-square grid to obtain a nine-square grid region corresponding to the target dam region, and a target gray value is obtained according to a gray value corresponding to the nine-square grid region.

[0066] S533. A target duty cycle corresponding to the nine-square grid region is calculated based on the target gray value, to obtain the target duty cycle.

[0067] S534. An analysis result is obtained based on the target duty cycle.

[0068] Specifically, in the embodiment of the present application, a preset proportion of duty cycles is screened out, and a dam region corresponding to the preset proportion of duty cycles is obtained, to obtain a target dam region. For example, a region corresponding to a duty cycle higher than 0.1 and lower than 0.7 is selected as the target dam region. Then, the target dam region is re-divided into a nine-square grid to obtain a nine-square grid region corresponding to the target dam region, and a target gray value is obtained according to a gray value corresponding to the nine-square grid region. Finally, an analysis result is obtained based on the target duty cycle. The analysis result includes a good condition, a suspected missing condition, and a warning condition.

[0069] Further, a specific embodiment of step S534 is provided, which includes: counting a number of target duty cycles within a first threshold value to obtain a number of good regions; counting a number of target duty cycles within a second threshold value to obtain a number of suspected regions; counting a number of target duty cycles within a third threshold value to obtain a number of missing regions; if the number of good regions is greater than the number of suspected regions and the sum of the number of suspected regions, the analysis result is a good condition; if the number of good regions is lower than or equal to the number of suspected regions and the sum of the number of suspected regions, the analysis result is a suspected missing condition; and if the number of missing regions is greater than the number of suspected regions, the analysis result is a warning condition.

[0070] It should be noted that the first threshold value, the second threshold value, and the third threshold value are set according to actual conditions, and are not limited herein. The size relationship of the values is first threshold value > second threshold value > third threshold value. In a specific embodiment, the first threshold value is [0.7, 1], the second threshold value is (0.1, 0.7), and the third threshold value is [0, 0.1].

[0071] In a specific embodiment, the algorithm parameter setting of Yolov5 can save the recognized target area screenshot. Then through the gray value analysis of the screenshot area, the dam contour will be displayed through the gray value, and the screenshot will be divided into nine grids, and the duty cycle of the nine areas will be calculated. First, eliminate the areas with duty cycle lower than 0.1 and duty cycle higher than 0.7, because the area with duty cycle lower than 0.1 is basically a river area, and the area with duty cycle higher than 0.7 is a large area of complete dam area. Then further cut the remaining areas into nine grids, and then analyze the gray value. If the duty cycle is higher than 0.7, it is marked as good, between 0.3 and 0.7 as suspected, and lower than 0.3 as missing. Then count the number of good, suspected and missing areas. If the number of good is higher than the sum of suspected and missing, it is defined as a good situation, if it is lower, it is defined as a suspected missing situation, and if the number of missing is higher than the number of suspected, it is defined as a warning situation.

[0072] S6. Based on the analysis result, the water conservancy facility is patrolled and warned.

[0073] Specifically, the analysis result includes good situation, suspected missing situation and warning situation. If the analysis result is good situation, no warning is needed for the water conservancy facility; if the analysis result is suspected missing situation, suspected missing warning is needed for the water conservancy facility; if the analysis result is warning situation, warning is needed for the water conservancy facility. The above embodiment can obtain the image data set to be tested in real time, and make analysis result in real time, and monitor the water conservancy facility in real time according to the analysis result.

[0074] Please refer to Figure 6 , Figure 6 A specific embodiment after step S6 is shown, which is described as follows:

[0075] S6A. Obtain the collected image data set in a preset time period as the initial image data set.

[0076] S6B. Output the screenshot image corresponding to the initial image data set by the first target detection model and the second target detection model.

[0077] S6C. Perform frame skipping processing on the screenshot image, and perform similarity judgment on the screenshot images of adjacent frames to determine the dam area at the same position.

[0078] S6D. Based on the dam area at the same position, the maximum value is calculated to obtain the number of same dam areas.

[0079] S6E. Based on the dam area data and the screenshot image, the dam of the water conservancy facility is analyzed to obtain the analysis result of the current time period, and based on the analysis result of the current time period, the water conservancy facility is patrolled and warned.

[0080] Specifically, an image dataset collected within a preset time period is acquired as the initial image dataset; for example, at a certain moment, the image dataset collected on that day is acquired as the initial image dataset. Then, screenshots corresponding to the initial image dataset are output using a first object detection model and a second object detection model. Next, frame skipping processing is performed on the screenshots, and similarity judgment is made between screenshots of adjacent frames to determine dam areas at the same location. The similarity judgment involves analyzing the overlap rate of the images across RGB channels to identify dam areas at the same location. Adjacent frame screenshots are screenshots of consecutive frames. Then, maxima calculation is performed based on the dam areas at the same location to obtain the number of identical dam areas. The number of maximum similarity points between consecutive frames represents the number of dams; dam types within a river segment are similar. Finally, based on the dam area data and screenshots, dam failure analysis is performed on the water conservancy facilities to obtain the analysis results for the current time period. Based on the analysis results for the current time period, inspection and early warning of water conservancy facilities are conducted. The embodiments of this application can summarize and analyze water conservancy facilities over a certain period of time to check for the possibility of dam failure and thus issue an early warning.

[0081] In this embodiment, an image dataset of water conservancy facility dams is acquired using an unmanned aerial vehicle (UAV) platform. This dataset includes optical images of the dams during the day and fused images of the dams at night. A YOLOv5 model is constructed, and a dam detection model is trained based on the optical images to obtain a first target detection model for the dams during the day. A second target detection model for the dams at night is trained based on the fused images. A dataset of images of the dams to be tested is acquired using the UAV platform and input into the first and second target detection models to output target screenshots. The target screenshots are divided into a nine-grid layout, and based on the dam area's occupancy rate within the grid, a dam failure analysis is performed to obtain the analysis results. Based on the analysis results, inspection and early warning systems are implemented for the water conservancy facilities. This application embodiment uses a drone platform to patrol and acquire image datasets, and constructs daytime and nighttime target detection models to achieve all-weather patrols. It can also divide target screenshots into a nine-square grid, and perform dam failure analysis on the dam area of ​​the water conservancy facility based on the occupancy ratio of the dam area in the nine-square grid. This enables continuous patrols of water conservancy facilities, which is beneficial to improving the patrol and early warning efficiency of water conservancy facilities, and can issue early warnings of dam failures.

[0082] Please refer to Figure 7 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a UAV-based water conservancy facility inspection and early warning system. This system embodiment is similar to... Figure 1The method embodiment shown corresponds to the system, which can be specifically applied to various electronic devices.

[0083] As Figure 7 shown, the unmanned aerial vehicle-based water conservancy facility patrol and early warning system of the embodiment includes an image data set acquisition unit 71, a first target detection model construction unit 72, a second target detection model construction unit 73, a target screenshot output unit 74, an analysis result generation unit 75, and a water conservancy facility patrol and early warning unit 76, wherein:

[0084] The image data set acquisition unit 71 is configured to acquire an image data set of a water conservancy facility dam based on an unmanned aerial vehicle platform, wherein the image data set includes an optical image corresponding to the water conservancy facility dam during the day and a fusion image corresponding to the water conservancy facility dam at night, and the fusion image is an image formed by fusing an infrared image and a visible light image;

[0085] The first target detection model construction unit 72 is configured to construct a YOLOv5 model and perform dam detection model training based on the optical image to obtain a first target detection model of the water conservancy facility dam during the day.

[0086] The second target detection model construction unit 73 is configured to perform dam detection model training based on the fusion image to obtain a second target detection model of the water conservancy facility dam at night.

[0087] The target screenshot output unit 74 is configured to acquire a to-be-tested image data set of the water conservancy facility dam based on the unmanned aerial vehicle platform, and input the to-be-tested image data set into the first target detection model and the second target detection model to output a target screenshot.

[0088] The analysis result generation unit 75 is configured to divide the target screenshot into a nine-square grid, and perform dam breach analysis on the water conservancy facility dam based on the duty cycle of the dam region in the nine-square grid to obtain an analysis result.

[0089] The water conservancy facility patrol and early warning unit 76 is configured to perform patrol and early warning on the water conservancy facility based on the analysis result.

[0090] Further, the analysis result generation unit 75 includes:

[0091] A gray value analysis unit is configured to perform gray value analysis on the target screenshot to obtain a gray value of the target screenshot.

[0092] A target screenshot division unit is configured to divide the target screenshot into a nine-square grid, and calculate the duty cycle of the dam region in the nine-square grid based on the gray value.

[0093] A target dam region screening unit is configured to screen out a target dam region based on the duty cycle of the dam region, and perform dam breach analysis on the target dam region again to obtain an analysis result.

[0094] Further, the target dam region screening unit comprises:

[0095] a duty cycle screening unit configured to screen out a preset proportion of duty cycles, and obtain a dam region corresponding to the preset proportion of duty cycles, to obtain the target dam region;

[0096] a target gray value acquisition unit configured to divide the target dam region into a nine-square grid, to obtain a nine-square grid region corresponding to the target dam region, and obtain a gray value corresponding to the nine-square grid region, to obtain a target gray value;

[0097] a target duty cycle generation unit configured to calculate a dam duty cycle corresponding to the nine-square grid region based on the target gray value, to obtain a target duty cycle;

[0098] a target duty cycle processing unit configured to obtain an analysis result based on the target duty cycle.

[0099] Further, the target duty cycle processing unit comprises:

[0100] a first statistical unit configured to count a number of target duty cycles within a first threshold value, to obtain a number of good regions;

[0101] a second statistical unit configured to count a number of target duty cycles within a second threshold value, to obtain a number of suspected regions;

[0102] a third statistical unit configured to count a number of target duty cycles within a third threshold value, to obtain a number of missing regions;

[0103] a first condition unit configured to, if the number of good regions is greater than the number of suspected regions and a sum of the number of suspected regions, determine that the analysis result is a good condition;

[0104] a second condition unit configured to, if the number of good regions is lower than or equal to the number of suspected regions and the sum of the number of suspected regions, determine that the analysis result is a suspected missing condition;

[0105] a third condition unit configured to, if the number of missing regions is greater than the number of suspected regions, determine that the analysis result is a pre-warning condition.

[0106] Further, the water conservancy facility inspection and pre-warning unit 76 further comprises:

[0107] an initial image data set acquisition unit configured to obtain an image data set collected within a preset time period as an initial image data set;

[0108] a screenshot image generation unit configured to output a screenshot image corresponding to the initial image data set through a first target detection model and a second target detection model;

[0109] The dam region determination unit is configured to determine the dam region at the same position by performing frame skipping processing on the screenshot images and performing similarity judgment on the screenshot images of adjacent frames.

[0110] The dam region quantity calculation unit is configured to calculate the maximum value based on the dam regions at the same position to obtain the quantity of the same dam regions.

[0111] The patrol early warning unit is configured to perform dam break analysis on the dam of the water conservancy facility based on the dam region data and the screenshot images, obtain an analysis result of a current time period, and perform patrol early warning on the water conservancy facility based on the analysis result of the current time period.

[0112] Further, the first target detection model construction unit 72 includes:

[0113] The label making unit is configured to return the optical image to the development end, so that the development end makes a Yolo format label for the optical image to obtain a training optical image, wherein the training optical image has a Yolo format label.

[0114] The YOLOv5 model construction unit is configured to construct a YOLOv5 model through a Pytorch framework.

[0115] The YOLOv5 model dam detection model is trained through the training optical image to obtain a training result.

[0116] The comparison result generation unit is configured to compare the label result in the training result with the Yolo format label to obtain a comparison result.

[0117] The iterative training unit is configured to iteratively optimize the weight based on the comparison result, re-train the YOLOv5 model dam detection model, and obtain the first target detection model of the dam of the water conservancy facility in the daytime.

[0118] Further, the target screenshot output unit 74 includes:

[0119] The to-be-measured image dataset acquisition unit is configured to acquire a to-be-measured image dataset of the dam of the water conservancy facility based on a UAV platform.

[0120] The first output screenshot unit is configured to input the optical image in the to-be-measured image dataset into the first target detection model and output a target screenshot through the first target detection model.

[0121] The second output screenshot unit is configured to input the fusion image in the to-be-measured image dataset into the second target detection model and output a target screenshot through the second target detection model.

[0122] In the embodiment, an image dataset of a water conservancy facility dam is acquired based on a UAV platform, the image dataset includes an optical image corresponding to the water conservancy facility dam in the daytime and a fusion image corresponding to the water conservancy facility dam at night, a YOLOv5 model is constructed, and dam detection model training is performed based on the optical image to obtain a first target detection model of the water conservancy facility dam in the daytime; dam detection model training is performed based on the fusion image to obtain a second target detection model of the water conservancy facility dam at night; a to-be-tested image dataset of the water conservancy facility dam is acquired based on the UAV platform, and the to-be-tested image dataset is input into the first target detection model and the second target detection model to output a target screenshot; the target screenshot is divided into a nine-square grid, and dam breach analysis is performed on the water conservancy facility dam based on the duty cycle of the dam region in the nine-square grid to obtain an analysis result; and based on the analysis result, the water conservancy facility is patrolled and early warning is performed. In the embodiment, the UAV platform is used to patrol and acquire an image dataset, and a target detection model in the daytime and at night is constructed to realize all-weather patrol, the target screenshot can be divided into a nine-square grid, dam breach analysis is performed on the water conservancy facility dam based on the duty cycle of the dam region in the nine-square grid, the water conservancy facility can be continuously patrolled, the patrol and early warning efficiency of the water conservancy facility is improved, and early warning can be performed on dam breach.

[0123] To solve the above technical problems, the embodiment of the present application also provides a computer device. For details, please refer to Figure 8 , Figure 8 The basic structure block diagram of the computer device in the embodiment is shown in the figure.

[0124] The computer device 8 includes a memory 81, a processor 82, and a network interface 83 which are connected to each other through a system bus. It should be noted that only the computer device 8 with three components, the memory 81, the processor 82, and the network interface 83 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0125] The computer device can be a desktop computer, a notebook computer, a palm computer, and a cloud server, etc. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad, or a voice control device, etc.

[0126] The memory 81 includes at least one type of readable storage medium, including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the computer device 8, such as a hard disk or a memory of the computer device 8. In other embodiments, the memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 8. Of course, the memory 81 can also include both the internal storage unit and the external storage device of the computer device 8. In the present embodiment, the memory 81 is generally used to store an operating system and various application software installed on the computer device 8, such as program codes of the above-mentioned unmanned aerial vehicle-based water conservancy facility patrol and early warning method, etc. In addition, the memory 81 can also be used to temporarily store various data that have been output or will be output.

[0127] The processor 82 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 82 is generally used to control the overall operation of the computer device 8. In the present embodiment, the processor 82 is used to run the program codes or process data stored in the memory 81, such as running the program codes of the above-mentioned unmanned aerial vehicle-based water conservancy facility patrol and early warning method, to implement various embodiments of the unmanned aerial vehicle-based water conservancy facility patrol and early warning method.

[0128] The network interface 83 can include a wireless network interface or a wired network interface, and the network interface 83 is generally used to establish a communication connection between the computer device 8 and other electronic devices.

[0129] The present application also provides another implementation, i.e., to provide a computer readable storage medium, which stores a computer program, and the computer program can be executed by at least one processor to make the at least one processor execute the steps of the above-mentioned unmanned aerial vehicle-based water conservancy facility patrol and early warning method.

[0130] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method of each embodiment of the present application.

[0131] Obviously, the above-described embodiments are only some of the embodiments of the present application, not all the embodiments, and the preferred embodiments of the present application are given in the drawings, but do not limit the patent scope of the present application. The present application can be implemented in many different forms, and conversely, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing specific embodiments, or make equivalent replacements to some technical features. Any equivalent structure made by using the content of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.

Claims

1. A method for water conservancy facility patrol and early warning based on a UAV, characterized in that, The method comprises the following steps: obtaining an image data set of a water conservancy facility dam based on a UAV platform, wherein the image data set comprises an optical image corresponding to the water conservancy facility dam during the day and a fusion image corresponding to the water conservancy facility dam at night, the fusion image being an image fused from an infrared image and a visible light image; constructing a YOLOv5 model and training a dam detection model based on the optical image to obtain a first target detection model of the water conservancy facility dam during the day; training a dam detection model based on the fusion image to obtain a second target detection model of the water conservancy facility dam at night; obtaining a to-be-tested image data set of the water conservancy facility dam based on the UAV platform and inputting the to-be-tested image data set into the first target detection model and the second target detection model to output a target screenshot; cutting the target screenshot into a nine-square grid and performing dam breach analysis on the water conservancy facility dam based on the area ratio of the dam region in the nine-square grid to obtain an analysis result; performing patrol and early warning on the water conservancy facility based on the analysis result; the step of cutting the target screenshot into a nine-square grid and performing dam breach analysis on the water conservancy facility dam based on the area ratio of the dam region in the nine-square grid to obtain an analysis result comprises: performing gray value analysis on the target screenshot to obtain the gray value of the target screenshot; cutting the target screenshot into a nine-square grid and calculating the area ratio of the dam region in the nine-square grid based on the gray value; based on the area ratio of the dam region, screening out a target dam region and re-performing dam breach analysis on the target dam region to obtain an analysis result. 2.The unmanned aerial vehicle based water conservancy facility patrol and early warning method according to claim 1, characterized in that, the step of screening out a target dam region based on the area ratio of the dam region and re-performing dam breach analysis on the target dam region to obtain an analysis result comprises: screening out a preset proportion of the area ratio and obtaining a dam region corresponding to the preset proportion of the area ratio to obtain the target dam region; cutting the target dam region into a nine-square grid to obtain a nine-square grid region corresponding to the target dam region and obtaining a target gray value corresponding to the nine-square grid region; based on the target gray value, calculating the dam area ratio corresponding to the nine-square grid region to obtain a target area ratio; based on the target area ratio, obtaining the analysis result. 3.The unmanned aerial vehicle based water conservancy facility patrol and early warning method according to claim 2, characterized in that, the step of obtaining the analysis result based on the target area ratio comprises: counting the number of target area ratios within a first threshold to obtain a good region number; counting the number of target area ratios within a second threshold to obtain a suspected region number; counting the number of target area ratios within a third threshold to obtain a missing region number; if the good region number is greater than the sum of the suspected region number and the suspected region number, the analysis result is a good condition; if the good region number is less than or equal to the sum of the suspected region number and the suspected region number, the analysis result is a suspected missing condition; if the missing region number is greater than the suspected region number, the analysis result is a warning condition. 4.The unmanned aerial vehicle based water conservancy facility patrol and early warning method according to claim 1, characterized in that, after performing patrol and early warning on the water conservancy facility based on the analysis result, the method further comprises: Acquire the collected image dataset in a preset time period as an initial image dataset; Output the screenshot image corresponding to the initial image dataset through the first target detection model and the second target detection model; Determine the dam area at the same position by performing frame skipping processing on the screenshot image and similarity judgment on screenshot images of adjacent frames; Perform maximum value calculation based on the dam area at the same position to obtain the number of same dam areas; Perform dam breach analysis on the water conservancy facility dam based on the number of same dam areas and the screenshot image, obtain the analysis result of the current time period, and perform patrol warning on the water conservancy facility based on the analysis result of the current time period. 5.The unmanned aerial vehicle based water conservancy facility patrol and early warning method according to claim 1, characterized in that, The YOLOv5 model is constructed, and the dam detection model training is performed based on the optical image to obtain the first target detection model of the water conservancy facility dam in the daytime, which includes: Return the optical image to the development end to enable the development end to make Yolo format labels for the optical image to obtain a training optical image, wherein the training optical image has Yolo format labels; The YOLOv5 model is constructed through the Pytorch framework; The YOLOv5 model dam detection model is trained through the training optical image to obtain a training result; Compare the label result in the training result with the Yolo format labels to obtain a comparison result; Iteratively optimize the weight based on the comparison result, retrain the YOLOv5 model dam detection model, and obtain the first target detection model of the water conservancy facility dam in the daytime. 6.The unmanned aerial vehicle based water conservancy facility patrol and early warning method according to any one of claims 1 to 5, characterized in that, The first target detection model and the second target detection model are based on the unmanned aerial vehicle platform to acquire the image dataset of the water conservancy facility dam to be tested, and input the image dataset to be tested into the first target detection model and the second target detection model to output a target screenshot, which includes: Acquire the image dataset of the water conservancy facility dam to be tested based on the unmanned aerial vehicle platform; Input the optical image in the image dataset to be tested into the first target detection model to output the target screenshot through the first target detection model; Input the fusion image in the image dataset to be tested into the second target detection model to output the target screenshot through the second target detection model.

7. An unmanned aerial vehicle (UAV) based water conservancy facility patrol and early warning system, characterized in that, It includes: An image dataset acquisition unit is configured to acquire an image dataset of a water conservancy facility dam based on an unmanned aerial vehicle platform, wherein the image dataset includes an optical image corresponding to the water conservancy facility dam in the daytime and a fusion image corresponding to the water conservancy facility dam at night, and the fusion image is an image formed by fusing an infrared image and a visible light image; A first target detection model construction unit is configured to construct a YOLOv5 model and perform dam detection model training based on the optical image to obtain a first target detection model of the water conservancy facility dam in the daytime; A second target detection model construction unit is configured to perform dam detection model training based on the fusion image to obtain a second target detection model of the water conservancy facility dam at night; The target screenshot output unit is configured to acquire a to-be-tested image data set of the water conservancy facility dam based on the unmanned aerial vehicle platform, and input the to-be-tested image data set into the first target detection model and the second target detection model to output a target screenshot. The analysis result generation unit is configured to split the target screenshot into a nine-square grid, and perform dam-break analysis on the water conservancy facility dam based on a duty cycle of a dam region in the nine-square grid to obtain an analysis result. The water conservancy facility patrol and early warning unit is configured to perform patrol and early warning on the water conservancy facility based on the analysis result. The analysis result generation unit includes: The gray value analysis unit is configured to perform gray value analysis on the target screenshot to obtain a gray value of the target screenshot. The target screenshot splitting unit is configured to split the target screenshot into a nine-square grid, and calculate a duty cycle of a dam region in the nine-square grid based on the gray value. The target dam region screening unit is configured to screen a target dam region based on the duty cycle of the dam region, and perform dam-break analysis on the target dam region again to obtain an analysis result.

8. A computer device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the unmanned aerial vehicle-based water conservancy facility patrol and early warning method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the unmanned aerial vehicle-based water conservancy facility patrol and early warning method according to any one of claims 1 to 6.

Images