Intelligent detection method and system for dam danger based on unmanned aerial vehicle platform

Abstract

The application discloses a dam danger intelligent detection method and system based on a UAV platform, and the method comprises the following steps: an edge server group receives a data stream acquired by a UAV, and identifies and detects dam data in the data stream to obtain a first dam image; then, interference components in the first dam image are filtered and processed through a dam mask extraction algorithm, and a second dam image is obtained through binarization; finally, preset parameter objects in the second dam image are calculated and analyzed through a self-defined dam danger analysis algorithm, a defect area in the second dam image is screened out, the defect area is labeled, and finally the labeled defect dam picture is returned to a background on-duty staff for checking.The application belongs to the technical field of intelligent detection, and the dam danger is intelligently and real-timely detected through the cooperative mode of the UAV and the edge server, so that the efficiency is improved, the cost is reduced, the real-time detection target of multiple sections and multiple UAVs is supported through the load-balanced edge server, and the stability and reliability of the whole system are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, and in particular to an intelligent detection method and system for dam hazards based on an unmanned aerial vehicle (UAV) platform. Background Technology

[0002] As a public welfare project for flood control and drought relief, dikes are built in large numbers in many coastal and riverside areas, and they need to be inspected and repaired every year.

[0003] There are two traditional methods for dam inspection. One method involves setting up numerous cameras on the dam's foundation, with on-site personnel observing the dam's condition in real time to determine if any danger exists. However, this method not only consumes a lot of resources but also lacks flexibility and efficiency in deployment. The other method uses drones as data acquisition devices to obtain dam data, which is then interpreted by staff to determine if any danger exists. Therefore, this method still relies on human judgment and requires a significant investment of manpower.

[0004] Currently, there is no effective solution to the problems of high cost and low efficiency in the detection of dam hazards in related technologies. Summary of the Invention

[0005] This invention provides an intelligent detection method, device, equipment, and medium for dam hazard detection based on an unmanned aerial vehicle (UAV) platform, aiming to solve the problems of high cost and low efficiency in current dam hazard detection methods.

[0006] In a first aspect, embodiments of the present invention provide an intelligent detection method for dam hazards based on an unmanned aerial vehicle (UAV) platform, applied to an intelligent dam hazard detection system, the system comprising an UAV and an edge server cluster, the method comprising:

[0007] The edge server cluster receives the data stream acquired by the drone and identifies and detects the dam data in the data stream to obtain a first dam image;

[0008] The interference components in the first dam image are filtered out by the dam mask extraction algorithm, and the second dam image is obtained by binarization.

[0009] By using a custom dam hazard analysis algorithm, preset parameter objects in the second dam image are calculated and analyzed to filter out the defective areas in the second dam image.

[0010] Secondly, embodiments of the present invention provide an intelligent dam hazard detection system based on an unmanned aerial vehicle (UAV) platform. The system includes UAVs and an edge server cluster.

[0011] The edge server cluster receives the data stream acquired by the drone and identifies and detects the dam data in the data stream to obtain a first dam image;

[0012] The edge server cluster uses a dam mask extraction algorithm to filter out interference components in the first dam image and binarizes it to obtain the second dam image.

[0013] The edge server cluster calculates and analyzes preset parameter objects in the second dam image using a custom dam hazard analysis algorithm, filters out the defective areas in the second dam image, and marks the defective areas.

[0014] Thirdly, embodiments of the present invention provide a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the method described in the first aspect.

[0016] This invention provides an intelligent method and system for detecting dam hazards based on an unmanned aerial vehicle (UAV) platform. An edge server cluster receives data streams acquired by the UAV and identifies and detects dam data within the data streams to obtain a first dam image. Next, an interference component in the first dam image is filtered using a dam mask extraction algorithm. The average grayscale value of the image is calculated as a binarization threshold, and then binarized to obtain a second dam image. Finally, a custom dam hazard analysis algorithm is used to calculate and analyze preset parameter objects in the second dam image, filtering out defective areas and marking them.

[0017] Compared to traditional methods that rely on manual assessment of dam safety, this invention employs a collaborative approach using drones and edge servers for intelligent, real-time dam safety monitoring. This not only reduces labor costs and improves efficiency but also alleviates the information reception burden on the backend management platform. Furthermore, the edge servers utilize a load-balanced cluster deployment, enabling real-time detection of targets across multiple locations and by multiple drones, effectively enhancing system stability and reliability. This solves the problems of high cost and low efficiency in dam safety monitoring. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the intelligent detection method for dam hazards based on an unmanned aerial vehicle (UAV) platform provided in this embodiment of the invention.

[0020] Figure 2 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] This invention provides an intelligent dam hazard detection system based on a drone platform. The system includes drones and an edge server cluster. The drones and the edge server cluster establish a network connection to achieve data transmission. Specifically, the drones are used to cruise and capture dam videos to obtain dam data streams. The edge server cluster is used to receive the data streams acquired by the drones, identify, detect, and filter the dam data in the data streams, and perform hazard analysis on the detected and filtered dam data to obtain marked defect areas.

[0026] In one embodiment, the system further includes a backend management platform for receiving and managing the defective areas in the second dam image obtained from the edge server, for staff to view and process.

[0027] The intelligent dam hazard detection system based on the UAV platform provided in this embodiment of the invention applies the following intelligent dam hazard detection method based on the UAV platform: an edge server cluster receives the data stream acquired by the UAV and identifies and detects the dam data in the data stream to obtain a first dam image; then, the interference components in the first dam image are filtered out using a dam mask extraction algorithm, the average value of the image grayscale is calculated as the binarization threshold, and then binarized to obtain a second dam image; finally, a custom dam hazard analysis algorithm is used to calculate and analyze the preset parameter objects in the second dam image, filter out the defective areas in the second dam image, and mark the defective areas.

[0028] The above system enables intelligent real-time detection of dam hazards, which not only improves efficiency and reduces costs, but also supports real-time detection of targets in multiple locations and by multiple drones through load-balanced edge servers, thereby improving the stability and reliability of the entire system.

[0029] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0030] This embodiment provides an intelligent method for detecting dam hazards based on an unmanned aerial vehicle (UAV) platform, applied to the aforementioned intelligent dam hazard detection system. Please refer to [link / reference needed]. Figure 1 , Figure 1 This is a flowchart illustrating the intelligent detection method for dam hazards based on an unmanned aerial vehicle (UAV) platform provided in an embodiment of the present invention. Figure 1 As shown, the method includes steps S110 to S130.

[0031] S110, the edge server cluster receives the data stream acquired by the drone, and identifies and detects the dam data in the data stream to obtain the first dam image.

[0032] When the drone is on a patrol and filming mission, the edge server receives the streaming video data acquired by the drone and starts the trained YOLOv5 model to identify and detect the dam data in the data stream, obtaining the first dam image. It should be noted that during the video data stream transmission, the system's FPS (Frames Per Second) is consistently above 30, meeting the system's real-time detection requirements.

[0033] In one embodiment, before the edge server cluster receives the data stream acquired by the drone, a dam data stream, such as dam video, is acquired by the drone. Then, the dam data stream is preprocessed. In this embodiment, the acquired dam data stream is subjected to equal-interval frame extraction, and images that capture the entire dam are selected from the extracted images and labeled to obtain the original dam dataset. Next, a YOLOv5 model is trained using the processed original dam dataset, and the trained YOLOv5 model is deployed to the edge server cluster.

[0034] S120. The interference components in the first dam image are filtered out using the dam mask extraction algorithm, and the second dam image is obtained by binarization.

[0035] Although step S110 can identify and acquire the dam image, the acquired image may contain many interfering components, such as river water, vegetation, and roads on the bank. Therefore, in order to analyze the dam's danger more accurately in the future, it is necessary to remove these interfering components to obtain a more accurate second dam image.

[0036] In one embodiment, morphological processing is performed using the OpenCV library. First, gradient search is used to eliminate interfering components such as river water and roads. Specifically, gradient calculations are performed on each component in the first dam image to distinguish and locate river water and roads with relatively smooth gradient changes, as well as the dam with larger gradient changes. Regions with smaller gradient changes are assigned a value of 0, and regions with larger changes are assigned a value of 1. Next, the image after gradient calculation undergoes continuous dilation and erosion processing to fill gaps between some detailed textures, reducing the computational load of the subsequent contour search. Then, contour search is performed to find contour boundaries with a length greater than a set adjustable threshold. The areas enclosed by these boundaries are assigned a value of 0, while the remaining areas are assigned a value of 1, thereby eliminating interfering components with insignificant gradient changes in the image and obtaining the first mask.

[0037] To address the potential issue of vegetation occlusion in the image, this embodiment extracts the pixels with the largest G component from the embankment image and calculates the Euclidean distance K between the G component and the R and B components of these pixels, respectively. GR K GB And calculate K GR K GBIf the sum of K is greater than the preset threshold, then the pixel is considered to be vegetation, and these pixels are assigned a value of 0 to remove them, thus obtaining the second mask that excludes vegetation.

[0038] Finally, the first mask is multiplied by the second mask, and then multiplied by the original first dam image. The average value of the grayscale image is calculated as the binarization threshold, and then binarized to obtain the second dam image that retains the dam components.

[0039] The above process can remove interference that affects the analysis of dam risks, making the subsequent risk analysis algorithm more accurate.

[0040] S130. The preset parameter objects in the second dam image are calculated and analyzed by a custom dam hazard analysis algorithm, the defective areas in the second dam image are selected and marked.

[0041] The pre-defined parameter objects in the second dam image are calculated and analyzed using a custom dam hazard analysis algorithm in the edge server. The specific steps include:

[0042] First, adaptive segmentation is performed based on the image size. For example, the length and width of the image are modulo 100 to obtain the corresponding number of blocks. If the length and width are less than 100, no segmentation is performed, and the number of blocks is 1. Next, the proportion of white points in the segmented image regions is calculated. If the proportion reaches a preset threshold, the region is retained. Since white points in the binarized image correspond to flat areas with strong reflections, while black points correspond to damaged, concave, or shadowed areas, defective areas will appear as a messy area with an equal number of black and white points in the image. By calculating the proportion, the regions in the image that meet the criteria can be initially located.

[0043] Then, the areas that are retained are subjected to expansion and erosion processing, and the change rate of the white duty cycle after processing is calculated. If it is a defective area, after the expansion operation, the defective area will be filled smoothly and completely, so the white ratio will be greatly improved. However, the ratio of intact areas that are similar in black and white ratio but have a high degree of differentiation will not change much. Therefore, if the change rate of the white duty cycle is greater than the preset threshold, the defective area can be screened out, marked, and fed back to the backend management platform.

[0044] In one embodiment, since there may be some areas where the dam meets the river or some spare piles of rubble left on the dam, if the image segmentation includes these areas, these areas will also be retained after the above two steps of screening. Therefore, the image after the above screening can be further screened to reduce the false positive rate. A large window dilation and erosion is performed on the second dam image after the above screening to distinguish areas with large damage from interference areas. After the distinction, a contour search is performed, the missing areas are assigned a value of 0, and the missing area is calculated. If the area is greater than a preset threshold, the area can be determined as a missing area.

[0045] In one embodiment, after identifying the marked areas of dam damage, the edge server cluster returns the marked images to the backend management platform for viewing and processing by on-duty personnel. The load-balanced edge server cluster not only effectively improves efficiency but also enables real-time acquisition and analysis of target detection objects from multiple drones, enhancing system stability while reducing the information receiving pressure on the backend platform.

[0046] Through steps S110 to S130 above, the intelligent dam hazard detection method based on an unmanned aerial vehicle (UAV) platform provided in this embodiment of the invention employs a collaborative approach between UAVs and edge servers to perform intelligent real-time dam hazard detection. This not only reduces labor costs but also improves efficiency and alleviates the information receiving pressure on the backend management platform. Simultaneously, the edge servers utilize a load-balanced cluster deployment, enabling real-time detection of targets across multiple locations and by multiple UAVs, effectively improving system stability and reliability. This solves the problems of high cost and low efficiency in dam hazard detection.

[0047] The aforementioned intelligent detection method for dam hazards based on a drone platform can be implemented as a computer program, which can be used in various applications such as... Figure 2 It runs on the computer device shown.

[0048] Please see Figure 2 , Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device can be used to execute an intelligent method for detecting dam hazards based on a drone platform, sending real-scene dam images captured by the drone to an edge server cluster for data stream identification, detection, and filtering, and performing hazard analysis on the detected and filtered dam data to obtain marked defect areas.

[0049] See Figure 2 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a storage medium 503 and internal memory 504.

[0050] The storage medium 503 can store the operating system 5031 and the computer program 5032. When the computer program 5032 is executed, it enables the processor 502 to execute an intelligent detection method for dam hazards based on an unmanned aerial vehicle platform. The storage medium 503 can be a volatile storage medium or a non-volatile storage medium.

[0051] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0052] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an intelligent detection method for dam hazards based on the UAV platform.

[0053] This network interface 505 is used for network communication, such as providing data transmission. Those skilled in the art will understand that... Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device 500 to which the present invention is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0054] The processor 502 is used to run the computer program 5032 stored in the memory to implement the corresponding functions in the above-mentioned intelligent detection method for dam hazards based on the UAV platform.

[0055] Those skilled in the art will understand that Figure 2 The embodiments of the computer device shown do not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only memory and a processor. In such embodiments, the structure and function of the memory and processor are different from those shown. Figure 2 The embodiments shown are consistent and will not be described again here.

[0056] It should be understood that, in this embodiment of the invention, the processor 502 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0057] In another embodiment of the invention, a computer-readable storage medium is provided. This computer-readable storage medium may be volatile or non-volatile. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps included in the above-described intelligent dam hazard detection method based on an unmanned aerial vehicle (UAV) platform.

[0058] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0059] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0060] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0061] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0062] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent detection of dam hazards based on an unmanned aerial vehicle (UAV) platform, characterized in that, The method is applied to an intelligent monitoring system for dam hazards, the system comprising drones and an edge server cluster, and the method includes: The edge server cluster receives the data stream acquired by the drone and identifies and detects the dam data in the data stream to obtain a first dam image; The interference components in the first dam image are filtered out by the dam mask extraction algorithm, and the second dam image is obtained by binarization. A custom dam hazard analysis algorithm is used to calculate and analyze preset parameter objects in the second dam image, filter out the defective areas in the second dam image, and mark the defective areas. The step of filtering out interference components in the first dam image using a dam mask extraction algorithm includes: Gradient calculation is performed on the first dam image, continuous dilation and erosion processing is applied to the calculated image, and contour search is performed. Interference components with insignificant gradient changes in the image are removed according to a set adjustable threshold to obtain the first mask. Extract the pixel with the largest G component from the dam image, and calculate the sum K of the Euclidean distances between the G component and the R and B components of the pixel. If K is greater than a preset threshold, remove the pixel to obtain the second mask. After multiplying the first mask and the second mask, it is then multiplied with the first dam image. The average value of the image grayscale is calculated as the binarization threshold, and the binarization yields the second dam image that retains the dam component. The step of calculating and analyzing preset parameter objects in the second dam image using a custom dam hazard analysis algorithm includes: Adaptive cropping is performed based on the image size, and the proportion of white points in the cropped image area is calculated. If the proportion reaches a preset threshold, the area is retained. The retained area is subjected to expansion and erosion processing, and the white duty cycle change rate after processing is calculated. If the white duty cycle change rate is greater than the preset threshold, the defective area is selected.

2. The method according to claim 1, characterized in that, Before the edge server cluster receives the data stream acquired by the drone, the method includes: The dam data stream is acquired by the drone, and the dam data stream is preprocessed to obtain the original dam dataset. A YOLOv5 model is trained using the original dam dataset, and the trained YOLOv5 model is deployed to the edge server cluster.

3. The method according to claim 2, characterized in that, The preprocessing of the dam data stream includes: The captured dam data stream is subjected to equal-interval frame extraction, and the extracted images are labeled to obtain the original dam dataset.

4. The method according to claim 1, characterized in that, After filtering out the missing areas in the second dam image, the process also includes: The second dam image after screening is subjected to a large-window dilation and erosion process, and a contour search is performed on the image after dilation and erosion to remove interfering components and further identify and screen out the defective areas.

5. An intelligent monitoring system for dam hazards based on an unmanned aerial vehicle (UAV) platform, characterized in that, The system includes drones and an edge server cluster. The edge server cluster receives the data stream acquired by the drone and identifies and detects the dam data in the data stream to obtain a first dam image; The edge server cluster uses a dam mask extraction algorithm to filter out interference components in the first dam image and binarizes it to obtain the second dam image. The edge server cluster calculates and analyzes preset parameter objects in the second dam image using a custom dam hazard analysis algorithm, filters out the defective areas in the second dam image, and marks the defective areas. When the edge server cluster performs the step of filtering interference components in the first dam image using the dam mask extraction algorithm and binarizing it to obtain the second dam image, it is specifically used for: Gradient calculation is performed on the first dam image, continuous dilation and erosion processing is applied to the calculated image, and contour search is performed. Interference components with insignificant gradient changes in the image are removed according to a set adjustable threshold to obtain the first mask. Extract the pixel with the largest G component from the dam image, and calculate the sum K of the Euclidean distances between the G component and the R and B components of the pixel. If K is greater than a preset threshold, remove the pixel to obtain the second mask. After multiplying the first mask and the second mask, it is then multiplied with the first dam image. The average value of the image grayscale is calculated as the binarization threshold, and the binarization yields the second dam image that retains the dam component. When the edge server cluster executes the step of calculating and analyzing the preset parameter objects in the second dam image using a custom dam hazard analysis algorithm, it specifically performs the following: Adaptive cropping is performed based on the image size, and the proportion of white points in the cropped image area is calculated. If the proportion reaches a preset threshold, the area is retained. The retained area is subjected to expansion and erosion processing, and the white duty cycle change rate after processing is calculated. If the white duty cycle change rate is greater than the preset threshold, the defective area is selected.

6. The system according to claim 5, characterized in that, The system also includes a backend management platform. The backend management platform is used to receive and manage the dam defect areas selected from the edge server.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images