Water conservancy project dam detection method and equipment based on unmanned aerial vehicle and medium

Abstract

The invention discloses a water conservancy project dam detection method and device based on an unmanned aerial vehicle and a medium, belongs to the technical field of water conservancy project dams, and aims to solve the problems that an existing water conservancy project dam often depends on manual inspection, the detection efficiency is low, the labor cost is high, missing detection and misjudgment are likely to occur, and the detection efficiency is high. And intelligent monitoring and early warning of dam risks are not facilitated. The method comprises the steps of collecting a real-time detection image of a dam based on a preset unmanned aerial vehicle flight plan; performing risk identification processing related to dam main body characteristics on the real-time detection image to obtain dam main body characteristic information; through preset dam laser radar data, carrying out calculation processing related to dam appearance characteristics on the real-time detection image to obtain dam appearance characteristic information; and according to the reflectivity data of the preset multispectral image, carrying out index judgment processing on dam environment characteristics on the real-time detection image to obtain dam environment characteristic information.

Description

Technical Field

[0001] The present application relates to the field of water conservancy project dams, and in particular to a water conservancy project dam detection method, equipment and medium based on unmanned aerial vehicle (UAV) Background Art

[0003] Water conservancy dam inspection is an important means of preventing natural disasters. Once the dam is damaged or leaking, it may lead to flooding. Through regular inspection and maintenance, potential safety hazards can be discovered in time, and effective measures can be taken to repair them, thereby ensuring the flood control capacity of the dam.

[0004] Water conservancy dams are an important part of water conservancy projects, and their operating status directly affects the performance of the entire water conservancy system. Through the inspection of dams, we can understand their structure, materials, strength and other aspects, and provide a basis for the operation and maintenance of water conservancy projects. At the same time, the inspection results can also provide a reference for the transformation and expansion of water conservancy projects.

[0005] The inspection of water conservancy dams can not only find and repair existing problems in time, but also extend their service life through regular maintenance. Through the inspection of dams, we can understand their aging and wear, take appropriate measures to repair or replace them, and ensure the long-term stable operation of the dams.

[0006] In large-scale water conservancy projects, dams are prone to cracks, deformation, settlement, leakage and other adverse factors. Early detection, early intervention and early prevention are needed. However, the detection efficiency of existing water conservancy dams is slow and often relies on manual inspections, which is time-consuming and labor-intensive. It is also easy to cause inaccurate detection, that is, missed inspections and misjudgments that may occur during manual inspections, which is not conducive to the rapid intelligent monitoring and early warning of dams, and it is difficult to promote the informatization and intelligence of water conservancy project management, which in turn affects the environment and ecology around the dams. Summary of the invention

[0007] The embodiments of the present application provide a method, equipment and medium for detecting water conservancy project dams based on drones, which are used to solve the following technical problems: existing water conservancy project dams often rely on manual inspections, which have low detection efficiency and high labor costs, are prone to missed detections and misjudgments, and are not conducive to intelligent monitoring and early warning of dam risks.

[0008] The present application embodiment adopts the following technical solutions:

[0009] On the one hand, an embodiment of the present application provides a method for detecting dams of water conservancy projects based on drones, including: based on a preset drone flight plan, collecting real-time detection images of the dam; performing risk identification processing on the real-time detection images regarding the main features of the dam to obtain main feature information of the dam; performing calculation processing on the real-time detection images regarding the appearance features of the dam using preset dam lidar data to obtain appearance feature information of the dam; performing index determination processing on the real-time detection images regarding the environmental features of the dam based on the reflectivity data of a preset multispectral image to obtain environmental feature information of the dam; performing safety assessment processing on the main feature information of the dam, the appearance feature information of the dam and the environmental feature information of the dam to obtain dam assessment information; performing visualization output processing on the dam information based on the dam assessment information to complete real-time detection of the dam.

[0010] The embodiment of the present application uses drones to quickly cover a wide area of ​​a large-scale water conservancy project dam in a short period of time, greatly saving time and labor costs. At the same time, it is equipped with a variety of sensors, such as high-resolution cameras, multi-spectral cameras, and lidars, to achieve multi-dimensional detection of dams. It can provide high-precision terrain information and three-dimensional point cloud data, and can accurately detect problems such as slight deformation, cracks, and collapse of dams. It can also achieve real-time monitoring and early warning, promote the informatization and intelligence of water conservancy project management, and improve the intelligence level of water conservancy project management. And it can use drone inspections to protect the environment and ecology.

[0011] In a feasible implementation, based on a preset UAV flight plan, real-time detection images of the dam are collected, specifically including: according to the basic parameters and environmental conditions of the dam, the coverage calculation of the shooting angle of view of the area where the dam is located is performed, and the height parameter and angle parameter of the UAV are determined; wherein, the basic parameters include: the length of the dam, the width of the dam and the height of the dam; the environmental conditions include: visibility on a sunny day, visibility on a cloudy day and water level; through the height parameter and the angle parameter, the UAV shooting route is selected for the dam area to generate the UAV flight plan; based on the UAV The UAV performs flight planning, and uses a high-resolution camera to collect and process image information of the dam area to obtain a high-definition image of the dam; uses a multi-spectral camera preset on the UAV to collect and process multi-spectral images of the dam area to obtain reflectivity data; wherein the reflectivity data is used to represent vegetation coverage information and soil moisture information; uses a lidar device preset on the UAV to perform terrain scanning on the dam area to obtain dam lidar data; and performs data integration processing on the dam high-definition image, the reflectivity data and the dam lidar data to obtain the real-time detection image.

[0012] In a feasible implementation manner, before performing risk identification processing on the real-time detection image regarding the main features of the dam to obtain the main feature information of the dam, the method further includes: performing radiation correction on the high-definition dam image in the real-time detection image, and eliminating the photometric difference data in the high-definition dam image to obtain a preprocessed high-definition dam image; performing correction processing on the preprocessed high-definition dam image regarding the flight attitude and camera parameters of the drone through a geometric correction algorithm, and converting the image coordinates in the preprocessed high-definition dam image into actual geographic coordinates to determine a high-quality high-definition dam image; performing band fusion processing on the multispectral image in the real-time detection image regarding the reflectivity through a preset principal component analysis technique to obtain a fused multispectral image; performing correction processing on the fused multispectral image regarding atmospheric factors according to an atmospheric radiation transmission model to obtain a high-quality multispectral image; and determining an optimized real-time detection image based on the high-quality high-definition dam image and the high-quality multispectral image.

[0013] In a feasible implementation manner, the real-time detection image is subjected to risk identification processing related to the main features of the dam to obtain the main feature information of the dam, specifically including: calculating the gradient change of the pixels of the main image of the dam for the high-quality high-definition image of the dam in the real-time detection image according to the Canny edge detection algorithm to determine the edge position of the main body of the dam; connecting the continuous edge pixels of the main body of the dam one by one based on the edge position to obtain the main contour information of the dam; identifying the risk feature points in the main contour information of the dam, and performing image matching processing on the risk feature points through a preset risk feature template to determine the real risk area features; wherein the risk feature points include at least: crack feature points, collapse feature points and slope deformation feature points; performing background image segmentation processing on the high-quality high-definition image of the dam according to the real risk area corresponding to the real risk area features to obtain some of the main feature information of the dam; wherein the main feature information of the dam represents the main risk factors of the dam.

[0014] In a feasible implementation manner, the real-time detection image is calculated and processed with respect to the dam shape features by using preset dam laser radar data to obtain dam shape feature information, specifically including: using a preset point cloud registration algorithm, the dam laser radar data at different time points in the real-time detection image is point cloud registered with respect to the iterative nearest point to obtain a transformation matrix that minimizes the distance between two point clouds; using the least squares method, deformation analysis calculation is performed on the transformation matrix to obtain a point cloud displacement and a point cloud shape variable that minimizes the distance between two point clouds; based on the point cloud displacement and the point cloud shape variable, the amplitude of the terrain shape variable is calculated for the terrain height data in the dam laser radar data at different time points to obtain terrain change amplitude information; based on the point cloud displacement and the point cloud shape variable and based on the digital elevation model corresponding to the dam laser radar data, the slope shape variable is calculated for the slope data in the dam laser radar data at different time points to obtain slope change information; the terrain change amplitude information and the slope change information are integrated to obtain the dam shape feature information based on the real-time detection image.

[0015] In a feasible implementation manner, the real-time detection image is subjected to index determination processing related to the dam environment characteristics according to the reflectivity data of a preset multispectral image to obtain the dam environment characteristic information, specifically including: using a preset NDVI algorithm, comparing and calculating the near-infrared band reflectivity and the red light band reflectivity in the reflectivity data to obtain a vegetation coverage index based on the real-time detection image; using a preset TUDI algorithm, calculating the temperature extreme value of the surface temperature change in the reflectivity data to obtain a soil moisture index based on the real-time detection image; extracting information of the dam environment characteristics from both the vegetation coverage index and the soil moisture index to determine the dam environment characteristic information.

[0016] In a feasible implementation manner, the main characteristic information of the dam, the shape characteristic information of the dam and the characteristic information of the environment to the dam are subjected to safety assessment processing to obtain dam assessment information, specifically including: determining the weight coefficient corresponding to each characteristic information based on the proportion of the safety factor of the main characteristic information of the dam, the shape characteristic information of the dam and the characteristic information of the environment to the dam; performing decentralized calculation of the relevant weight coefficients on the main characteristic information of the dam, the shape characteristic information of the dam and the characteristic information of the environment to the dam through a preset dam safety assessment index system to determine the risk score of the dam; performing threshold division on the risk score to obtain the dam assessment information; wherein the dam assessment information includes: safety assessment information, early warning assessment information and danger assessment information.

[0017] In a feasible implementation manner, based on the dam assessment information, information visualization output processing is performed on the dam, specifically including: based on the dam assessment information, risk labeling processing is performed on the real-time detection image to obtain two-dimensional dam hidden danger information; based on the three-dimensional dam model established by the real-time detection image, the dam assessment information is divided and labeled into model areas to generate a three-dimensional dam hidden danger model; based on the dam assessment information, the dam main body feature information, the dam shape feature information and the dam environment feature information are sorted in reports on the degree of hidden danger, and dam hidden danger report information is determined; wherein the dam hidden danger report information is used to be transmitted to the mobile communication device of the inspection personnel; based on the two-dimensional dam hidden danger information, the three-dimensional dam hidden danger model and the dam hidden danger report information, the real-time detection of the dam is completed.

[0018] In the second aspect, an embodiment of the present application also provides a drone-based water conservancy project dam detection device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute a drone-based water conservancy project dam detection method as described in any of the above embodiments.

[0019] In the third aspect, an embodiment of the present application also provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes a drone-based water conservancy project dam detection method described in any of the above embodiments.

[0020] The embodiment of the present application provides a method, device and medium for detecting dams of water conservancy projects based on drones. Compared with the prior art, the embodiment of the present application has the following beneficial technical effects:

[0021] 1. Improve detection efficiency

[0022] 1. Rapid coverage: UAVs can quickly cover a wide area of ​​a large water conservancy project dam in a short period of time, greatly saving time and labor costs compared to traditional manual inspection methods. For example, for a dam that is tens or even hundreds of kilometers long, a drone can complete a comprehensive inspection in a few hours, while a manual inspection may take several days or even longer.

[0023] 2. Multi-task parallelism: UAVs can carry multiple sensors at the same time, such as high-resolution cameras, multi-spectral cameras and lidar, to achieve multi-dimensional detection of dams. Different sensors can work at the same time to obtain different types of data, which improves the efficiency and comprehensiveness of detection.

[0024] 2. Enhance detection accuracy

[0025] 1. High-precision data collection: LiDAR and other sensors can provide high-precision terrain information and three-dimensional point cloud data, and can accurately detect small deformations, cracks and collapses of dams. Compared with traditional measurement methods such as total stations and levels, LiDAR has higher measurement accuracy, reaching centimeters or even millimeters.

[0026] 2. Objective data analysis: By objectively analyzing and processing the collected data, the interference of human factors can be reduced and the accuracy of the detection results can be improved. For example, by using image processing and analysis technology, cracks and collapsed areas of the dam can be automatically identified, avoiding missed inspections and misjudgments that may occur during manual inspections.

[0027] 3. Realize real-time monitoring and early warning

[0028] 1. Dynamic monitoring: UAVs can regularly inspect dams and dynamically monitor the safety of dams. By comparing data at different time points, changes in dams, such as changes in terrain, vegetation coverage, and soil moisture, can be discovered in a timely manner, providing a basis for timely maintenance measures.

[0029] 2. Early warning function: When a potential safety hazard is detected in a dam, the system can send out an early warning signal in time to notify relevant personnel to take measures. The early warning function can greatly improve the timeliness and effectiveness of dam safety management and avoid the occurrence of safety accidents.

[0030] 4. Promote the informatization and intelligence of water conservancy project management

[0031] 1. Data management and sharing: The technical solution of the present invention can manage the collected data in an informationized manner and establish a database to facilitate engineering personnel to query, analyze and process data. At the same time, data can be shared through the network to provide decision support for water conservancy project management departments.

[0032] 2. Intelligent analysis: Using machine learning and artificial intelligence technology, we can conduct intelligent analysis on the collected data, establish a dam safety prediction model, and predict the safety status of the dam in the future. This helps engineers take preventive measures in advance and improve the intelligent level of water conservancy project management.

[0033] V. Protecting the Environment and Ecology

[0034] 1. Reduce the impact on the environment: Compared with traditional manual inspection methods, drone inspection has less impact on the environment. Drones can conduct inspections without damaging the vegetation and ecological environment of the dam, avoiding problems such as trampling vegetation and damaging the soil that may occur during manual inspections.

[0035] 2. Promote ecological protection: By monitoring the vegetation coverage and soil moisture around the dam, data support can be provided for ecological protection. For example, a reasonable vegetation protection and restoration plan can be formulated based on the monitoring results to improve the ecological environment quality around the dam. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0037] Figure 1 A flow chart of a method for detecting a dam in a water conservancy project based on a drone provided in an embodiment of the present application;

[0038] Figure 2 A schematic structural diagram of a water conservancy project dam inspection device based on an unmanned aerial vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0040] The present application embodiment provides a method for detecting a dam in a water conservancy project based on a drone, such as Figure 1 As shown, the water conservancy project dam detection method based on drone specifically includes steps S101-S106:

[0041] S101. Based on the preset UAV flight plan, collect real-time detection images of the dam.

[0042] It should be noted that when selecting a drone, you can choose a quad-rotor with a large-capacity battery to ensure its stability and long endurance. Integrate high-definition cameras, temperature and humidity, infrared thermal imaging, lidar and other sensors in a reasonable way.

[0043] Specifically, it is necessary to first calculate the coverage of the shooting angle of the dam area based on the basic parameters and environmental conditions of the dam, and determine the height parameters and angle parameters of the drone. Among them, the basic parameters include: dam length, dam width and dam height. Environmental conditions include: visibility on sunny days, visibility on cloudy days and water level.

[0044] Furthermore, the altitude parameter and the angle parameter are used to select the route for drone photography in the dam area, and a drone flight plan is generated.

[0045] Furthermore, based on the flight planning of the UAV, the image information of the dam area is collected and processed by a high-resolution camera to obtain a high-definition image of the dam. The multispectral camera preset on the UAV is used to collect and process the multispectral image of the dam area to obtain reflectivity data. The reflectivity data is used to represent the vegetation coverage information and soil moisture information. The laser radar equipment preset on the UAV is used to perform terrain scanning on the dam area to obtain the laser radar data of the dam.

[0046] As a feasible implementation method, the flight route of the drone is planned according to the length, terrain and environmental conditions of the dam of the water conservancy project. Ensure that the flight route can fully cover the dam area and shoot at different heights and angles to obtain richer information. It is also necessary to consider flight plans under different times and weather conditions, such as multiple collections on sunny days, cloudy days, and different water levels to improve the accuracy and completeness of the data. When the drone flight plan is determined, the operator releases the drone according to the planned route and monitors the flight status and sensor data of the drone in real time through the ground control station. The high-resolution camera captures high-definition images of the dam, and the multispectral camera obtains information such as vegetation coverage and soil moisture, that is, reflectivity data; the lidar equipment measures the terrain information of the dam, and obtains the dam lidar data after scanning and processing. At the same time, during the flight, it is also necessary to ensure the stability and continuity of data collection to avoid data loss and repeated collection.

[0047] Furthermore, the high-definition images of the dam, reflectivity data and lidar data of the dam are integrated and processed to obtain real-time detection images.

[0048] S102, performing risk identification processing on the real-time detection image regarding the main features of the dam to obtain the main feature information of the dam.

[0049] Specifically, radiation correction is performed on the high-definition image of the dam in the real-time detection image, and photometric difference data in the high-definition image of the dam is eliminated to obtain a pre-processed high-definition image of the dam.

[0050] Furthermore, through the geometric correction algorithm, the pre-processed dam high-definition image is corrected with respect to the UAV flight attitude and camera parameters, and the image coordinates in the pre-processed dam high-definition image are converted into actual geographic coordinates to determine a high-quality dam high-definition image.

[0051] In one embodiment, the collected real-time detection image China's high-resolution dam HD image is first subjected to radiation correction to eliminate the image brightness difference caused by factors such as uneven illumination. For example, a method based on histogram matching is used to adjust the brightness distribution of the image to a standard range. Then a geometric correction is performed to correct the image deformation caused by factors such as the flight posture of the drone and camera parameters. The image coordinates can be converted into actual geographic coordinates using ground control points and geometric correction algorithms. Then denoising is performed to remove noise interference in the image. Methods such as median filtering and Gaussian filtering can be used to reduce random noise in the image, improve image quality, and ultimately determine a high-quality dam HD image.

[0052] Furthermore, the multispectral image in the real-time detection image is subjected to a band fusion process of relevant reflectivity through the preset principal component analysis technology to obtain a fused multispectral image. Then, according to the atmospheric radiation transmission model, the fused multispectral image is corrected for relevant atmospheric factors to obtain a high-quality multispectral image.

[0053] In one embodiment, it is also necessary to perform band fusion on the multispectral image in the real-time detection image, fuse the image information of different bands together, and enhance the expressiveness of the image. For example, band fusion is performed using methods such as principal component analysis (PCA). Then, atmospheric correction is performed on the fused multispectral image to eliminate the influence of the atmosphere on the multispectral image and improve the accuracy of information such as vegetation coverage and soil moisture. For example, an atmospheric radiation transfer model can be used to perform atmospheric correction, and finally a high-quality multispectral image is obtained.

[0054] Furthermore, based on the high-quality dam HD images and high-quality multispectral images, an optimized real-time detection image is determined.

[0055] Furthermore, according to the Canny edge detection algorithm, the gradient change of the pixels of the main body of the dam is calculated for the high-quality high-definition image of the dam in the real-time detection image to determine the edge position of the main body of the dam.

[0056] Furthermore, based on the edge position, the continuous edge pixels of the dam body are connected one by one to obtain the contour information of the dam body.

[0057] Furthermore, the risk feature points in the main contour information of the dam are identified, and the risk feature points are image matched through the preset risk feature template to determine the real risk area characteristics. Among them, the risk feature points include at least: crack feature points, collapse feature points and slope deformation feature points.

[0058] Furthermore, according to the real risk area corresponding to the real risk area characteristics, the high-quality dam high-definition image is segmented into background images to obtain some dam main feature information, wherein the dam main feature information represents the main risk factors of the dam.

[0059] In one embodiment, image processing and analysis techniques can be used to extract characteristic information such as the profile, slope, cracks, and collapse of the dam:

[0060] 1) Edge detection: The Canny edge detection algorithm is used to extract the contour edge of the dam. The edge position of the dam body is determined by calculating the gradient change of pixels in the image, and then the continuous edge pixels are connected to form the contour of the dam, that is, the contour information of the dam body.

[0061] 2) Feature point extraction: Use algorithms such as SIFT or SURF to extract representative feature points from the image. These feature points can be used for image matching and target recognition, that is, the risk feature points can be image matched using the preset risk feature template to determine the real risk area features, which is important for the identification of cracks and collapse areas.

[0062] 3) Image segmentation: Use threshold segmentation, region growing or watershed algorithm to segment the image into different regions, so as to obtain some main feature information of the dam to reflect the main risk factors of the dam. For example, cracks and collapsed areas can be separated from the background for further analysis and measurement.

[0063] S103, using preset dam laser radar data, calculating and processing the real-time detection image about the dam shape features to obtain dam shape feature information.

[0064] Specifically, through a preset point cloud registration algorithm, the dam lidar data at different time points in the real-time detection image are processed with point cloud registration related to the iterative closest point (ICP) to obtain a transformation matrix that minimizes the transformation matrix between the two point clouds.

[0065] Furthermore, the least square method is used to perform deformation analysis on the transformation matrix to obtain the point cloud displacement and point cloud shape variable that minimizes the two point clouds.

[0066] In one embodiment, the dam deformation is analyzed by using point cloud registration and other related analysis methods in the three-dimensional point cloud deformation analysis, and the iterative closest point (ICP) algorithm is used for point cloud registration, wherein the basic idea is to determine the transformation relationship between the two point clouds by minimizing the distance error between them. Assume that the two dam point clouds in the dam lidar data at different time points are P = [p1, p2, p3...p n ] and Q = [q1, q2, q3...q n ], the goal is to find a transformation matrix T so that the distance between the point cloud P and the point cloud Q after transformation is minimized.

[0067] Define the objective function: Among them, qi' is the point cloud Q and point p i The nearest point. Then the least squares method is used to solve the transformation matrix T, which usually includes the rotation matrix R and the translation matrix t. The transformation matrix can be continuously updated in an iterative manner until the convergence condition is met.

[0068] Calculate the point cloud displacement and point cloud shape. Assume that the point clouds at two time points are and For each point P, calculate its displacement vector at two time points For each point Pi, calculate its displacement vector at two time points

[0069] Then calculate the point cloud displacement: Then calculate the deformation of the point cloud: the root mean square error (RMSE) can be used:

[0070] Furthermore, according to the point cloud displacement and point cloud deformation, the terrain height data in the dam lidar data at different time points are used to calculate the amplitude of the terrain deformation to obtain terrain change amplitude information. For example, for each point, the height difference between two time points is calculated to obtain the terrain change amplitude.

[0071] Furthermore, based on the point cloud displacement and point cloud deformation, and based on the digital elevation model generated by the dam lidar data, the slope deformation variable of the slope data in the dam lidar data at different time points is calculated to obtain the slope change information. In other words, the slope of the dam can be calculated based on the digital elevation model (DEM) generated by the dam lidar data. By comparing the slope data at different time points, the slope change can be determined.

[0072] Furthermore, the terrain change amplitude information and the slope change information are integrated to obtain the dam shape feature information based on the real-time detection image.

[0073] S104, according to the reflectivity data of the preset multi-spectral image, the real-time detection image is subjected to the index determination processing related to the dam environment characteristics to obtain the dam environment characteristic information.

[0074] Specifically, through the preset NDVI algorithm, the near-infrared band reflectivity and the red light band reflectivity in the reflectivity data are compared and calculated to obtain the vegetation coverage index based on the real-time detection image.

[0075] Furthermore, the preset TUDI algorithm is used to calculate the temperature extremes of the surface temperature changes in the reflectivity data to obtain a soil moisture index based on the real-time detection image.

[0076] In one embodiment, vegetation coverage calculation: According to the reflectance data of the multispectral image, the vegetation coverage index can be calculated using the pixel dichotomy method or the NDVI method:

[0077] ① Pixel dichotomy calculation formula: vegetation coverage = number of vegetation pixels / total number of pixels. By classifying the multispectral image, the vegetation pixels and non-vegetation pixels are determined, and then the vegetation coverage is calculated.

[0078] ②NDVI calculation formula: NDVI = (NIR-Red) / (NIR + Red), where NIR is the reflectance of the near-infrared band and Red is the reflectance of the red light band. Determine the level of vegetation coverage based on the range of NDVI values.

[0079] Soil moisture calculation: Using the reflectivity of the near-infrared band and short-wave infrared band in the multispectral image, the thermal inertia method or TVDI method is used to calculate the soil moisture index:

[0080] TVDI calculation formula: TVDI = (Ts-Tsmin) / (Tsmax-Tsmin), where Ts is the surface temperature, Tsmin and Tsmax are the minimum and maximum surface temperatures under a specific vegetation coverage. By calculating the TVDI value, the soil moisture level can be determined.

[0081] Furthermore, the vegetation coverage index and the soil moisture index are subjected to information extraction and processing of the dam environment characteristics to determine the dam environment characteristic information.

[0082] S105, performing safety assessment processing on the dam main body characteristic information, the dam shape characteristic information and the dam environment characteristic information to obtain dam assessment information.

[0083] Specifically, based on the proportion of the safety factor of the dam main body characteristic information, the dam shape characteristic information and the dam environment characteristic information, the weight coefficient corresponding to each characteristic information is determined.

[0084] Furthermore, through the preset dam safety assessment index system, the weight coefficients of the dam main feature information, dam shape feature information and dam environmental feature information are decentralized calculated to determine the risk score of the dam.

[0085] Furthermore, the risk score is divided into thresholds to obtain dam assessment information, which includes safety assessment information, early warning assessment information and danger assessment information.

[0086] As a feasible implementation method, an evaluation index system can be established to comprehensively consider factors such as crack length, collapsed area, slope change, vegetation coverage, soil moisture, etc. Determine the weight for each factor. For example, the weights of crack length and collapsed area can be relatively high because they directly reflect the structural stability of the dam. Then, different safety levels are divided, for example, into three levels: safe, early warning, and dangerous. According to the evaluation index system and the decentralized calculation of the weight coefficient, the comprehensive score of each area is calculated to determine its safety level.

[0087] In one embodiment, an early warning can also be issued based on the assessed dam assessment information. When a potential safety hazard is detected in the dam, an alarm can be issued in a timely manner. Relevant personnel can be notified through text messages, emails, mobile phone applications, etc. An early warning response process can also be established to clarify the responsibilities and action plans of relevant personnel. For example, after receiving the early warning information, the engineering management personnel should immediately organize personnel to conduct on-site inspections and take corresponding maintenance measures.

[0088] S106. Based on the dam assessment information, the dam information is processed by visual output to complete the real-time detection of the dam.

[0089] Specifically, according to the dam assessment information, the real-time detection image is processed for risk annotation to obtain two-dimensional dam hidden danger information.

[0090] Furthermore, based on the three-dimensional dam model established by the real-time detection image, the dam assessment information is divided and marked in the model area to generate a three-dimensional dam hidden danger model.

[0091] Furthermore, according to the dam assessment information, the dam main body feature information, the dam shape feature information and the dam environment feature information are sorted and processed according to the report of the degree of hidden danger, and the dam hidden danger report information is determined. Among them, the dam hidden danger report information is used to transmit to the mobile communication device of the inspection personnel.

[0092] Furthermore, based on the two-dimensional dam hidden danger information, the three-dimensional dam hidden danger model and the dam hidden danger report information, the real-time detection of the dam is completed.

[0093] In one embodiment, the risk detection results can also be output in a visualized form, such as generating a three-dimensional model of the dam, image annotation, and report. For example, the three-dimensional model can intuitively display the topography and safety status of the dam, image annotation can highlight cracks, collapse and other safety hazard areas, and the report can provide detailed data analysis results and safety assessment reports. The visualized risk detection results can be stored in a database for subsequent query and analysis. At the same time, the results can be shared with relevant departments and personnel to provide a decision-making basis for the management and maintenance of water conservancy projects.

[0094] As a feasible implementation method, the drone-based water conservancy project dam detection method of this application can be used for river and lake management: to provide accurate data support for river and lake demarcation, shoreline protection and utilization planning. Through the safety detection of dams, problems in river and lake management can be discovered and dealt with in a timely manner to protect water resources and the ecological environment.

[0095] The drone-based water conservancy project dam detection method of this application can be used for water conservancy project design: in water conservancy project design, the stability and changes of the shoreline are considered to optimize the project design scheme. For example, according to the terrain changes and safety conditions of the dam, the location and structure of the hydraulic structure are adjusted to improve the safety and reliability of the project.

[0096] The drone-based water conservancy project dam detection method of this application can be used for flood prevention and disaster reduction: timely grasp the changes in the shoreline and provide a decision-making basis for flood prevention and disaster reduction. During floods, through real-time monitoring of dams, dangerous situations can be discovered and handled in a timely manner.

[0097] In addition, the present application also provides a water conservancy project dam detection device based on a drone, such as Figure 2 As shown, the water conservancy project dam detection device 200 based on the drone specifically includes:

[0098] At least one processor 201. And a memory 202 in communication with the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, so that the at least one processor 201 can execute:

[0099] Based on the preset UAV flight plan, real-time inspection images of the dam are collected;

[0100] Perform risk identification processing on the main features of the dam on the real-time detection image to obtain the main feature information of the dam;

[0101] Through the preset dam laser radar data, the real-time detection image is calculated and processed to obtain the dam shape feature information;

[0102] According to the reflectivity data of the preset multi-spectral image, the real-time detection image is processed for the index determination of the dam environment characteristics to obtain the dam environment characteristic information;

[0103] Conduct safety assessment on the dam main body feature information, dam shape feature information and dam environment feature information to obtain dam assessment information;

[0104] Based on the dam assessment information, the dam information is visualized and output to complete the real-time detection of the dam.

[0105] The embodiment of the present application uses drones to quickly cover a wide area of ​​a large-scale water conservancy project dam in a short period of time, greatly saving time and labor costs. At the same time, it is equipped with a variety of sensors, such as high-resolution cameras, multi-spectral cameras, and lidars, to achieve multi-dimensional detection of dams. It can provide high-precision terrain information and three-dimensional point cloud data, and can accurately detect problems such as slight deformation, cracks, and collapse of dams. It can also achieve real-time monitoring and early warning, promote the informatization and intelligence of water conservancy project management, and improve the intelligence level of water conservancy project management. And it can use drone inspections to protect the environment and ecology.

[0106] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0107] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0111] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0112] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting dams of water conservancy projects based on drones, characterized in that: The method comprises: Based on the preset UAV flight plan, real-time inspection images of the dam are collected; Performing risk identification processing on the real-time detection image regarding the main features of the dam to obtain the main feature information of the dam; By using preset dam laser radar data, the real-time detection image is subjected to calculation processing on the dam shape features to obtain dam shape feature information; According to the reflectivity data of the preset multi-spectral image, the real-time detection image is subjected to the index determination processing related to the dam environment characteristics to obtain the dam environment characteristic information; Performing safety assessment processing on the dam main body characteristic information, the dam shape characteristic information and the dam environment characteristic information to obtain dam assessment information; Based on the dam assessment information, the dam information is processed for visual output to complete real-time detection of the dam.

2. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: Based on the preset UAV flight plan, real-time detection images of the dam are collected, including: According to the basic parameters and environmental conditions of the dam, the coverage calculation of the shooting angle of view of the area where the dam is located is performed to determine the height parameter and angle parameter of the drone; wherein the basic parameters include: dam length, dam width and dam height; the environmental conditions include: visibility on sunny days, visibility on cloudy days and water level; The drone shooting route selection process is performed on the dam area by using the height parameter and the angle parameter to generate the drone flight plan; Based on the UAV flight planning, and using a high-resolution camera, image information of the dam area is collected and processed to obtain a high-definition image of the dam; The multispectral image of the dam area is collected and processed by a multispectral camera preset on the drone to obtain reflectance data; wherein the reflectance data is used to represent vegetation coverage information and soil moisture information; The laser radar equipment preset on the UAV is used to perform terrain scanning processing on the dam area to obtain the dam laser radar data; The high-definition image of the dam, the reflectivity data and the laser radar data of the dam are integrated and processed to obtain the real-time detection image.

3. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: Before performing risk identification processing on the real-time detection image regarding the main features of the dam to obtain the main feature information of the dam, the method further includes: Performing radiation correction on the high-definition image of the dam in the real-time detection image, and eliminating the photometric difference data in the high-definition image of the dam, to obtain a pre-processed high-definition image of the dam; The pre-processed dam high-definition image is corrected with respect to the flight attitude and camera parameters of the drone by using a geometric correction algorithm, and the image coordinates in the pre-processed dam high-definition image are converted into actual geographic coordinates to determine a high-quality dam high-definition image; By using a preset principal component analysis technique, the multispectral image in the real-time detection image is subjected to a band fusion process of relevant reflectivity to obtain a fused multispectral image; According to the atmospheric radiation transmission model, the fused multispectral image is corrected for atmospheric factors to obtain a high-quality multispectral image; Based on the high-quality dam high-definition image and the high-quality multi-spectral image, an optimized real-time detection image is determined.

4. The method for detecting dams of water conservancy projects based on drones according to claim 3 is characterized in that: The real-time detection image is subjected to risk identification processing related to the main features of the dam to obtain the main feature information of the dam, specifically including: According to the Canny edge detection algorithm, the gradient change of the pixels of the main body of the dam is calculated for the high-quality high-definition image of the dam in the real-time detection image to determine the edge position of the main body of the dam; Based on the edge position, continuous edge pixels of the main body of the dam are connected one by one to obtain the main body contour information of the dam; Performing target identification on the risk feature points in the main contour information of the dam, and performing image matching processing on the risk feature points through a preset risk feature template to determine the real risk area characteristics; wherein the risk feature points at least include: crack feature points, collapse feature points and slope deformation feature points; According to the real risk area corresponding to the real risk area characteristics, the background image segmentation processing is performed on the high-quality high-definition image of the high-quality dam to obtain some of the main feature information of the dam; wherein the main feature information of the dam represents the main risk factors of the dam.

5. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: The real-time detection image is processed by calculating the dam shape features using the preset dam laser radar data to obtain dam shape feature information, including: Through a preset point cloud registration algorithm, the dam laser radar data at different time points in the real-time detection image are subjected to point cloud registration processing of the iterative nearest point to obtain a transformation matrix that minimizes the transformation matrix between the two point clouds; By using the least square method, the transformation matrix is ​​subjected to deformation analysis calculation to obtain the point cloud displacement and point cloud deformation amount between the two point clouds that are minimized; According to the point cloud displacement and the point cloud deformation, the amplitude of the terrain deformation is calculated for the terrain height data in the dam laser radar data at different time points to obtain terrain change amplitude information; According to the point cloud displacement and the point cloud deformation, and based on the digital elevation model generated corresponding to the dam laser radar data, the slope deformation variable of the slope data in the dam laser radar data at different time points is calculated to obtain the slope change information; The terrain change amplitude information and the slope change information are integrated to obtain the dam shape feature information based on the real-time detection image.

6. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: According to the reflectivity data of the preset multi-spectral image, the real-time detection image is subjected to the index determination processing related to the dam environment characteristics to obtain the dam environment characteristic information, which specifically includes: By using a preset NDVI algorithm, the near-infrared band reflectance and the red light band reflectance in the reflectance data are compared and calculated to obtain a vegetation coverage index based on the real-time detection image; By using a preset TUDI algorithm, the temperature extreme value calculation is performed on the surface temperature change in the reflectivity data to obtain a soil moisture index based on the real-time detection image; The vegetation coverage index and the soil moisture index are both subjected to information extraction processing of dam environment characteristics to determine the dam environment characteristic information.

7. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: The dam main body feature information, the dam shape feature information and the dam environment feature information are subjected to safety assessment processing to obtain dam assessment information, specifically including: Determine the weight coefficient corresponding to each feature information based on the proportion of the safety factor of the dam main body feature information, the dam shape feature information and the dam environment feature information; Through a preset dam safety assessment index system, the dam main body feature information, the dam shape feature information and the dam environment feature information are subjected to a decentralized calculation of relevant weight coefficients to determine a risk score of the dam; The risk score is divided into thresholds to obtain the dam assessment information; wherein the dam assessment information includes: safety assessment information, early warning assessment information and danger assessment information.

8. The method for detecting dams of water conservancy projects based on drones according to claim 1 is characterized in that: Based on the dam assessment information, visual output processing of the dam information is performed, specifically including: According to the dam assessment information, risk labeling processing is performed on the real-time detection image to obtain two-dimensional dam hidden danger information; Based on the three-dimensional dam model established by the real-time detection image, the dam assessment information is divided and marked in the model area to generate a three-dimensional dam hidden danger model; According to the dam assessment information, the dam main body feature information, the dam shape feature information and the dam environment feature information are sorted according to the report of the degree of hidden danger, and the dam hidden danger report information is determined; wherein the dam hidden danger report information is used to be transmitted to the mobile communication device of the inspection personnel; Based on the two-dimensional dam hidden danger information, the three-dimensional dam hidden danger model and the dam hidden danger report information, the real-time detection of the dam is completed.

9. A water conservancy project dam detection device based on drone, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the water conservancy project dam detection method based on a drone according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, which stores at least one program. Each of the programs includes instructions. When the instructions are executed by the terminal, the terminal executes a water conservancy project dam detection method based on a drone according to any one of claims 1-8.

Images