A method and system for detecting water surface area based on lidar and deep learning

Abstract

The application discloses a method and system for detecting water surface area based on laser radar and deep learning, comprising: obtaining flight data information of a UAV; obtaining point cloud data and image data within a preset range; determining whether the point cloud data contains first water surface data; when it is determined that the point cloud data contains the first water surface data, determining whether the image data contains second water surface data; when it is determined that the image data contains the second water surface data, calculating a water surface area in combination with the second water surface data and the flight data information, and saving the water surface area. The point cloud data is collected by a laser radar carried by a UAV, the image data is collected by an onboard camera, the point cloud data and the image data are processed and calculated in real time, real-time surveying of a water surface by the UAV is realized, and therefore the intelligent identification function of the UAV in a road surface surveying or water surface detection task is realized, the identification of a water surface area is more accurate, and the detection precision of the water surface area is improved.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically to a method and system for detecting water surface area based on lidar and deep learning. Background Technology

[0002] Unmanned Aerial Vehicles (UAVs) are widely used in various fields, but due to factors such as variable environments and complex terrain, UAV inspection and surveying tasks still require human intervention. Currently, UAVs primarily function as data collection tools in most tasks. In ground surveying tasks, UAVs cannot identify the specific conditions of the road surface in real time; for example, they cannot identify surface water in real time, thus requiring human intervention for these tasks.

[0003] In existing technologies, most water surface identification techniques rely on remote sensing data. While remote sensing images offer high accuracy, reaching decimeter-level precision, filtering and selecting target data from large-scale images requires extensive processing and time. Furthermore, much of this processing is done manually, lacking automation. These technologies are particularly costly in terms of time, manpower, and resources for identifying water surfaces such as canals, streams, ponds, and rainwater deposits. The final identification results may also be inaccurate due to the small size and constant change of these water surfaces. Summary of the Invention

[0004] The purpose of this invention is to solve the problems that UAVs cannot identify the specific conditions of the road surface in real time, and that the cost of identifying water surface areas using remote sensing data is high. This invention provides a method and system for detecting water surface area based on lidar and deep learning. By processing point cloud data and image data through an onboard computer, the target detection area can be detected and identified in real time, and the water surface area can be determined. This saves human resource costs and improves detection efficiency and accuracy.

[0005] The embodiments in this specification provide the following technical solutions:

[0006] This invention provides a method for detecting water surface area based on lidar and deep learning, comprising:

[0007] Acquire flight data information from drones;

[0008] Acquire point cloud data and image data within a preset range;

[0009] Determine whether the point cloud data contains the first water surface data;

[0010] When it is determined that the point cloud data contains the first water surface data, determine whether the image data contains the second water surface data.

[0011] When it is determined that the image data contains second water surface data, the water surface area is calculated by combining the second water surface data and flight data information, and the water surface area is saved.

[0012] During implementation, flight data is acquired in real time through a flight control module installed on the drone. This flight data includes the drone's flight position, altitude, and the image scale of the detection area. Preferably, the flight data is saved to a flight data file in real time.

[0013] In implementation, point cloud data within a preset range is collected using lidar. Taking advantage of the characteristic that lidar signals cannot echo normally on the water surface, a point cloud data clustering algorithm is used to analyze and cluster the point cloud data collected by the lidar, calculating the amount of point cloud data loss. When the amount of loss reaches a first threshold, it is determined that the point cloud data includes the first water surface data. Preferably, the point cloud data is saved to a point cloud data file in real time.

[0014] In implementation, image data within a preset range is acquired using an airborne camera. This image data is then input into a deep learning-based model. The model uses this model to obtain image features of each region within the preset range, and the model determines whether the image data contains second water surface data based on these features. Preferably, the image data is saved to an image data file in real time.

[0015] In practice, when it is determined that the point cloud data within the preset range contains the first water surface data, the image data acquired by the airborne camera within the same range is detected. A deep learning algorithm is used to detect whether the same range contains the second water surface data. When the image data contains the second water surface data, the water surface area is calculated according to the actual ratio based on the second water surface data, combined with information such as flight altitude, position and image ratio of the detection area in the flight data information, and then saved.

[0016] In practice, lidar can perform long-range detection both day and night, and is unaffected by weather factors such as light, rain, and fog. It can monitor a long distance and has high monitoring efficiency. While camera modules are more accurate in monitoring images, they have higher processing costs for image data, requiring extensive processing to extract useful information. Therefore, the system determines whether the second water surface is included in the image data collected by the camera after lidar detects the first water surface. This makes the monitoring of data within the preset range more accurate, while reducing the amount of data processing and lowering system costs.

[0017] In the embodiments of this invention, point cloud data is acquired in real time using a lidar system, and image data is acquired in real time using an airborne camera. The point cloud data and image data are then processed and calculated by an airborne computer. This fusion of lidar and airborne camera enables real-time detection of the water surface, thereby achieving intelligent identification capabilities for UAVs in road surveying or water surface detection tasks.

[0018] When the onboard computer determines that the point cloud data contains a water surface area, it then performs calculations on the image data within the same range, reducing the calculation process for the image data. At the same time, the fusion calculation of point cloud data and image data improves the accuracy of water surface area recognition and enhances detection precision.

[0019] In some implementations, a method for detecting water surface area based on lidar and deep learning further includes: when it is determined that the image data does not contain second water surface data, saving the image data, labeling the image data based on flight data information, and training the image data using a deep learning algorithm.

[0020] In implementation, a training model is built based on a deep learning algorithm. When it is determined that the point cloud data within a preset range contains first water surface data and the image data does not contain second water surface data, targets are selected and manually labeled. These targets are then input into the training model for training to ensure that the trained targets do not contain water surface data. By using manually labeled targets for training, the deep learning algorithm in the training model is corrected to improve its recognition rate.

[0021] In some implementations, a method for detecting water surface area based on lidar and deep learning further includes: when it is determined that the point cloud data does not contain the first water surface data, updating a preset range and detecting the point cloud data within the updated preset range.

[0022] During implementation, point cloud data within a preset range is collected using lidar. Taking advantage of the characteristic that lidar signals cannot echo properly on water, a point cloud data clustering algorithm is used to analyze and cluster the collected point cloud data. The number of lost point cloud data points is calculated. When the number of lost points is less than a first threshold, it is determined that the point cloud data does not include the first water surface data. The flight control module then controls the UAV to continue operating, updating the detection range, collecting the updated point cloud data within the preset range, and performing analysis and clustering.

[0023] In practice, when the airborne computer determines that the point cloud data contains a water surface area, it then performs calculations on the image data within the same range. When the point cloud data does not contain a water surface area, there is no need to process the image data, reducing the calculation process and effectively improving work efficiency.

[0024] In some implementations, determining whether the point cloud data contains first water surface data includes:

[0025] Preprocess the point cloud data;

[0026] Based on point cloud data clustering algorithm, calculate the number of point cloud data losses within a preset range;

[0027] When the number of losses is not less than the first threshold, it is determined that the point cloud data includes the first water surface data.

[0028] In implementation, the point cloud data acquired by the lidar is preprocessed, including filtering and segmentation. Utilizing the characteristic that lidar signals cannot echo properly on water surfaces, a point cloud data clustering algorithm is used to analyze and cluster the acquired point cloud data, calculating the amount of data loss. If the amount of loss is not less than a first threshold, the point cloud data is determined to include the first water surface data. If the amount of loss is less than the first threshold, the point cloud data is determined to not include the first water surface data. Preferably, the point cloud data is saved to a point cloud data file in real time.

[0029] It should be noted that the first threshold is a parameter value used to determine whether the amount of loss in the point cloud data meets the requirements for the signal loss of the water surface reflection lidar. The specific value can be set according to the actual hardware equipment used, etc., and is not limited here.

[0030] In some implementations, it is determined whether the image data contains second water surface data;

[0031] Preprocess the image data;

[0032] The processed image data is trained using the deep learning Unet algorithm to obtain a deep learning model;

[0033] Based on a deep learning model, determine whether the image data contains data about a second water surface.

[0034] In practice, the image data acquired by the airborne camera is preprocessed, including image enhancement. A training model is established based on a deep learning algorithm. The preprocessed image data is input into the training model for training, and image features are extracted from the image data. When the image features include information that matches the water surface data, it is determined that the image data contains the second water surface data. When the image features do not include information that matches the water surface data, it is determined that the image data does not contain the second water surface data. The first water surface data identified by the lidar is manually labeled, and the deep learning algorithm in the training model is corrected based on the manually labeled information.

[0035] The present invention also provides a system for detecting water surface area based on lidar and deep learning. The system is used to execute the method steps of any one of the embodiments of the present invention, including:

[0036] Flight control module: The flight control module is installed on the drone and is used to acquire the drone's flight data information;

[0037] The radar module acquires and outputs point cloud data within a preset detection range based on lidar.

[0038] The camera module acquires and outputs image data within a preset detection range based on the airborne camera;

[0039] The onboard computer processes point cloud data and image data, and combines this with flight data to calculate and save the water surface area.

[0040] In practice, the flight control module is integrated with the UAV to acquire the UAV's flight data and control its flight path. This flight data includes the UAV's flight position, altitude, and the scale of the detected area image, and is transmitted to the onboard computer via a corresponding SDK.

[0041] By using an onboard computer to process and calculate point cloud data and image data in real time, it can determine whether there is a water surface in the data collected by lidar and airborne camera. By integrating the calculations from lidar and airborne camera data, real-time detection of the water surface is achieved, thereby enabling UAVs to intelligently identify water surfaces in road surveying or water surface detection tasks. This reduces costs while improving the detection accuracy of water surface areas.

[0042] As a further improvement of the present invention, the airborne computer includes:

[0043] The first judgment module processes the point cloud data and determines whether the point cloud data contains the first water surface data.

[0044] The second judgment module processes the image data and determines whether the image data contains the second water surface data.

[0045] The data processing module calculates and saves the water surface area based on the second water surface data and flight data.

[0046] In practice, the first judgment module performs preprocessing such as filtering and segmentation on the point cloud data collected by the lidar. Taking advantage of the fact that lidar signals cannot echo normally on the water surface, the point cloud data is analyzed and clustered based on the point cloud data clustering algorithm. The loss of the point cloud data is calculated. When the loss is not less than the first threshold, it is determined that the point cloud data includes the first water surface data.

[0047] In practice, the second judgment module performs preprocessing such as image enhancement on the image data collected by the airborne camera, and uses a model trained with a deep learning algorithm based on Unet to identify the water surface in the image after processing the image data.

[0048] The beneficial effects of this invention are:

[0049] Point cloud data is collected by lidar on the drone and image data is collected by the onboard camera. The onboard computer processes and calculates the point cloud data and image data in real time. When it is determined that there is a water surface area in the point cloud data and there is also a water surface area in the same location in the image data, the area of ​​the water surface area is calculated based on the image data information and flight data information. No manual calculation is required, saving human resource costs.

[0050] Meanwhile, the integrated use of lidar and airborne cameras enables drones to conduct real-time surveys of water surfaces, thereby enabling drones to perform intelligent identification in road surveys or water surface detection tasks, making the identification of water surface areas more accurate and improving the detection precision of water surface areas.

[0051] The above description of the invention is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0052] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0053] Figure 1 This is a flowchart of a method for detecting water surface area based on lidar and deep learning according to the present invention;

[0054] Figure 2 This is a structural diagram of a system for detecting water surface area based on lidar and deep learning according to the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only one preferred embodiment of this invention and are only used to explain this invention. They do not limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0056] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures; the process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0057] This invention provides a method for detecting water surface area based on lidar and deep learning, such as... Figure 1 As shown, it includes: acquiring flight data information in real time through a flight control module installed on the drone, collecting point cloud data within a preset range through a radar module, and collecting image data within a preset range through a camera module;

[0058] Flight data, point cloud data, and image data are transmitted to the onboard computer in real time.

[0059] The airborne computer processes point cloud data based on a point cloud data clustering algorithm. It takes advantage of the fact that lidar signals cannot echo normally on the water surface to detect whether the point cloud data includes the first water surface data. When the point cloud data does not include the first water surface data, the flight control module controls the UAV to continue operating, update the detection range, collect the point cloud data within the updated preset range, and perform analysis and clustering.

[0060] When a water surface area is detected in the point cloud data, the image data within the same range is calculated. Using a model based on a deep learning algorithm, the image features of each region within a preset range are obtained. It is then determined whether the image data contains second water surface data. If the image data contains second water surface data, the water surface area is calculated by combining the second water surface data and flight data information, and then saved.

[0061] When the image data does not contain the second water surface data, the deep learning algorithm in the training model is corrected by using manually labeled targets for training, thereby improving the recognition rate of the deep learning algorithm.

[0062] In the embodiments of this invention, the integration of lidar and airborne camera enables real-time detection of the water surface, thereby enabling the UAV to intelligently identify water surface areas in road surveying or water surface detection tasks, improving the accuracy of water surface area identification and detection precision.

[0063] In one specific implementation, it is necessary to monitor the water surface area of ​​a pond in a park. First, the radar module on the UAV acquires point cloud data of the entire park. After calculation of the point cloud data, it can be detected that the point cloud data includes the first water surface data, namely the point cloud data of the pond. The onboard computer processes the image data collected by the recording camera and determines that the image data contains the second water surface data, namely the image data of the pond. Based on the image data of the pond and the flight data information, the area of ​​the pond is calculated.

[0064] This invention also provides a system for detecting water surface area based on lidar and deep learning, such as... Figure 2 As shown, it includes: a flight control module, which is integrated with the UAV and used to acquire the UAV's flight data and control its flight path. The flight data includes the UAV's flight position, flight altitude, and the image scale of the detection area, etc., and is transmitted to the onboard computer via a corresponding SDK.

[0065] Preferably, the drone is a DJI M300RTK.

[0066] Preferably, the onboard computer is the DJI Manifold 2, which has strong processing power, fast response speed, and is compatible with multiple flight control systems.

[0067] The radar module, with the radar installed on the bottom of the drone, is used to detect data within a preset range below the drone during flight, acquire point cloud data, and transmit the point cloud data to the onboard computer via a network cable.

[0068] Preferably, the lidar is the Livox AVIA, which can output up to 720,000 points / second of point cloud data. The lidar has a data port of 100Mbps and provides point cloud data to the onboard computer in real time via Ethernet cable.

[0069] The camera module consists of an airborne camera mounted on the bottom of the drone. It is used to capture images of information within a preset area below the drone during flight, obtain image data, and transmit the image data to the onboard computer via the corresponding SDK. The onboard computer processes the point cloud data and image data, and calculates and saves the water surface area by combining it with flight data.

[0070] The specific embodiments described above are preferred embodiments of the method and system for detecting water surface area based on lidar and deep learning of the present invention, and are not intended to limit the specific scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A method for detecting water surface area based on lidar and deep learning, characterized in that, include: Acquire flight data information from drones; Acquire point cloud data and image data within a preset range; Determine whether the point cloud data contains first water surface data; When it is determined that the point cloud data contains the first water surface data, it is determined whether the image data contains the second water surface data; When it is determined that the image data contains the second water surface data, the water surface area is calculated by combining the second water surface data and the flight data information, and the water surface area is saved. The determination of whether the point cloud data contains first water surface data includes: The point cloud data is preprocessed; Based on a point cloud data clustering algorithm, the number of point cloud data losses within a preset range is calculated; When the number of losses is not less than the first threshold, it is determined that the point cloud data includes the first water surface data; The determination is made as to whether the image data contains second water surface data; The image data is preprocessed; The processed image data is trained using the deep learning Unet algorithm to obtain a deep learning model; Based on the deep learning model, it is determined whether the image data contains the second water surface data.

2. The method for detecting water surface area based on lidar and deep learning according to claim 1, characterized in that, The method further includes: When it is determined that the image data does not contain the second water surface data, the image data is saved, and the image data is labeled based on the flight data information. The image data is then trained using a deep learning algorithm.

3. The method for detecting water surface area based on lidar and deep learning according to claim 1, characterized in that, The method further includes: When it is determined that the point cloud data does not contain the first water surface data, the preset range is updated, and the point cloud data within the updated preset range is detected.

4. A system for detecting water surface area based on lidar and deep learning, characterized in that, The system is used to perform the steps of the method as described in any one of claims 1 to 3, the system comprising: A flight control module, which is installed on the UAV, is used to acquire the flight data information of the UAV; The radar module acquires and outputs point cloud data within a preset detection range based on lidar. The camera module acquires and outputs image data within the preset detection range based on the airborne camera; The airborne computer processes the point cloud data and the image data, and calculates and saves the water surface area by combining the flight data information.

5. The system for detecting water surface area based on lidar and deep learning according to claim 4, characterized in that, The airborne computer includes: The first judgment module processes the point cloud data and determines whether the point cloud data contains the first water surface data. The second judgment module processes the image data and determines whether the image data contains the second water surface data. The data processing module calculates the water surface area based on the second water surface data and the flight data information, and saves the water surface area.

Images