Unmanned aerial vehicle-based remote sensing image acquisition and processing method and system, and storage medium

Abstract

The invention discloses a remote sensing image acquisition and processing method and system based on an unmanned aerial vehicle, and a storage medium, and relates to the field of remote sensing image analysis. Collecting a visible light image and a thermal infrared image of the landform data by using an unmanned aerial vehicle; for the visible light image, terrain and river features are obtained from landform data, structured light and multi-view stereoscopic vision are adopted to carry out three-dimensional reconstruction on the terrain and river features, and a Canny edge detection algorithm is adopted to identify edge information in a three-dimensional reconstruction image; for the thermal infrared image, based on single-channel temperature data, binarization processing is carried out on the image by setting a fixed temperature threshold value, and masks of a water body area and a non-water body area are generated; and matching the edge information with the mask, and identifying the position of the river bank line. According to the method, RGB and thermal infrared (TIR) images are combined, and through the structured light and multi-view stereoscopic vision technology, edge detection and a machine learning algorithm, accurate analysis of the river form and identification of a torrential zone and a calm zone are achieved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image analysis, and more specifically to a method, system and storage medium for acquiring and processing remote sensing images based on unmanned aerial vehicles. Background Art

[0002] In recent years, with the rapid development of remote sensing technology, the field of environmental and geomorphological research has benefited greatly. The widespread use of unmanned aerial vehicles (UAVs) is particularly significant. Their low cost, high precision and flexible operation have gradually replaced traditional satellite remote sensing and ground survey methods. Compared with traditional methods, UAVs can flexibly collect high-resolution images under complex environmental conditions, which provides more reliable and accurate data for hydrological, ecological and geomorphological research.

[0003] In drone image data collection, RGB cameras are used to capture visible light images with high resolution, which are suitable for the detection and analysis of geomorphic features. However, the performance of RGB images may be limited by environmental conditions such as shadows and lighting, especially in water bodies or low reflectivity areas, where it may be difficult to distinguish different features. To solve this problem, thermal infrared (TIR) ​​images record the distribution information of surface temperature and can identify objects under different lighting conditions. TIR images provide additional physical property support for hydrological geomorphological analysis, especially in separating water bodies and land areas and identifying thermal features of river channels.

[0004] By fusing RGB and TIR images, researchers can more comprehensively detect environmental features, especially in areas with complex river morphology. The temperature data provided by TIR images can effectively enhance the recognition ability of RGB images in physical similarities or insufficient lighting conditions, greatly improving the detection accuracy of water bodies and other landforms.

[0005] However, there are still many problems in the existing technology. For example, in image processing, edge detection is a key step in extracting surface features. How to ensure the accuracy of the edge detection process? In river landform research, accurate identification of river banks is crucial for river channel analysis. How to clearly identify rocks and water flows in turbulent areas, etc. Summary of the invention

[0006] In view of this, the present invention provides a method, system and storage medium for acquiring and processing remote sensing images based on unmanned aerial vehicles, combining RGB and thermal infrared (TIR) ​​images, and using structured light and multi-viewpoint stereo vision (SfM-MVS) technology, edge detection and machine learning algorithms to achieve accurate analysis of river morphology and identification of turbulent and calm areas.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for acquiring and processing remote sensing images based on unmanned aerial vehicles comprises the following steps:

[0009] Using drones to collect visible light and thermal infrared images of geomorphic data;

[0010] For visible light images, terrain and river features are obtained from geomorphic data, and structured light and multi-viewpoint stereo vision are used to reconstruct terrain and river features in three dimensions, and the Canny edge detection algorithm is used to identify edge information in the three-dimensional reconstructed image;

[0011] For thermal infrared images, based on single-channel temperature data, the images are binarized by setting a fixed temperature threshold to generate masks for water and non-water areas;

[0012] Match edge information and masks to identify the location of riverbanks.

[0013] Optionally, it also includes analyzing environmental features in visible light images and thermal infrared images, and using machine learning algorithms to automatically adjust edge detection thresholds and mask generation parameters according to different data sets to improve recognition accuracy.

[0014] Optionally, the drone is equipped with RGB and TIR cameras, with precise positioning through the integrated GPS and GLONASS global navigation satellite systems, and RTK technology to further improve the image georeferencing accuracy.

[0015] Optionally, drones are used to collect topographic data to fully cover the target area via a preset flight path.

[0016] Optionally, before applying the Canny algorithm, the visible light image is converted to grayscale and subjected to noise processing by a Gaussian filter.

[0017] A remote sensing image acquisition and processing system based on unmanned aerial vehicle, comprising:

[0018] Image acquisition module: used to collect visible light images and thermal infrared images of landform data using drones;

[0019] 3D reconstruction module: used to obtain terrain and river features from landform data for visible light images, use structured light and multi-viewpoint stereo vision to reconstruct terrain and river features in 3D, and use the Canny edge detection algorithm to identify edge information in the 3D reconstructed image;

[0020] Binarization processing module: for thermal infrared images, based on single-channel temperature data, by setting a fixed temperature threshold to perform binary processing on the image, and generate masks for water and non-water areas;

[0021] Riverbank recognition module: used to match edge information and masks to identify the riverbank location.

[0022] A computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the methods for acquiring and processing remote sensing images based on unmanned aerial vehicles are implemented.

[0023] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a method, system and storage medium for acquiring and processing remote sensing images based on drones, which have the following beneficial effects:

[0024] 1. The present invention adopts the Canny edge detection algorithm, which is widely used in geomorphological research, especially riverbank detection and river morphological feature recognition due to its high efficiency in noise suppression and edge detection. In order to ensure the detection accuracy, the RGB image first needs to be grayscale converted and denoised. The present invention uses a grayscale conversion method based on the average value of the red, green and blue color spectra. This method can simplify the subsequent image processing steps and enhance the contrast between the water body and the ground. In terms of noise reduction, the image is smoothed by using a Gaussian filter to reduce the interference caused by lighting differences, shadows and background noise, thereby ensuring the accuracy of the edge detection process.

[0025] 2. In order to identify rivers and other geomorphic features, the present invention processes the TIR image through binarization technology and sets a fixed temperature threshold to generate a mask of the water area. This method can accurately distinguish water from other areas with significant temperature differences, such as rocks or vegetation. The TIR mask combined with the edge detection results in the RGB image greatly improves the detection accuracy of riverbanks and other key geomorphic features.

[0026] 3. The present invention uses an algorithm based on correlation matching to further improve the detection accuracy of riverbanks. The identification of turbulent and calm areas in the river is particularly critical for the study of water flow and river morphology. TIR images can accurately detect areas with large temperature differences. By combining thermal imaging and edge information in RGB images, rocks and water in turbulent areas can be clearly identified, further refining the analysis of river characteristics.

[0027] 4. The present invention introduces machine learning technologies such as convolutional neural networks (CNNs) to achieve adaptive parameter tuning. Through machine learning, the toolbox can automatically adjust the parameters of edge detection and mask generation according to the characteristics of different data sets, thereby ensuring consistent analysis results under various environmental conditions. The machine learning algorithm can adaptively adjust the filter parameters and edge detection thresholds by analyzing different noise, light and environmental characteristics to ensure that the detection results in complex environments have high accuracy and stability. The application of this technology enables the present invention to achieve efficient recognition and analysis of river characteristics in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0029] Figure 1 It is a schematic diagram of the process of the present invention;

[0030] Figure 2 This is a framework diagram of the turbulent zone-calm zone identification technology of the present invention;

[0031] Figure 3 It is a specific image schematic diagram of one embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Example 1

[0034] The embodiment of the present invention discloses a method for acquiring and processing remote sensing images based on unmanned aerial vehicles, such as Figure 1 As shown, the following steps are included:

[0035] Step 1: Use drones to collect visible light images and thermal infrared images of landform data;

[0036] Step 2: For visible light images, terrain and river features are obtained from geomorphic data, and structured light and multi-view stereo vision are used to reconstruct the terrain and river features in three dimensions, and the Canny edge detection algorithm is used to identify edge information in the three-dimensional reconstructed image;

[0037] Step 3: For thermal infrared images, based on single-channel temperature data, the image is binarized by setting a fixed temperature threshold to generate masks for water and non-water areas;

[0038] Step 4: Match edge information and mask to identify the riverbank location.

[0039] Furthermore, in step 1, the present invention uses a DJI Mavic Enterprise Advanced drone for data collection. The drone is equipped with RGB and TIR cameras, which can simultaneously capture high-resolution visible light and thermal infrared images. The resolution of the RGB camera is 48 million pixels, and the thermal infrared camera uses an integrated radiometric thermal sensor with a focal length of 38 mm. The drone is accurately positioned through the integrated GPS and GLONASS global navigation satellite system (GNSS), and the RTK technology is used to further improve the image geo-registration accuracy, ensuring that centimeter-level coordinate accuracy can be provided at different altitudes and different terrain conditions.

[0040] During data collection, a pre-programmed flight path was used, combined with manual operation to avoid obstacles and ensure full coverage of the target area. The flight altitude was maintained at 70 meters to obtain a constant ground resolution, and the forward and lateral overlap rates of RGB and TIR images were set to 70% to ensure good alignment of the images. In the end, 1,400 RGB and TIR static images were collected, covering the upstream and downstream of the study area, laying the data foundation for subsequent image processing and analysis.

[0041] Furthermore, in step 2, in order to obtain terrain and river features from the collected image data, the present invention uses structured light and multi-view stereo vision (SfM-MVS) technology for three-dimensional reconstruction. This method generates high-precision digital elevation models (DEMs) and orthophotos through multi-view image sequences. Agisoft Metashape Pro v1.5.3 software is used to perform three-dimensional reconstruction of RGB and TIR images using its mature SfM-MVS processing flow. During the processing, the images are first aligned and corrected using GNSS data to extract sparse point clouds, which are then further refined through automatically generated binding points to generate high-resolution dense point clouds.

[0042] When processing the RGB images after 3D reconstruction, the Canny edge detection algorithm is used. This algorithm is widely used in image processing and can efficiently identify the edges of objects in the image. Before applying the Canny algorithm, the RGB image is first converted to grayscale and processed with a Gaussian filter to reduce interference caused by lighting differences or background objects.

[0043] Furthermore, in step 3, for TIR images, the conventional image alignment method may not be able to obtain good results due to the difficulty in matching its thermal features. Therefore, artificial markers are introduced to enhance the alignment effect of thermal images, ensuring that the same accuracy as RGB images can be achieved when generating digital elevation models and orthophotos of TIR images.

[0044] For TIR images, based on their single-channel temperature data, the images are binarized by setting a fixed temperature threshold to generate masks for water and non-water areas. Water bodies in TIR images usually have lower temperatures, so the temperature threshold can be used to clearly distinguish water bodies from other landforms. Combined with the edge detection results in RGB images, the TIR mask further enhances the recognition of riverbanks and river features, especially in areas with poor lighting conditions, where TIR data can provide additional information support.

[0045] Furthermore, in step 4, if Figure 2 As shown in the figure, accurate riverbank recognition is a key step in river morphology analysis. By matching the edge information in the RGB image with the mask generated by TIR, the present invention uses an algorithm based on correlation matching to improve the detection accuracy of the riverbank. Through the combination of this multi-source data, the influence caused by illumination or background noise can be eliminated, and the location of the riverbank can be more accurately identified.

[0046] The identification of turbulent and calm areas is done by analyzing the temperature differences in the TIR mask. In turbulent areas, the water flows quickly, and the rocks and turbulent water show a high temperature difference in the TIR image, making them easy to distinguish. In calm areas, the water flows slowly and the thermal infrared features are more uniform. The mask after binary processing can effectively separate these areas. Combined with the Canny edge detection results, the turbulent areas, stone distribution, and water flow characteristics in the river can be accurately identified.

[0047] To further improve the adaptability of the toolbox under different data sets and environmental conditions, this paper introduces machine learning technologies such as convolutional neural networks (CNNs) to achieve adaptive tuning of parameters. By analyzing environmental features in the image (such as noise, light, temperature differences, etc.), the machine learning algorithm can automatically adjust the threshold of edge detection and the parameters of mask generation according to different data sets.

[0048] In the specific implementation process, the toolbox first extracts features from the image through a convolutional neural network, and trains the model in combination with historical data to optimize parameter settings. This adaptive tuning process can significantly improve the accuracy of image processing in complex environments. For example, under different lighting conditions, the machine learning model can automatically adjust the parameters of the Gaussian filter to ensure the effectiveness of noise processing. At the same time, during the mask generation process of the TIR image, the model can adjust the threshold range of binarization according to the temperature characteristics of water bodies and other landforms to ensure accurate identification of water body and river characteristics.

[0049] This embodiment also discloses a system for acquiring and processing remote sensing images based on an unmanned aerial vehicle, comprising:

[0050] Image acquisition module: used to collect visible light images and thermal infrared images of landform data using drones;

[0051] 3D reconstruction module: used to obtain terrain and river features from landform data for visible light images, use structured light and multi-viewpoint stereo vision to reconstruct terrain and river features in 3D, and use the Canny edge detection algorithm to identify edge information in the 3D reconstructed image;

[0052] Binarization processing module: for thermal infrared images, based on single-channel temperature data, by setting a fixed temperature threshold to perform binary processing on the image, and generate masks for water and non-water areas;

[0053] Riverbank recognition module: used to match edge information and masks to identify the riverbank location.

[0054] Finally, a computer storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, any one of the steps of the method for acquiring and processing remote sensing images based on unmanned aerial vehicles is implemented.

[0055] Example 2

[0056] like Figure 3 As shown in the figure, the following technical scheme is to verify the accuracy of the detection results. The center line, river area, average width and tortuosity of the river channel are extracted from the images collected from Jiufeng Reservoir, and the field data are used for comparative verification. Among the extracted river channel features, the error between the average width of the river and the measured data is kept within the range of ±3%, and the extraction error of the center line of the river channel is within ±5 meters. The overall error is small, indicating that the technical scheme shows high accuracy in the detection and extraction of hydrological features.

[0057] In addition, the generated 3D data was used to perform a statistical analysis of the distribution of turbulent and calm areas. Experimental data showed that turbulent areas are mainly distributed in areas where water flow is obstructed and in river bends, while calm areas are concentrated in areas where water flow is gentle and there is no significant resistance. The detection accuracy of turbulent areas and rock distribution is high, which is highly consistent with the results of field surveys, verifying the applicability of the toolbox under different hydrological conditions.

[0058] In order to verify the accuracy of the detection results, the centerline, river area, average width and tortuosity of the river channel were extracted from the images collected from Jiufeng Reservoir, and compared with field data for verification. Among the extracted river channel features, the error between the average width of the river and the measured data was kept within the range of ±3%, and the extraction error of the centerline of the river channel was within ±5 meters. The overall error was small, indicating that the technical solution showed high accuracy in the detection and extraction of hydrological features.

[0059] In addition, the generated 3D data was used to perform a statistical analysis of the distribution of turbulent and calm areas. Experimental data showed that turbulent areas are mainly distributed in areas where water flow is obstructed and in river bends, while calm areas are concentrated in areas where water flow is gentle and there is no significant resistance. The detection accuracy of turbulent areas and rock distribution is high, which is highly consistent with the results of field surveys, verifying the applicability of the toolbox under different hydrological conditions.

[0060] In the Jiufeng Reservoir case study, the pyRPA toolbox demonstrated its applicability and robustness in complex environments by integrating multi-source data. When dealing with complex areas where bridges and roads intersect, the toolbox can accurately distinguish different landforms and enhance the ability to distinguish water bodies from other landforms through TIR images. In the identification of turbulent and calm areas, the pyRPA toolbox can use the temperature data of TIR images to effectively improve the recognition accuracy of rocks and water in the river.

[0061] When processing the experimental data of Jiufeng Reservoir, the computing speed and storage efficiency of the toolbox were also fully verified. Through GPU acceleration and optimization algorithms, the time consumption of the entire data processing process was significantly shortened. Even when faced with the processing volume of thousands of sets of images, the toolbox can complete 3D reconstruction, river channel detection and feature extraction in a relatively short time. Overall, the experimental results of the pyRPA toolbox in Jiufeng Reservoir show that it has strong practicality and can be applied to large-scale river morphology analysis and hydrological monitoring projects.

[0062] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0063] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for acquiring and processing remote sensing images based on unmanned aerial vehicles, characterized in that: The following steps are involved: Using drones to collect visible light and thermal infrared images of geomorphic data; For visible light images, terrain and river features are obtained from geomorphic data, and structured light and multi-viewpoint stereo vision are used to reconstruct terrain and river features in three dimensions, and the Canny edge detection algorithm is used to identify edge information in the three-dimensional reconstructed image; For thermal infrared images, based on single-channel temperature data, the images are binarized by setting a fixed temperature threshold to generate masks for water and non-water areas; Match edge information and masks to identify the location of riverbanks.

2. The method for acquiring and processing remote sensing images based on unmanned aerial vehicles according to claim 1, characterized in that: It also includes analyzing environmental features in visible light images and thermal infrared images, and using machine learning algorithms to automatically adjust edge detection thresholds and mask generation parameters based on different data sets to improve recognition accuracy.

3. The method for acquiring and processing remote sensing images based on unmanned aerial vehicles according to claim 1 is characterized in that: The drone is equipped with RGB and TIR cameras, and uses integrated GPS and GLONASS global navigation satellite systems for precise positioning, and RTK technology to further improve the accuracy of image georeferencing.

4. The method for acquiring and processing remote sensing images based on unmanned aerial vehicles according to claim 1 is characterized in that: Use drones to collect topographic data and fully cover the target area through a preset flight path.

5. The method for acquiring and processing remote sensing images based on unmanned aerial vehicles according to claim 1 is characterized in that: Before applying the Canny algorithm, the visible light image is converted to grayscale and subjected to noise processing by a Gaussian filter.

6. A remote sensing image acquisition and processing system based on unmanned aerial vehicle, characterized in that: include: Image acquisition module: used to collect visible light images and thermal infrared images of landform data using drones; 3D reconstruction module: used to obtain terrain and river features from landform data for visible light images, use structured light and multi-viewpoint stereo vision to reconstruct terrain and river features in 3D, and use the Canny edge detection algorithm to identify edge information in the 3D reconstructed image; Binarization processing module: for thermal infrared images, based on single-channel temperature data, by setting a fixed temperature threshold to perform binary processing on the image, and generate masks for water and non-water areas; Riverbank recognition module: used to match edge information and masks to identify the riverbank location.

7. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by the processor, the steps of a method for acquiring and processing remote sensing images based on unmanned aerial vehicles as described in any one of claims 1 to 5 are implemented.

Images