Real-time aerial image processing and intelligent mapping system based on deep learning

Abstract

The invention relates to the technical field of image processing, in particular to a real-time aerial image processing and intelligent mapping system based on deep learning, which comprises an image acquisition module, a preprocessing module, a deep learning analysis module, an image splicing module, an intelligent mapping module and a data storage and management module. Multi-type sensor image data are collected in real time through an unmanned aerial vehicle, preprocessing and deep learning analysis are carried out, and target recognition, scene classification and object segmentation of images are achieved. And then, the system splices the plurality of images into a complete map through a feature point matching algorithm and a deep learning optimization algorithm, and generates a high-resolution two-dimensional map and a high-resolution three-dimensional terrain model according to different requirements. Meanwhile, the system supports real-time storage, management and encrypted transmission of data, and ensures the security and integrity of processed image data. According to the invention, the precision and efficiency of aerial image processing are significantly improved, and the method is suitable for complex scenes requiring real-time response, such as disaster monitoring and environmental protection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a real-time aerial image processing and intelligent mapping system based on deep learning. Background Art

[0002] With the rapid development of drone technology, aerial images have been widely used in disaster monitoring, land surveying and mapping, urban planning, environmental protection and other fields. Drones can acquire a wide range of high-resolution image data in real time by carrying high-definition cameras and a variety of sensors, and provide strong support for various geographic information systems (GIS). However, with the complexity of application scenarios, the demand for real-time processing and mapping of aerial images is increasing. Traditional image processing methods often have certain limitations when dealing with complex terrain, dynamic scenes and high real-time requirements.

[0003] In the prior art, traditional aerial image processing methods mainly rely on rule-based image processing algorithms, such as image denoising, feature point matching, and image stitching. The processing accuracy is relatively dependent on environmental conditions, such as lighting and shooting angle. These algorithms have difficulty coping with problems such as lighting changes, image noise, and multi-view stitching errors when facing complex scenes. In addition, existing image processing systems often lack intelligent capabilities and are unable to automatically identify and classify targets in images, resulting in a processing process that relies on a large amount of manual intervention, low efficiency, and low accuracy of the resulting images. Summary of the invention

[0004] Based on the above objectives, the present invention provides a real-time aerial image processing and intelligent mapping system based on deep learning.

[0005] A real-time aerial image processing and intelligent mapping system based on deep learning, comprising an image acquisition module, a preprocessing module, a deep learning analysis module, an image stitching module, an intelligent mapping module and a data storage and management module, wherein; The image acquisition module is used to collect aerial images in real time through the drone, and the collection process includes the synchronous recording of environmental data. The image acquisition module supports the integration of multiple camera sensors to meet the needs of different scenarios; The preprocessing module is used to perform preliminary processing on the collected images, and the preliminary processing includes image denoising, brightness balancing, color correction and distortion correction, and improves the image quality through an image enhancement algorithm based on a convolutional neural network; The deep learning analysis module is used to perform deep learning analysis on the preprocessed images, and the analysis includes target recognition, scene classification and object segmentation functions. Based on a multi-layer convolutional neural network model, the module can automatically detect and classify different targets in the aerial images and generate corresponding feature maps; The image stitching module is used to stitch multiple processed aerial images into a complete map, and the stitching process is performed by an algorithm based on feature point matching and combined with optimization adjustment of a deep learning network; The intelligent mapping module is used to generate the final map from the stitched images, and the map includes a high-resolution two-dimensional image and a three-dimensional modeling result. The intelligent mapping module automatically generates corresponding image outputs according to different application requirements, and can achieve accurate matching between aerial images and actual geographic data by combining with a geographic information system; The data storage and management module is used to store and manage the processed images and generated images, and the data storage and management module supports real-time backup and compressed storage of images.

[0006] Optionally, the image acquisition module is configured with various types of camera devices, including infrared cameras, high-definition cameras and thermal imaging sensors, and different camera devices are switched according to real-time environmental changes to adapt to different lighting and climate conditions; The image acquisition module includes an environmental sensor unit for synchronously recording environmental data related to the aerial image, including temperature, humidity, wind speed and flight altitude.

[0007] Optionally, the preprocessing module includes an image denoising algorithm based on a generative adversarial network, an image brightness adaptive equalization algorithm, and a distortion correction unit, wherein; The image denoising algorithm based on the generative adversarial network is used to reduce the noise in the aerial image and maintain the integrity of the image details; The image brightness adaptive equalization algorithm dynamically adjusts the brightness and contrast of the image according to the brightness distribution of the aerial image to ensure the clarity of the image under various lighting conditions; The distortion correction unit automatically identifies and corrects geometric distortion problems in the image through a distortion detection and correction algorithm based on deep learning.

[0008] Optionally, the deep learning analysis module includes a target detection submodule, an object segmentation submodule and a scene classification submodule, wherein; The target detection submodule uses the YOLOv5 target detection model to identify geographic features in aerial images in real time, including roads, buildings, and vegetation; The object segmentation submodule uses the Mask R-CNN model to segment the target objects in the image and generate accurate target boundary information; The scene classification submodule performs scene classification based on a convolutional neural network to distinguish different types of geographical environments, such as cities, villages, forests, etc., so that the intelligent mapping module can generate corresponding maps according to different scenes.

[0009] Optionally, the image stitching module includes a feature point matching algorithm, a stitching optimization algorithm based on deep learning and a stitching image quality assessment unit, wherein; The feature point matching algorithm extracts and matches image feature points based on SURF or SIFT, automatically finds overlapping areas under different angles and lighting conditions during aerial photography, and performs image stitching; The deep learning-based stitching optimization algorithm combines the result of image feature point matching and optimizes the stitching accuracy through a convolutional neural network; The stitching image quality assessment unit performs quality assessment on the stitched image based on an image quality scoring model, and the assessment indicators include smoothness of the stitching, image clarity and color consistency.

[0010] Optionally, the intelligent mapping module includes a two-dimensional mapping submodule, a three-dimensional modeling submodule and a dynamic mapping optimization submodule, wherein; The two-dimensional mapping submodule generates a high-resolution two-dimensional map based on the spliced ​​aerial images; The 3D modeling submodule generates a high-precision 3D terrain model based on multi-view aerial images using sparse point cloud and dense point cloud reconstruction algorithms; The dynamic image optimization submodule dynamically adjusts the generated image results by combining the real-time flight environment data.

[0011] Optionally, the data storage and management module includes a distributed storage unit, a data compression unit and a data security management unit, wherein; The distributed storage unit performs distributed storage on the processed aerial image data based on a distributed file system; The data compression unit compresses and stores the image data by using a lossless compression algorithm based on deep learning; The data security management unit combines blockchain technology to encrypt and trace the stored data.

[0012] Beneficial effects of the present invention: The present invention uses deep learning technology to perform intelligent analysis of aerial images, including image denoising, target recognition, scene classification, etc., which significantly improves the accuracy and efficiency of image processing. The use of generative adversarial networks for image denoising can maintain the integrity of image details in complex lighting and noise environments. At the same time, through the YOLOv5 model for target detection and Mask R-CNN for object segmentation, the system can quickly identify and accurately classify different targets. The overall stitching accuracy is also greatly improved through the deep learning stitching optimization algorithm based on SURF / SIFT feature points, which solves the problem of stitching errors under different angles and lighting conditions, and is suitable for high-precision map generation under complex terrain.

[0013] The present invention uses distributed storage technology and blockchain technology to ensure that the processed image data is not lost or tampered with during storage and transmission. Through the lossless compression algorithm based on deep learning, the system can efficiently compress and store large-scale aerial photography data to ensure the security of long-term storage. The data encryption and traceability management functions ensure the safe application of the system in scenarios with high confidentiality requirements, avoiding potential risks caused by data leakage or tampering. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 Schematic diagram of a system flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0017] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0018] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0019] like Figure 1As shown, a real-time aerial image processing and intelligent mapping system based on deep learning includes an image acquisition module, a preprocessing module, a deep learning analysis module, an image stitching module, an intelligent mapping module and a data storage and management module, wherein; The image acquisition module is used to collect aerial images in real time through drones. The collection process includes the synchronous recording of environmental data. The image acquisition module supports the integration of multiple camera sensors to meet the needs of different scenarios. The preprocessing module is used to perform preliminary processing on the collected images, including image denoising, brightness balance, color correction and distortion correction. The image quality is improved through the image enhancement algorithm based on the convolutional neural network, providing high-quality image input for subsequent processing; The deep learning analysis module is used to perform deep learning analysis on preprocessed images. The analysis includes target recognition, scene classification and object segmentation functions. Based on a multi-layer convolutional neural network model, the module can automatically detect and classify different targets in aerial images and generate corresponding feature maps. The image stitching module is used to stitch multiple processed aerial images into a complete map. The stitching process is performed through an algorithm based on feature point matching and combined with the optimization adjustment of the deep learning network to ensure the generation of high-precision stitching images under different viewing angles and lighting conditions. The intelligent mapping module is used to generate the final map from the stitched images. The map includes high-resolution two-dimensional images and three-dimensional modeling results. The intelligent mapping module automatically generates corresponding image outputs according to different application requirements, and can achieve accurate matching between aerial images and actual geographic data by combining with the geographic information system. The data storage and management module is used to store and manage processed images and generated images. The data storage and management module supports real-time backup and compressed storage of images to ensure that data is not lost during processing and transmission, and has efficient image retrieval and call functions.

[0020] The image acquisition module is equipped with various types of camera devices, including infrared cameras, high-definition cameras and thermal imaging sensors. Different camera devices can be switched according to real-time environmental changes to adapt to different lighting and climate conditions. The image acquisition module includes an environmental sensor unit, which is used to synchronously record environmental data related to the aerial images, including temperature, humidity, wind speed and flight altitude, to assist in subsequent image analysis and mapping processing, and ensure the integrity and timeliness of the image data.

[0021] The preprocessing module includes an image denoising algorithm based on a generative adversarial network, an image brightness adaptive equalization algorithm, and a distortion correction unit, wherein; The image denoising algorithm based on generative adversarial network is used to reduce the noise in aerial images and maintain the integrity of image details; The image brightness adaptive equalization algorithm dynamically adjusts the brightness and contrast of the image according to the brightness distribution of the aerial image to ensure the clarity of the image under various lighting conditions; The distortion correction unit automatically identifies and corrects geometric distortion problems in images through a deep learning-based distortion detection and correction algorithm. It is particularly suitable for image distortion processing under different camera shooting angles.

[0022] The deep learning analysis module includes a target detection submodule, an object segmentation submodule, and a scene classification submodule, among which; The target detection submodule uses the YOLOv5 (You Only Look Once) target detection model to identify geographic features in aerial images in real time. Geographic features include roads, buildings, and vegetation. The recognition results serve as the basic data for subsequent stitching and mapping. The object segmentation submodule uses the Mask R-CNN (Region-based Convolutional Neural Networks) model to segment the target objects in the image, generate accurate target boundary information, and provide accurate geometric contours for image stitching and 3D modeling; The scene classification submodule performs scene classification based on convolutional neural networks to distinguish different types of geographical environments, such as cities, villages, forests, etc., so that the intelligent mapping module can generate corresponding maps according to different scenes.

[0023] The image stitching module includes a feature point matching algorithm, a stitching optimization algorithm based on deep learning, and a stitching image quality assessment unit, wherein; The feature point matching algorithm extracts and matches image feature points based on SURF (Speed ​​Up Robust Features) or SIFT (Scale-Invariant Feature Transform), automatically finds overlapping areas under different angles and lighting conditions during aerial photography, and performs image stitching; The stitching optimization algorithm based on deep learning combines the results of image feature point matching, optimizes the stitching accuracy through convolutional neural networks, and reduces the stitching error caused by differences in image shooting angles. It is particularly suitable for environments with complex terrain. The stitching image quality assessment unit performs quality assessment on the stitched image based on the image quality scoring model. The assessment indicators include the smoothness of the stitching, image clarity and color consistency.

[0024] The intelligent mapping module includes a 2D mapping submodule, a 3D modeling submodule and a dynamic mapping optimization submodule, among which; The 2D mapping submodule generates high-resolution 2D maps based on the stitched aerial images, which is particularly suitable for scenarios that require accurate geographic information, such as disaster monitoring and urban planning. The 3D modeling submodule generates high-precision 3D terrain models based on multi-view aerial images using sparse point cloud and dense point cloud reconstruction algorithms, which is suitable for accurate modeling of complex terrains. The dynamic imaging optimization submodule dynamically adjusts the generated image results by combining real-time flight environmental data, such as wind speed, flight altitude, lighting and other factors, to ensure the accuracy and consistency of imaging results generated under different conditions.

[0025] The data storage and management module includes a distributed storage unit, a data compression unit and a data security management unit, wherein; The distributed storage unit performs distributed storage of processed aerial image data based on a distributed file system to ensure data security and reliability, and is particularly suitable for long-term storage of large-scale aerial image data. The data compression unit compresses and stores image data through a lossless compression algorithm based on deep learning, reducing the pressure of data storage and transmission without affecting the image quality; The data security management unit combines blockchain technology to encrypt and trace the stored data, ensuring the data is tamper-proof and secure during storage and transmission. It is particularly suitable for scenarios with high requirements for data confidentiality.

[0026] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0027] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A real-time aerial image processing and intelligent mapping system based on deep learning, characterized in that: It includes image acquisition module, preprocessing module, deep learning analysis module, image stitching module, intelligent mapping module and data storage and management module, among which; The image acquisition module is used to collect aerial images in real time through the drone, and the collection process includes the synchronous recording of environmental data. The image acquisition module supports the integration of multiple camera sensors to meet the needs of different scenarios; The preprocessing module is used to perform preliminary processing on the collected images, and the preliminary processing includes image denoising, brightness balancing, color correction and distortion correction, and improves the image quality through an image enhancement algorithm based on a convolutional neural network; The deep learning analysis module is used to perform deep learning analysis on the preprocessed images, and the analysis includes target recognition, scene classification and object segmentation functions. Based on a multi-layer convolutional neural network model, the module can automatically detect and classify different targets in the aerial images and generate corresponding feature maps; The image stitching module is used to stitch multiple processed aerial images into a complete map, and the stitching process is performed by an algorithm based on feature point matching and combined with optimization adjustment of a deep learning network; The intelligent mapping module is used to generate the final map from the stitched images, and the map includes a two-dimensional image and a three-dimensional modeling result. The intelligent mapping module automatically generates corresponding image outputs according to different application requirements, and can achieve accurate matching between aerial images and actual geographic data by combining with a geographic information system; The data storage and management module is used to store and manage the processed images and generated images, and the data storage and management module supports real-time backup and compressed storage of images.

2. According to claim 1, a real-time aerial image processing and intelligent mapping system based on deep learning is characterized in that: The image acquisition module is equipped with various types of camera equipment, including infrared cameras, high-definition cameras and thermal imaging sensors, and switches different camera equipment according to real-time environmental changes to adapt to different lighting and climate conditions; The image acquisition module includes an environmental sensor unit for synchronously recording environmental data related to the aerial image, including temperature, humidity, wind speed and flight altitude.

3. The real-time aerial image processing and intelligent mapping system based on deep learning according to claim 1, characterized in that: The preprocessing module includes an image denoising algorithm based on a generative adversarial network, an image brightness adaptive equalization algorithm and a distortion correction unit, wherein; The image denoising algorithm based on the generative adversarial network is used to reduce the noise in the aerial image and maintain the integrity of the image details; The image brightness adaptive equalization algorithm dynamically adjusts the brightness and contrast of the image according to the brightness distribution of the aerial image; The distortion correction unit automatically identifies and corrects geometric distortion problems in the image through a distortion detection and correction algorithm based on deep learning.

4. The real-time aerial image processing and intelligent mapping system based on deep learning according to claim 1, characterized in that: The deep learning analysis module includes a target detection submodule, an object segmentation submodule and a scene classification submodule, wherein; The target detection submodule uses the YOLOv5 target detection model to identify geographic features in aerial images in real time, including roads, buildings, and vegetation; The object segmentation submodule uses the Mask R-CNN model to segment the target object in the image and generate target boundary information; The scene classification submodule performs scene classification based on a convolutional neural network to distinguish different types of geographical environments.

5. The real-time aerial image processing and intelligent mapping system based on deep learning according to claim 1, characterized in that: The image stitching module includes a feature point matching algorithm, a stitching optimization algorithm based on deep learning and a stitching image quality assessment unit, wherein; The feature point matching algorithm extracts and matches image feature points based on SURF or SIFT, automatically finds overlapping areas under different angles and lighting conditions during aerial photography, and performs image stitching; The deep learning-based stitching optimization algorithm combines the result of image feature point matching and optimizes the stitching accuracy through a convolutional neural network; The stitching image quality assessment unit performs quality assessment on the stitched image based on an image quality scoring model, and the assessment indicators include smoothness of the stitching, image clarity and color consistency.

6. The real-time aerial image processing and intelligent mapping system based on deep learning according to claim 1, characterized in that: The intelligent mapping module includes a two-dimensional mapping submodule, a three-dimensional modeling submodule and a dynamic mapping optimization submodule, wherein; The two-dimensional mapping submodule generates a two-dimensional map based on the spliced ​​aerial images; The 3D modeling submodule generates a 3D terrain model based on multi-view aerial images using sparse point cloud and dense point cloud reconstruction algorithms; The dynamic image optimization submodule dynamically adjusts the generated image results by combining the real-time flight environment data.

7. The real-time aerial image processing and intelligent mapping system based on deep learning according to claim 1, characterized in that: The data storage and management module includes a distributed storage unit, a data compression unit and a data security management unit, wherein; The distributed storage unit performs distributed storage on the processed aerial image data based on a distributed file system; The data compression unit compresses and stores the image data by using a lossless compression algorithm based on deep learning; The data security management unit combines blockchain technology to encrypt and trace the stored data.

Images