An unmanned aerial vehicle aerial image enhancement processing method and system

Abstract

The application discloses an unmanned aerial vehicle aerial photograph image enhancement processing method and system, relates to the technical field of image processing, and comprises the following steps: constructing a three-dimensional regional map model of a target aerial photograph region, determining a target aerial photograph route and a target aerial photograph state of an unmanned aerial vehicle device; controlling the unmanned aerial vehicle device to collect images of the target aerial photograph region based on the target aerial photograph route and the target aerial photograph state, and generating initial sub-aerial photograph image data; performing geometric correction processing on the initial sub-aerial photograph image data to obtain first sub-aerial photograph image data; performing denoising processing on the first sub-aerial photograph image data to obtain second sub-aerial photograph image data; performing image enhancement processing on the second sub-aerial photograph image data to obtain target sub-aerial photograph image data; and performing fusion processing on a plurality of target sub-aerial photograph image data to obtain enhanced aerial photograph image data, so that the aerial photograph route and the state of the unmanned aerial vehicle can be optimized, and the aerial photograph image can be corrected, denoised, enhanced and fused, and the quality and the definition of the aerial photograph image are significantly improved.

Description

Technical Field

[0001] This invention relates to a method and system for enhancing aerial images taken by unmanned aerial vehicles (UAVs), belonging to the field of image processing technology. Background Technology

[0002] Drone aerial photography technology has been widely used in various fields, but due to limitations in drone flight conditions, camera parameters, and environmental factors, the aerial images obtained often suffer from problems such as insufficient resolution, lack of clarity, and geometric distortion.

[0003] While existing aerial image processing methods have improved image quality to some extent, they cannot effectively handle image data under complex environmental conditions. Furthermore, these methods are often complex to operate, inefficient, and have limited applicability.

[0004] Therefore, it is necessary to provide a method and system for enhancing drone aerial images to solve the above-mentioned technical problems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and system for enhancing aerial images captured by unmanned aerial vehicles (UAVs). This addresses the limitations of existing UAV aerial photography technologies, which are constrained by UAV flight conditions, camera parameters, and environmental factors, resulting in aerial images with insufficient resolution, inadequate clarity, and geometric distortion. Furthermore, existing aerial image processing methods are unable to effectively handle image data under complex environmental conditions, and are characterized by complex operation, low processing efficiency, and limited applicability.

[0006] This invention provides a method for enhancing aerial images captured by unmanned aerial vehicles (UAVs), the enhancement method comprising:

[0007] Construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model;

[0008] Based on the target aerial photography route and target aerial photography status, the UAV equipment is controlled to acquire images of the target aerial photography area and generate initial sub-aerial photography image data.

[0009] The initial sub-aerial image data is subjected to geometric correction processing to obtain the corresponding first sub-aerial image data;

[0010] The first sub-aerial image data is denoised to obtain the corresponding second sub-aerial image data;

[0011] The second sub-aerial image data is subjected to image enhancement processing to obtain the corresponding target sub-aerial image data;

[0012] Multiple target sub-aerial image data are fused to obtain corresponding enhanced aerial image data.

[0013] Preferably, the construction of the three-dimensional regional map model of the target aerial photography area specifically includes:

[0014] A preset scanning position is obtained, the drone is controlled to fly to the preset scanning position, and the target aerial photography area is scanned and identified by the area vision sensor on the drone, generating three-dimensional area scanning data of the target aerial photography area;

[0015] The three-dimensional region scanning data is processed by point cloud registration, and a three-dimensional region map model is constructed based on the three-dimensional region scanning data after point cloud registration, using a surface reconstruction algorithm.

[0016] Preferably, determining the target aerial photography route of the UAV equipment based on the three-dimensional regional map model specifically includes:

[0017] Based on the three-dimensional regional map model, dynamic path planning is performed on the UAV equipment, and the starting point and ending point of the aerial photography route of the UAV equipment are set.

[0018] Based on the shortest path algorithm, the estimated shortest route between the map nodes in the 3D regional map model and the starting point of the aerial photography route is iteratively updated. The corresponding calculation formula is as follows:

[0019]

[0020] In the formula, dist[v] represents the shortest route estimate between map node v and the starting point of the aerial photography route in the 3D regional map model; F represents the set of all aerial photography routes of the UAV equipment; dist[u] represents the shortest route estimate between the predecessor node u corresponding to map node v and the starting point of the aerial photography route; w(u,v) represents the weight of the route formed by map node v and its corresponding predecessor node u; min represents the minimum value operation.

[0021] When the map node is updated to the end point of the aerial photography route, the iterative update process of the shortest route estimate is stopped, and the target aerial photography route of the UAV equipment is obtained.

[0022] Preferably, determining the target aerial photography status of the UAV equipment based on the three-dimensional regional map model specifically includes:

[0023] The system monitors the preceding aerial photography status of the UAV and, based on the state transition equation, calculates the optimal solution for the current aerial photography status of the UAV according to the preceding flight status. The corresponding calculation formula is as follows:

[0024]

[0025] In the formula, dp[i] represents the optimal solution of the current aerial photography state i of the UAV; dp[j] represents the optimal solution of the preceding aerial photography state j corresponding to the current aerial photography state i of the UAV; cost(j,i) represents the cost of the UAV to switch from the preceding aerial photography state j to the current aerial photography state i; Q(i) represents the set of preceding aerial photography states corresponding to the current aerial photography state i of the UAV; min represents the minimum value operation.

[0026] The target aerial photography state is determined based on the optimal solution of the current aerial photography state of the drone equipment.

[0027] Preferably, the step of performing geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data specifically includes:

[0028] The original pixel positions of the initial sub-aerial image data are transformed based on perspective transformation technology, and the corresponding calculation formula is as follows:

[0029]

[0030] In the formula, (x ′ ,y ′ ) represents the pixel coordinates of the first sub-aerial image data; w ′ Indicates the projection parameters of the perspective transformation; The coefficient matrix represents the perspective transformation; (x,y) represents the original pixel coordinates of the initial sub-aerial image data;

[0031] The non-integer coordinates in the pixel coordinates of the first sub-aerial image data are determined, and the pixel values ​​at the non-integer coordinates are updated based on the bilinear interpolation algorithm. The corresponding calculation formula is as follows:

[0032]

[0033] In the formula, f(x″,y″) represents the updated pixel value at the non-integer coordinates; (x1,y1), (x2,y1), (x1,y2), and (x2,y2) represent the original pixel coordinates around the non-integer coordinates; and f(x1,y1), f(x2,y1), f(x1,y2), and f(x2,y2) represent the pixel values ​​at the original pixel coordinates around the non-integer coordinates.

[0034] Preferably, the step of denoising the first sub-aerial image data to obtain the corresponding second sub-aerial image data specifically includes:

[0035] The first sub-aerial image data is filtered and denoised using a mean filtering algorithm to obtain the second sub-aerial image data. The corresponding calculation formula is as follows:

[0036]

[0037] In the formula, I ′ (x ′ ,y ′ (x) represents the image grayscale value of the second sub-aerial image data after filtering and denoising; ′ ,y ′ ) represents the pixel coordinates of the first sub-aerial image data; S x′yv Represents the pixel coordinates (x) of the first sub-aerial image data ′ ,y ′ The set of gray values ​​in the neighborhood of ); |S x′y′ | represents the set of gray values ​​in the neighborhood S x′y′ The number of elements in; S represents the set of gray values ​​in the neighborhood. x′u′ The sum of the grayscale values ​​of all pixels within the range.

[0038] Preferably, the step of performing image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data specifically includes:

[0039] Color correction processing is performed on the second sub-aerial image data based on linear stretching technology;

[0040] The target sub-aerial image data is obtained by sharpening the second sub-aerial image data after color correction based on the Laplacian sharpening algorithm.

[0041] Preferably, the step of fusing multiple target sub-aerial image data to obtain corresponding enhanced aerial image data specifically includes:

[0042] The multiple target sub-aerial image data are classified to obtain sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image;

[0043] Image fusion processing is performed on the sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image to obtain the enhanced aerial image data. The corresponding calculation formula is as follows:

[0044] ZQTX=QJTX*TMD+BJTX*(1-TMD)

[0045] In the formula, ZQTX represents enhanced aerial image data; QJTX represents sub-aerial foreground image; TMD represents sub-aerial transparency image; and BJTX represents sub-aerial background image.

[0046] An enhancement processing system for drone aerial images, the enhancement processing system comprising:

[0047] The aerial photography route status module is used to construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model.

[0048] The image data acquisition module is used to control the UAV equipment to acquire images of the target aerial photography area based on the target aerial photography route and target aerial photography status, and generate initial sub-aerial photography image data;

[0049] A geometric correction processing module is used to perform geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data;

[0050] The image denoising processing module is used to denoise the first sub-aerial image data to obtain the corresponding second sub-aerial image data.

[0051] The image enhancement processing module is used to perform image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data;

[0052] The fusion enhancement processing module is used to fuse multiple target sub-aerial image data to obtain corresponding enhanced aerial image data.

[0053] Compared with related technologies, the UAV aerial image enhancement method and system provided by this invention have the following advantages:

[0054] Beneficial effects:

[0055] This invention constructs a 3D regional map model of the target aerial photography area and determines the target aerial photography route and status of the UAV based on the 3D regional map model. Based on the target aerial photography route and status, the UAV is controlled to acquire images of the target aerial photography area, generating initial sub-aerial image data. Geometric correction processing is performed on the initial sub-aerial image data to obtain corresponding first sub-aerial image data. Noise reduction processing is performed on the first sub-aerial image data to obtain corresponding second sub-aerial image data. Image enhancement processing is performed on the second sub-aerial image data to obtain corresponding target sub-aerial image data. Multiple target sub-aerial image data are fused to obtain corresponding enhanced aerial image data. This allows for optimization of the UAV's aerial photography route and status, and performs correction, noise reduction, enhancement, and fusion processing on the acquired aerial images, thereby significantly improving the quality and clarity of the aerial images.

[0056] This invention significantly improves the image acquisition efficiency of drones by optimizing their aerial photography routes and states. Geometric correction of aerial images eliminates geometric distortions caused by attitude changes during drone flight, ensuring the authenticity and accuracy of the images. Mean filtering enhances image clarity and readability. Linear stretching corrects image colors, ensuring color accuracy and consistency. Laplacian sharpening enhances image details, improving resolution and clarity. Finally, fusion processing of multiple image data integrates valuable information from various images, significantly increasing the clarity and information content of aerial images. Attached Figure Description

[0057] Figure 1 This is a flowchart of a method for enhancing aerial images taken by a drone, provided in an embodiment of the present invention.

[0058] Figure 2 This is a system block diagram of an aerial image enhancement processing system provided in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] like Figure 1 The diagram shown is a flowchart of a method for enhancing aerial images taken by a drone, provided in an embodiment of the present invention. Figure 1The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. Steps S1 to S6 are detailed as follows:

[0062] S1, construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model;

[0063] In practical applications, to achieve efficient and accurate airspace information acquisition, a three-dimensional regional map model of the target aerial photography area can be constructed first. Specifically, by utilizing Geographic Information System (GIS), Global Positioning System (GPS), high-resolution remote sensing imagery technology, and advanced algorithms such as stereo image pair matching and digital elevation model (DEM) generation, a precise three-dimensional spatial model of the target aerial photography area can be constructed.

[0064] Then, based on this three-dimensional regional map model, the shortest path algorithm and state transition equation can be used to determine the target aerial photography route and target aerial photography status when the UAV equipment performs aerial photography mission.

[0065] S2, based on the target aerial photography route and target aerial photography status, control the UAV equipment to acquire images of the target aerial photography area and generate initial sub-aerial photography image data;

[0066] Among them, the drone equipment can be controlled to automatically perform image acquisition tasks according to the target aerial photography route and status, generate initial sub-aerial photography image data, and these data contain multi-angle and multi-scale views of the target aerial photography area.

[0067] By using the above methods, it can be ensured that the drone equipment covers the entire target aerial photography area with the optimal path, while ensuring the rationality of parameters such as shooting angle, altitude, and speed, thereby maximizing the quality and efficiency of image acquisition.

[0068] S3, Perform geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data;

[0069] Understandably, perspective transformation techniques and bilinear interpolation algorithms can be used to perform geometric correction processing on the acquired initial sub-aerial image data, thereby eliminating image geometric distortion caused by factors such as drone attitude changes and lens distortion, and obtaining accurately aligned first sub-aerial image data.

[0070] S4, Denoise the first sub-aerial image data to obtain the corresponding second sub-aerial image data;

[0071] Among them, the mean filtering algorithm can be used to effectively suppress random noise and sensor noise in the image, improve the clarity and signal-to-noise ratio of the aerial image, and generate the corresponding second sub-aerial image data.

[0072] S5, perform image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data;

[0073] It should be noted that, in order to further enhance the visualization effect of the aerial images, image enhancement processing can be performed on the second sub-aerial image data. The aerial images are color corrected by linear stretching technology, and the Laplacian sharpening algorithm is used to sharpen the color corrected images, thereby improving the contrast, brightness and detail of the images, and obtaining the target sub-aerial image data.

[0074] S6, perform fusion processing on multiple target sub-aerial image data to obtain corresponding enhanced aerial image data.

[0075] In practical applications, in order to integrate all target sub-aerial image data and form comprehensive and high-quality enhanced aerial image data, image fusion processing can be performed on multiple target sub-aerial image data.

[0076] The above methods can effectively integrate image information from multiple perspectives, different times, or different spectral bands, eliminate redundant information in overlapping areas of images, and enhance the texture details, color richness, and spatial coherence of images.

[0077] In the specific implementation process, the construction of a three-dimensional regional map model of the target aerial photography area specifically includes:

[0078] A preset scanning position is obtained, the drone is controlled to fly to the preset scanning position, and the target aerial photography area is scanned and identified by the area vision sensor on the drone, generating three-dimensional area scanning data of the target aerial photography area;

[0079] The three-dimensional region scanning data is processed by point cloud registration, and a three-dimensional region map model is constructed based on the three-dimensional region scanning data after point cloud registration, using a surface reconstruction algorithm.

[0080] First, the system can acquire preset scanning positions for the drone equipment. These positions, based on prior planning and analysis, can comprehensively cover key feature points of the target aerial photography area. Then, the drone equipment can be precisely controlled to fly along a predetermined route to the preset scanning positions.

[0081] During this process, high-precision regional vision sensors, such as LiDAR and stereo cameras, mounted on the drone equipment can perform scanning and identification tasks on the target aerial photography area, thereby capturing key information such as the surface texture, height changes and obstacle distribution of the area, and generating high-precision three-dimensional regional scanning data.

[0082] Furthermore, point cloud registration processing can be performed on the collected 3D region scan data, and the point cloud data obtained from different perspectives can be precisely aligned to eliminate geometric errors caused by perspective changes and ensure data consistency and integrity.

[0083] Finally, surface reconstruction algorithms can be used to further process the 3D region scan data and construct a 3D region map model that conforms to the actual terrain features.

[0084] It should be noted that this 3D regional map model not only includes the 3D geometry of the target aerial photography area, but also reflects the subtle textures and structural features of the regional surface.

[0085] Determining the target aerial photography route of the UAV based on the aforementioned 3D regional map model specifically includes:

[0086] Based on the three-dimensional regional map model, dynamic path planning is performed on the UAV equipment, and the starting point and ending point of the aerial photography route of the UAV equipment are set.

[0087] Based on the shortest path algorithm, the estimated shortest route between the map nodes in the 3D regional map model and the starting point of the aerial photography route is iteratively updated. The corresponding calculation formula is as follows:

[0088]

[0089] In the formula, dist[v] represents the shortest route estimate between map node v and the starting point of the aerial photography route in the 3D regional map model; F represents the set of all aerial photography routes of the UAV equipment; dist[u] represents the shortest route estimate between the predecessor node u corresponding to map node v and the starting point of the aerial photography route; w(u,v) represents the weight of the route formed by map node v and its corresponding predecessor node u; min represents the minimum value operation.

[0090] When the map node is updated to the end point of the aerial photography route, the iterative update process of the shortest route estimate is stopped, and the target aerial photography route of the UAV equipment is obtained.

[0091] Determining the target aerial photography status of the UAV equipment based on the aforementioned 3D regional map model specifically includes:

[0092] The system monitors the preceding aerial photography status of the UAV and, based on the state transition equation, calculates the optimal solution for the current aerial photography status of the UAV according to the preceding flight status. The corresponding calculation formula is as follows:

[0093]

[0094] In the formula, dp[i] represents the optimal solution of the current aerial photography state i of the UAV; dp[j] represents the optimal solution of the preceding aerial photography state j corresponding to the current aerial photography state i of the UAV; cost(j,i) represents the cost of the UAV to switch from the preceding aerial photography state j to the current aerial photography state i; Q(i) represents the set of preceding aerial photography states corresponding to the current aerial photography state i of the UAV; min represents the minimum value operation.

[0095] The target aerial photography state is determined based on the optimal solution of the current aerial photography state of the drone equipment.

[0096] In determining the target aerial photography route for drone equipment, dynamic path planning can be performed on the drone equipment based on a precisely constructed 3D regional map model, ensuring that the drone equipment can complete the aerial photography task efficiently and safely.

[0097] Specifically, the first step is to set the start and end points for the drone aerial photography. The start point is usually determined based on mission requirements, such as the center of the area to be photographed or a specific target location; the end point may be the edge of the photographed area or the coordinates for returning to base. After setting the start and end points, the optimal aerial photography route for the drone can be determined using a shortest path algorithm.

[0098] In a 3D regional map model, map nodes represent the geographical locations that a drone might traverse, while the connections between nodes and their weights reflect factors such as distance, altitude changes, and obstacle avoidance during the drone's flight. By iteratively updating the shortest route estimate from each map node to the starting point of the aerial photography route, the global optimum can be gradually approximated.

[0099] When the iterative process updates the map nodes to the end point of the aerial photography route, the shortest path from the start point to the end point of the aerial photography route can be determined, which is the target aerial photography route of the drone equipment.

[0100] In practical applications, the aerial photography status of the drone's forward flight can be monitored in real time. Then, the optimal solution corresponding to the current aerial photography status can be calculated through iterative calculation using the state transition equation, thereby determining the target aerial photography status and ensuring that the drone achieves the best shooting effect at the lowest cost throughout the entire flight.

[0101] The step of performing geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data specifically includes:

[0102] The original pixel positions of the initial sub-aerial image data are transformed based on perspective transformation technology, and the corresponding calculation formula is as follows:

[0103]

[0104] In the formula, (x ′ ,y ′ ) represents the pixel coordinates of the first sub-aerial image data; w ′ Indicates the projection parameters of the perspective transformation; The coefficient matrix represents the perspective transformation; (x,y) represents the original pixel coordinates of the initial sub-aerial image data;

[0105] The non-integer coordinates in the pixel coordinates of the first sub-aerial image data are determined, and the pixel values ​​at the non-integer coordinates are updated based on the bilinear interpolation algorithm. The corresponding calculation formula is as follows:

[0106]

[0107] In the formula, f(x″,y″) represents the updated pixel value at the non-integer coordinates; (x1,y1), (x2,y1), (x1,y2), and (x2,y2) represent the original pixel coordinates around the non-integer coordinates; and f(x1,y1), f(x2,y1), f(x1,y2), and f(x2,y2) represent the pixel values ​​at the original pixel coordinates around the non-integer coordinates.

[0108] Understandably, perspective transformation technology can be used to accurately transform the original pixel coordinates of the initial sub-aerial image data to obtain the pixel coordinates of the first sub-aerial image data.

[0109] It should be noted that for non-integer coordinates that may occur after perspective transformation, bilinear interpolation can be used to update pixel values, thereby ensuring the integrity and accuracy of the first sub-aerial image data.

[0110] The step of denoising the first sub-aerial image data to obtain the corresponding second sub-aerial image data specifically includes:

[0111] The first sub-aerial image data is filtered and denoised using a mean filtering algorithm to obtain the second sub-aerial image data. The corresponding calculation formula is as follows:

[0112]

[0113] In the formula, I ′ (x ′ ,y ′ (x) represents the image grayscale value of the second sub-aerial image data after filtering and denoising; ′ ,y ′ ) represents the pixel coordinates of the first sub-aerial image data; S x′y′ Represents the pixel coordinates (x) of the first sub-aerial image data ′ ,y ′ The set of gray values ​​in the neighborhood of ); |S x′y′ | represents the set of gray values ​​in the neighborhood S x′y′ The number of elements in; S represents the set of gray values ​​in the neighborhood. x′y′ The sum of the grayscale values ​​of all pixels within the range.

[0114] In practical applications, the first sub-aerial image data can be denoised to obtain a clearer second sub-aerial image data with less noise.

[0115] Mean filtering algorithms can effectively reduce random noise interference in images while preserving their basic features through image smoothing techniques. Specifically, this algorithm defines a specific neighborhood range around each pixel in the first sub-aerial image data, calculates the average grayscale value of all pixels within that neighborhood, and uses this average value as the new grayscale value of the corresponding pixel after filtering and denoising, thereby generating the second sub-aerial image data.

[0116] The step of performing image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data specifically includes:

[0117] Color correction processing is performed on the second sub-aerial image data based on linear stretching technology;

[0118] The target sub-aerial image data is obtained by sharpening the second sub-aerial image data after color correction based on the Laplacian sharpening algorithm.

[0119] Understandably, linear stretching can be used first to perform color correction on the second sub-aerial image data. Linear stretching stretches the grayscale range of the image by performing a linear transformation on the image's grayscale values, which can diffuse the originally concentrated distribution of grayscale values, thereby enhancing the image's contrast.

[0120] This process can significantly improve the overall visual effect of the second sub-aerial image data, highlighting details in both bright and dark areas of the image.

[0121] Furthermore, the Laplacian sharpening algorithm can be used to sharpen the color-corrected second sub-aerial image data. As a second-order differential operator, the Laplacian operator can accurately capture edge and detail information in an image. By calculating the second derivative of the image, the Laplacian sharpening algorithm can enhance the high-frequency components in the image, namely edge and texture information, making image edges clearer and details more prominent.

[0122] The above methods can further improve image clarity and effectively enhance spatial resolution, enabling the target sub-aerial image data to achieve optimal visual effects.

[0123] The process of fusing multiple target sub-aerial image data to obtain corresponding enhanced aerial image data specifically includes:

[0124] The multiple target sub-aerial image data are classified to obtain sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image;

[0125] Image fusion processing is performed on the sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image to obtain the enhanced aerial image data. The corresponding calculation formula is as follows:

[0126] ZQTX=QJTX*TMD+BJTX*(1-TMD)

[0127] In the formula, ZQTX represents enhanced aerial image data; QJTX represents sub-aerial foreground image; TMD represents sub-aerial transparency image; and BJTX represents sub-aerial background image.

[0128] In practical applications, multiple target sub-aerial image data can first be classified and processed according to the characteristics of the image content, and then divided into sub-aerial foreground images, sub-aerial background images, and sub-aerial transparency images.

[0129] It should be noted that the sub-aerial foreground image mainly includes the most prominent and attention-grabbing objects or areas in the aerial scene, such as buildings, vehicles, or people; the sub-aerial background image presents the overall environment or distant view in the aerial scene, such as the sky, the ground, or the water surface; and the sub-aerial transparency image reflects the transparency information of the foreground and background images during the fusion process, that is, the visibility of the foreground image in the final fused image, which helps to achieve natural and realistic image fusion.

[0130] Furthermore, these categorized image data can be fused to combine the sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image to generate enhanced aerial image data.

[0131] Understandably, the enhanced aerial image data after fusion is determined by multiple components. The foreground image retains key information about the aerial scene, the background image provides the scene environment, and the transparency image ensures a natural transition between the foreground and background during the fusion process.

[0132] The above fusion process can improve the overall visual effect of aerial images and enhance the information content and sense of depth of the images.

[0133] like Figure 2 The diagram shown is a system block diagram of an aerial image enhancement system provided in an embodiment of the present invention. The enhancement system includes:

[0134] The aerial photography route status module is used to construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model.

[0135] The image data acquisition module is used to control the UAV equipment to acquire images of the target aerial photography area based on the target aerial photography route and target aerial photography status, and generate initial sub-aerial photography image data;

[0136] A geometric correction processing module is used to perform geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data;

[0137] The image denoising processing module is used to denoise the first sub-aerial image data to obtain the corresponding second sub-aerial image data.

[0138] The image enhancement processing module is used to perform image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data;

[0139] The fusion enhancement processing module is used to fuse multiple target sub-aerial image data to obtain corresponding enhanced aerial image data.

[0140] Figure 2 The apparatus of the illustrated embodiment can be used to perform corresponding actions. Figure 1 The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0141] See Figure 3This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 30 includes: a processor 31, a memory 32, and a computer program; wherein...

[0142] The memory 32 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0143] Processor 31 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0144] Alternatively, the memory 32 can be either standalone or integrated with the processor 31.

[0145] When the memory 32 is a device independent of the processor 31, the device may further include:

[0146] Bus 33 is used to connect the memory 32 and the processor 31.

[0147] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0148] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0149] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0150] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0151] Through the above embodiments, the present invention provides a method and system for enhancing UAV aerial images. This involves constructing a three-dimensional regional map model of the target aerial photography area, and determining the target aerial photography route and status of the UAV based on the model. Based on the target aerial photography route and status, the UAV is controlled to acquire images of the target area, generating initial sub-aerial image data. Geometric correction is applied to the initial sub-aerial image data to obtain corresponding first sub-aerial image data. Noise reduction is applied to the first sub-aerial image data to obtain corresponding second sub-aerial image data. Image enhancement is applied to the second sub-aerial image data to obtain corresponding target sub-aerial image data. Multiple target sub-aerial image data are then fused to obtain corresponding enhanced aerial image data. This process optimizes the UAV's aerial photography route and status, and performs correction, noise reduction, enhancement, and fusion processing on the acquired aerial images, thereby significantly improving the quality and clarity of the aerial images.

[0152] This invention significantly improves the image acquisition efficiency of drones by optimizing their aerial photography routes and states. Geometric correction of aerial images eliminates geometric distortions caused by attitude changes during drone flight, ensuring the authenticity and accuracy of the images. Mean filtering enhances image clarity and readability. Linear stretching corrects image colors, ensuring color accuracy and consistency. Laplacian sharpening enhances image details, improving resolution and clarity. Finally, fusion processing of multiple image data integrates valuable information from various images, significantly increasing the clarity and information content of aerial images.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing aerial images captured by unmanned aerial vehicles, characterized in that, The enhancement processing method includes: Construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model; Based on the target aerial photography route and target aerial photography status, the UAV equipment is controlled to acquire images of the target aerial photography area and generate initial sub-aerial photography image data. The initial sub-aerial image data is subjected to geometric correction processing to obtain the corresponding first sub-aerial image data; The first sub-aerial image data is denoised to obtain the corresponding second sub-aerial image data; The second sub-aerial image data is subjected to image enhancement processing to obtain the corresponding target sub-aerial image data; The aerial image data of multiple targets are fused to obtain the corresponding enhanced aerial image data. Specifically, determining the target aerial photography status of the UAV equipment based on the three-dimensional regional map model includes: The system monitors the preceding aerial photography status of the UAV and, based on the state transition equation, calculates the optimal solution for the current aerial photography status of the UAV. The corresponding calculation formula is as follows: In the formula, Indicates the current aerial photography status of the drone equipment. The optimal solution; Indicates the current aerial photography status of the drone equipment. Corresponding front-wheel drive aerial photography status The optimal solution; This indicates that the drone equipment has transitioned from the forward-launch aerial photography state. Switch to current aerial photography status The cost; Indicates the current aerial photography status of the drone equipment. The corresponding set of aerial photography statuses for the front-end; This indicates the operation of finding the minimum value; The target aerial photography state is determined based on the optimal solution of the current aerial photography state of the drone equipment.

2. The method for enhancing UAV aerial images according to claim 1, characterized in that, The construction of the three-dimensional regional map model of the target aerial photography area specifically includes: A preset scanning position is obtained, the drone is controlled to fly to the preset scanning position, and the target aerial photography area is scanned and identified by the area vision sensor on the drone, generating three-dimensional area scanning data of the target aerial photography area; The three-dimensional region scanning data is processed by point cloud registration, and a three-dimensional region map model is constructed based on the three-dimensional region scanning data after point cloud registration, using a surface reconstruction algorithm.

3. The method for enhancing UAV aerial images according to claim 2, characterized in that, Determining the target aerial photography route of the UAV based on the aforementioned 3D regional map model specifically includes: Based on the three-dimensional regional map model, dynamic path planning is performed on the UAV equipment, and the starting point and ending point of the aerial photography route of the UAV equipment are set. Based on the shortest path algorithm, the estimated shortest route between the map nodes in the 3D regional map model and the starting point of the aerial photography route is iteratively updated. The corresponding calculation formula is as follows: In the formula, Represents map nodes in a 3D regional map model The shortest route estimate from the starting point of the aerial photography route; This represents the set of all aerial photography routes taken by the drone equipment. Represents map nodes corresponding predecessor node The shortest route estimate from the starting point of the aerial photography route; Represents map nodes With the corresponding predecessor node The weights of the routes; This indicates the operation of finding the minimum value; When the map node is updated to the end point of the aerial photography route, the iterative update process of the shortest route estimate is stopped, and the target aerial photography route of the UAV equipment is obtained.

4. The method for enhancing UAV aerial images according to claim 1, characterized in that, The step of performing geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data specifically includes: The original pixel positions of the initial sub-aerial image data are transformed based on perspective transformation technology, and the corresponding calculation formula is as follows: In the formula, Represents the pixel coordinates of the first sub-aerial image data; Indicates the projection parameters of the perspective transformation; The coefficient matrix representing the perspective transformation; Represents the original pixel coordinates of the initial sub-aerial image data; The non-integer coordinates in the pixel coordinates of the first sub-aerial image data are determined, and the pixel values ​​at the non-integer coordinates are updated based on the bilinear interpolation algorithm. The corresponding calculation formula is as follows: In the formula, This represents the pixel value at the updated non-integer coordinates; , , , Represents the original pixel coordinates surrounding the non-integer coordinates; , , , Represents the pixel value at the original pixel coordinates surrounding the non-integer coordinates.

5. The method for enhancing UAV aerial images according to claim 1, characterized in that, The step of denoising the first sub-aerial image data to obtain the corresponding second sub-aerial image data specifically includes: The first sub-aerial image data is filtered and denoised using a mean filtering algorithm to obtain the second sub-aerial image data. The corresponding calculation formula is as follows: In the formula, This represents the image grayscale value of the second sub-aerial image data after filtering and denoising. Represents the pixel coordinates of the first sub-aerial image data; Represents the pixel coordinates of the first sub-aerial image data The set of gray values ​​in the neighborhood; Represents the set of gray values ​​in the neighborhood The number of elements in; Represents the set of gray values ​​in the neighborhood The sum of the grayscale values ​​of all pixels within the range.

6. The method for enhancing UAV aerial images according to claim 1, characterized in that, The step of performing image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data specifically includes: Color correction processing is performed on the second sub-aerial image data based on linear stretching technology; The target sub-aerial image data is obtained by sharpening the second sub-aerial image data after color correction based on the Laplacian sharpening algorithm.

7. The method for enhancing UAV aerial images according to claim 1, characterized in that, The process of fusing multiple target sub-aerial image data to obtain corresponding enhanced aerial image data specifically includes: The multiple target sub-aerial image data are classified to obtain sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image; The sub-aerial foreground image, sub-aerial background image, and sub-aerial transparency image are subjected to image fusion processing to obtain the enhanced aerial image data. The corresponding calculation formula is as follows: In the formula, This indicates enhanced aerial image data; This represents a foreground image captured by an aerial photograph. This represents a sub-aerial image with transparency. This represents the background image captured by the drone.

8. A system for enhancing aerial images captured by unmanned aerial vehicles, characterized in that, The system employs an enhancement processing method for UAV aerial images as described in any one of claims 1-7, wherein the enhancement processing system comprises: The aerial photography route status module is used to construct a three-dimensional regional map model of the target aerial photography area, and determine the target aerial photography route and target aerial photography status of the UAV equipment based on the three-dimensional regional map model. The image data acquisition module is used to control the UAV equipment to acquire images of the target aerial photography area based on the target aerial photography route and target aerial photography status, and generate initial sub-aerial photography image data; A geometric correction processing module is used to perform geometric correction processing on the initial sub-aerial image data to obtain the corresponding first sub-aerial image data; The image denoising processing module is used to denoise the first sub-aerial image data to obtain the corresponding second sub-aerial image data. The image enhancement processing module is used to perform image enhancement processing on the second sub-aerial image data to obtain the corresponding target sub-aerial image data; The fusion enhancement processing module is used to fuse multiple target sub-aerial image data to obtain corresponding enhanced aerial image data.

Images