An Unmanned Aerial Vehicle Low-Altitude Photography Image Enhancement Method and System

Abstract

The present invention discloses a method and system for enhancing low-altitude photography images of unmanned aerial vehicles. The method includes data preparation, motion joint deblurring optimization, multi-source data registration, scene adaptive enhancement, and image comprehensive enhancement. The present invention relates to the technical field of unmanned aerial vehicle image enhancement, and integrates a blur kernel prediction guided by an inertial measurement unit, a cross-modal graph structure matching and elastic transformation optimization, and a dynamic adjustment mechanism for enhancing strategies based on scene parsing, realizing high-precision deblurring, accurate registration, and adaptive visual enhancement of unmanned aerial vehicle images, significantly improving image clarity, structural consistency, and scene readability, and being applicable to image processing tasks in complex low-altitude flight environments.

Description

Technical Field

[0001] The present invention relates to the technical field of UAV image enhancement, and specifically refers to a method and system for enhancing low-altitude photography images of UAVs. Background Art

[0002] The method for enhancing low-altitude photography images of UAVs refers to using image processing and multi-source data fusion technology to optimize the images collected by UAVs during low-altitude flight in terms of clarity, contrast, color restoration, etc., so as to improve the visual quality and information availability of the images; this method can effectively address image degradation problems caused by factors such as flight jitter, light changes, motion blur, etc., and is widely used in fields such as agricultural monitoring, urban inspection, and disaster assessment, which helps to improve the efficiency of image interpretation and the accuracy of intelligent analysis.

[0003] However, in the existing methods for enhancing low-altitude photography images of UAVs, there are technical problems such as unstable quality, missing details, and obvious noise residue in the final enhanced images due to jitter, dynamic changes, blur, and weak multi-source heterogeneous information fusion ability during the image acquisition process; in the existing image deblurring optimization methods, there are technical problems such as inaccurate modeling of complex motion blur in the non-stable flight state of UAVs, weak edge structure restoration ability of traditional degradation models, and lack of physical rationality in end-to-end network training; in the existing multi-source data registration methods, there are technical problems such as insufficient cross-modal information fusion ability, low registration accuracy between images of different resolutions, and difficulty in aligning semantic structures with traditional geometric registration methods; in the existing scene enhancement methods, there are technical problems such as poor generalization of enhancement strategies, rough scene content recognition, and insensitivity of enhancement operations to edge structure and regional brightness control. Summary of the Invention

[0004] In view of the above situation, to overcome the defects of the prior art, the present invention provides a method and system for enhancing low-altitude photography images of unmanned aerial vehicles. In the existing methods for enhancing low-altitude photography images of unmanned aerial vehicles, there are technical problems such as jitter, dynamic changes, blurring, and weak multi-source heterogeneous information fusion ability during the image acquisition process, resulting in unstable quality, missing details, and obvious noise residue in the final enhanced images. This solution creatively adopts a comprehensive image enhancement method combining deblurring optimization, multi-source data registration, and scene enhancement, realizing a closed-loop processing chain from image acquisition, deblurring, registration to intelligent enhancement. By modular design, various interference factors are effectively separated, improving the robustness of the enhancement system and significantly enhancing the clarity, structural consistency, and scene readability of low-altitude photography images, which is applicable to various low-altitude flight environments such as complex lighting and dynamic perspective changes. In the existing image deblurring optimization methods, there are technical problems such as inaccurate modeling of complex motion blur in the non-stable flight state of unmanned aerial vehicles, weak edge structure restoration ability of traditional degradation models, and lack of physical rationality in end-to-end network training. This solution creatively adopts a neuro-fuzzy kernel prediction method combining physical constraints and inertial measurement unit guidance for motion joint deblurring optimization, realizing physical consistency modeling of the blur kernel prediction and image reconstruction processes. By inverting the six-degree-of-freedom flight trajectory through IMU data and combining Kalman filter smoothing, blur kernel network prediction, and improved U-shaped network reconstruction, the problems of spatial distortion and edge smear caused by high-dynamic scenes are effectively alleviated, enhancing the edge sharpness and texture detail fidelity of the image. In the existing multi-source data registration methods, there are technical problems such as insufficient cross-modal information fusion ability, low registration accuracy between images of different resolutions, and difficulty in aligning semantic structures by traditional geometric registration methods. This solution creatively adopts an elastic transformation improved registration method combining cross-spectral feature attention matching for multi-source data registration, realizing a cross-modal alignment process from point cloud generation to graph structure matching, effectively improving the spatial consistency and structural integrity of the enhanced multi-source images after alignment. In the existing scene enhancement methods, there are technical problems such as poor generalization of enhancement strategies, rough scene content recognition, and insensitivity of enhancement operations to edge structure and regional brightness control. This solution creatively adopts a dynamic physical model coupling enhancement method combining scene perception enhancement for scene adaptive enhancement, obtaining scene adaptive enhanced image data, realizing on-demand dynamic adaptation of enhancement strategies, effectively avoiding the problems of "over-enhancement" and "artifact introduction" in traditional enhancement methods, and comprehensively improving the visual realism and expression efficiency of the image.

[0005] The technical solution adopted by the present invention is as follows: A method for enhancing low-altitude photography images of unmanned aerial vehicles provided by the present invention includes the following steps:

[0006] Step S1: Data preparation;

[0007] Step S2: Motion Joint Deblurring Optimization;

[0008] Step S3: Multi-Source Data Registration;

[0009] Step S4: Scene Adaptive Enhancement;

[0010] Step S5: Low-Altitude Aerial Photography Image Enhancement.

[0011] Further, in step S1, the data preparation is used to collect the original low-altitude aerial photography images and perform preliminary preprocessing. Specifically, through multi-source image acquisition by the unmanned aerial vehicle (UAV), the original low-altitude aerial photography image data is obtained, and through time synchronization, spatial alignment, radiometric calibration, and data index structuring in sequence, the data is preliminarily preprocessed to obtain the optimized data of the UAV low-altitude aerial photography images.

[0012] Further, in step S2, the motion joint deblurring optimization is used to eliminate the motion blur caused by UAV jitter and high-speed movement. Specifically, based on the optimized data of the UAV low-altitude aerial photography images, a neural fuzzy kernel prediction method combining physical constraints and inertial measurement unit (IMU) guidance is adopted to perform motion joint deblurring optimization to obtain deblurring optimization image data, including the following steps:

[0013] Step S21: Motion Trajectory Modeling. Specifically, the angular velocity and linear acceleration data of the UAV are collected by the inertial measurement unit, and the velocity data noise is eliminated through Kalman filtering. By constructing a six-degree-of-freedom motion equation, the differential trajectory of the UAV motion is modeled, and through parametric fitting, the smooth optimized motion state feature data is obtained;

[0014] Step S22: Blur Kernel Prediction. Specifically, based on the smooth optimized motion state feature data, a long short-term memory (LSTM) fuzzy kernel network fusion model is constructed, and the blur kernel prediction is performed in combination with the physical constraint loss to obtain the physically constrained deblurring prediction data;

[0015] Step S23: Physical Constraint Reconstruction. Specifically, based on the physically constrained deblurring prediction data, a non-uniform blur degradation model is constructed to perform physical constraint reconstruction to obtain the physically reconstructed feature intervention data. By constructing an improved U-shaped network and training the physical constraint image reconstruction model according to the joint loss function, and in combination with the physically reconstructed feature intervention data, the physical constraint image reconstruction model is used to reconstruct the deblurring optimization image to obtain the deblurring optimization image output;

[0016] Step S24: Motion Joint Deblurring. Specifically, edge sharpening is performed on the deblurring optimization image output to obtain the deblurring optimization image data.

[0017] Further, in step S3, the multi-source data registration is used to achieve pixel-level alignment of data. Specifically, based on the deblurred and optimized image data, an improved registration method combining elastic transformation with cross-spectrum feature attention matching is adopted for multi-source data registration to obtain pixel-aligned enhanced data, including the following steps:

[0018] Step S31: Cross-modal feature extraction. Specifically, based on the visible light image data in the deblurred and optimized image data and the multi-spectral data in the optimized data of the low-altitude drone photography images, through lidar point cloud projection, reference data of feature depth maps and feature intensity maps are generated. And through constructing a three-branch feature extraction network, cross-modal feature extraction is performed, and through feature fusion, multi-source data registration feature data is obtained;

[0019] Step S32: Graph structure matching. Specifically, based on the multi-source data registration feature data, an improved linear iterative clustering algorithm is adopted for pixel segmentation, and through the construction of graph nodes and node features, pixel node graph structure data is obtained. And by adopting a 3-layer graph attention network, node similarity calculation is performed to obtain a set of cross-modal registration matching pair data;

[0020] The improved linear iterative clustering algorithm optimizes the image pixel structure matching clustering by introducing a multi-spectral feature + cross-modal feature double consistency optimization term on the basis of the standard simple linear iterative clustering algorithm;

[0021] Step S33: Elastic transformation optimization. Specifically, based on the set of cross-modal registration matching pair data, an improved thin plate spline transformation solution algorithm is constructed for elastic transformation optimization to obtain resampled registration feature data;

[0022] Step S34: Multi-source data registration. Specifically, based on the resampled registration feature data, through sequentially performing local alignment mutual information inspection and adaptive iterative local registration optimization, pixel-aligned enhanced data is obtained.

[0023] Further, in step S4, the scene adaptive enhancement is used to intelligently adjust the enhancement strategy according to the image scene content. Specifically, based on the pixel-aligned enhanced data, a dynamic physical model coupling enhancement method combining scene perception enhancement is adopted for scene adaptive enhancement to obtain scene adaptive enhanced image data, including the following steps:

[0024] Step S41: Scene parsing improvement. Specifically, by constructing a lightweight segmentation pre-training network, scene parsing is performed, and reference data for scene parsing is obtained;

[0025] Step S42: Quantitative analysis of regional characteristics. Specifically, according to the specific scene type in the scene analysis reference data, construct feature indicators, and construct a multi-scale feature fusion pyramid structure based on the feature indicators to perform quantitative analysis of regional characteristics, and obtain scene region quantitative feature data;

[0026] Step S43: Dynamic scene enhancement. Specifically, based on the scene region quantitative feature data, construct a classification enhancement sub-library, and perform dynamic scene enhancement according to the scene type and scene feature quantitative value in the classification enhancement sub-library and the scene region quantitative feature data to obtain scene dynamic preliminary enhancement data;

[0027] Step S44: Scene adaptive image enhancement. Specifically, based on the scene dynamic preliminary enhancement data, adopt edge consistency processing and bilateral filtering smooth transition operations to perform comprehensive adaptive enhancement of the image, and obtain scene adaptive enhancement image data.

[0028] Further, in step S5, the low-altitude photography image enhancement is used to integrate the previous processing results to output the final enhanced image. Specifically, based on the deblurring optimized image data, the pixel alignment enhanced data, and the scene adaptive enhancement image data, perform comprehensive enhancement of the low-altitude photography image to obtain the drone low-altitude photography comprehensive enhancement image data.

[0029] An unmanned aerial vehicle (UAV) low-altitude photography image enhancement system provided by the present invention includes a data preparation module, a deblurring optimization module, a multi-source data registration module, a scene adaptive enhancement module, and a low-altitude photography image enhancement module;

[0030] The data preparation module is used for data preparation. Through data preparation, obtain UAV low-altitude photography image optimization data, and send the UAV low-altitude photography image optimization data to the deblurring optimization module;

[0031] The deblurring optimization module is used for motion joint deblurring optimization. Through motion joint deblurring optimization, obtain deblurring optimized image data, and send the deblurring optimized image data to the multi-source data registration module and the low-altitude photography image enhancement module;

[0032] The multi-source data registration module is used for multi-source data registration. Through multi-source data registration, obtain pixel alignment enhanced data, and send the pixel alignment enhanced data to the scene adaptive enhancement module and the low-altitude photography image enhancement module;

[0033] The scene adaptive enhancement module is used for scene adaptive enhancement. Through scene adaptive enhancement, obtain scene adaptive enhancement image data, and send the scene adaptive enhancement image data to the low-altitude photography image enhancement module;

[0034] The low-altitude photography image enhancement module is used for enhancing low-altitude photography images. Through the enhancement of low-altitude photography images, the comprehensive enhanced image data of drone low-altitude photography is obtained.

[0035] The beneficial effects achieved by the present invention using the above solution are as follows:

[0036] (1) In view of the technical problems in the existing drone low-altitude photography image enhancement methods, such as jitter, dynamic changes, blurring, and weak multi-source heterogeneous information fusion ability during the image acquisition process, resulting in unstable quality, missing details, and obvious noise residue in the final enhanced images, this solution creatively adopts a comprehensive image enhancement method combining deblurring optimization, multi-source data registration, and scene enhancement, realizing a closed-loop processing chain from image acquisition, deblurring, registration to intelligent enhancement. Through modular design, various interference factors are effectively separated, improving the robustness of the enhancement system, significantly enhancing the clarity, structural consistency, and scene readability of low-altitude photography images, and being applicable to various low-altitude flight environments such as complex lighting and dynamic perspective changes;

[0037] (2) In view of the technical problems in the existing image deblurring optimization methods, such as inaccurate modeling of complex motion blur in the non-steady flight state of drones, weak edge structure restoration ability of traditional degradation models, and lack of physical rationality in end-to-end network training, this solution creatively adopts a neuro-fuzzy kernel prediction method combining physical constraints and inertial measurement unit guidance for motion joint deblurring optimization, realizing physical consistency modeling of the blur kernel prediction and image reconstruction processes. By inverting the six-degree-of-freedom flight trajectory through IMU data, combining Kalman filter smoothing, blur kernel network prediction, and improved U-shaped network reconstruction, the problems of spatial distortion and edge smear caused by high-dynamic scenes are effectively alleviated, and the edge sharpness and texture detail fidelity of the image are enhanced;

[0038] (3) In view of the technical problems in the existing multi-source data registration methods, such as insufficient cross-modal information fusion ability, low registration accuracy between images of different resolutions, and difficulty in aligning semantic structures by traditional geometric registration methods, this solution creatively adopts an improved registration method of elastic transformation combining cross-spectral feature attention matching for multi-source data registration, realizing a cross-modal alignment process from point cloud generation to graph structure matching, and effectively improving the spatial consistency and structural integrity of the enhanced multi-source images after alignment;

[0039] (4)In the existing scene enhancement methods, there are technical problems such as poor generalization of enhancement strategies, rough recognition of scene content, and insensitivity of enhancement operations to edge structure and regional brightness control. This solution creatively adopts a dynamic physical model coupling enhancement method combined with scene perception enhancement to perform scene adaptive enhancement, obtaining scene adaptive enhanced image data, realizing the on-demand dynamic adaptation of enhancement strategies, effectively avoiding the problems of "over-enhancement" and "artifact introduction" in traditional enhancement methods, and comprehensively improving the visual realism and expression efficiency of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic flowchart of a method for enhancing low-altitude photography images of unmanned aerial vehicles provided by the present invention;

[0041] Figure 2 It is a schematic diagram of a system for enhancing low-altitude photography images of unmanned aerial vehicles provided by the present invention;

[0042] Figure 3 It is a schematic flowchart of the motion joint deblurring optimization in step S2;

[0043] Figure 4 It is a schematic flowchart of the multi-source data registration in step S3;

[0044] Figure 5 It is a schematic flowchart of the scene adaptive enhancement in step S4.

[0045] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the present invention.

[0048] Example 1, refer to Figure 1, a method for enhancing low-altitude photography images of drones provided by the present invention, the method comprising the following steps:

[0049] Step S1: Data preparation;

[0050] Step S2: Motion joint deblurring optimization;

[0051] Step S3: Multi-source data registration;

[0052] Step S4: Scene adaptive enhancement;

[0053] Step S5: Low-altitude photography image enhancement.

[0054] By performing the above operations, in the existing method for enhancing low-altitude photography images of drones, there are technical problems such as jitter, dynamic changes, blurring, and weak multi-source heterogeneous information fusion ability during the image acquisition process, resulting in unstable quality, missing details, and obvious noise residue in the final enhanced image. This solution creatively adopts a comprehensive image enhancement method combining deblurring optimization, multi-source data registration, and scene enhancement, realizing a closed-loop processing chain from image acquisition, deblurring, registration to intelligent enhancement. Through modular design, various interference factors are effectively separated, improving the robustness of the enhancement system, significantly enhancing the clarity, structural consistency, and scene readability of low-altitude photography images, and being applicable to various low-altitude flight environments such as complex lighting and dynamic perspective changes.

[0055] Example two, refer to Figure 1 and Figure 2 , based on the above example, in step S1, the data preparation is used to collect and preliminarily preprocess the original low-altitude photography images. Specifically, through multi-source image acquisition by the drone, the original low-altitude photography image data is obtained, and through time synchronization, spatial alignment, radiometric correction, and data index structuring in sequence, the data is preliminarily preprocessed to obtain the optimized data of the low-altitude photography images of the drone;

[0056] The original low-altitude photography image data specifically includes the original image and the original image parameters. The original image includes visible light image data, infrared image data, multispectral image data, lidar point cloud data, and thermal imaging data; the original image parameters specifically include the acquisition timestamp, positioning information data, attitude information data, sensor parameters, and image resolution parameters;

[0057] The optimized data of the low-altitude photography images of the drone specifically includes uniformly formatted image data, spatio-temporal consistency parameter data, radiometric characteristic correction data, and data index annotation structured data.

[0058] Example three, refer to Figure 1 , Figure 2 and Figure 3, this embodiment is based on the above embodiment. In step S2, the motion joint deblurring optimization is used to eliminate the motion blur caused by the drone jitter and high-speed movement. Specifically, according to the optimized data of the drone low-altitude photography image, a neural fuzzy kernel prediction method combining physical constraints and inertial measurement unit guidance is adopted to perform motion joint deblurring optimization to obtain deblurred and optimized image data, including the following steps:

[0059] Step S21: Motion trajectory modeling. Specifically, the angular velocity and linear acceleration data of the drone are collected through the inertial measurement unit, and the velocity data noise is eliminated through Kalman filtering. By constructing a six-degree-of-freedom motion equation, the differential trajectory modeling of the drone motion is carried out, and through parametric fitting, the smooth optimized motion state characteristic data is obtained;

[0060] The six-degree-of-freedom motion equation is used for the differential trajectory modeling of the drone, and the calculation formula is:

[0061] ;

[0062] In the formula, R is the attitude rotation matrix, which is used to describe the conversion from the drone body coordinate system to the geographical coordinate system, is the skew-symmetric matrix of the angular velocity, is the matrix cross product operator, v is the drone velocity vector, t is the time index, a is the linear acceleration vector, which is specifically collected through the inertial measurement unit, is the earth's angular velocity vector, and p is the drone position vector;

[0063] The parametric fitting specifically adopts a cubic B-spline curve parametric trajectory, and the calculation formula is:

[0064] ;

[0065] In the formula, T(·) is the smooth trajectory point at time t, which is used to represent the trajectory data point in the smooth optimized motion state characteristic data, n is the total number of control points, i is the control point index, N i,3 (·) is the cubic B-spline basis function corresponding to the i-th control point, P i is the spline curve control point body;

[0066] Step S22: Fuzzy kernel prediction. Specifically, according to the smooth optimized motion state characteristic data, a long short-term memory fuzzy kernel network fusion model is constructed, and combined with the physical constraint loss, the fuzzy kernel prediction is carried out to obtain the physical constraint deblurring prediction data;

[0067] The long short-term memory fuzzy kernel network fusion model includes a motion encoder, a spatial attention mechanism block, and a fuzzy physical constraint loss function;

[0068] The motion encoder specifically uses a standard bidirectional long short-term memory neural network to extract the time series of the UAV motion features based on the smoothed and optimized motion state feature data. The calculation formula is as follows:

[0069] ;

[0070] In the formula, h t is the output hidden state of the motion encoder, LSTM(·) is the standard bidirectional long short-term memory neural network function, is the change in the motion state feature corresponding to time t;

[0071] The fuzzy physical constraint loss function introduces a motion energy conservation loss term as the model loss function. The calculation formula is as follows:

[0072] ;

[0073] In the formula, is the fuzzy physical constraint loss function, x is the pixel horizontal index, y is the pixel vertical index, K(·) is the predicted pixel value, T a is the total time length, t is the time index, is the change in the motion state feature corresponding to time t;

[0074] Step S23: Physical constraint reconstruction. Specifically, according to the physical constraints, the blurred prediction data is deblurred, a non-uniform blur degradation model is constructed for physical constraint reconstruction to obtain physical reconstruction feature intervention data, and an improved U-shaped network is constructed. According to the joint loss function, the physical constraint image reconstruction model is trained, and combined with the physical reconstruction feature intervention data, the physical constraint image reconstruction model is used to reconstruct the deblurred and optimized image to obtain the deblurred and optimized image output;

[0075] The non-uniform blur degradation model is used to reconstruct the structure of the image that is locally uniform but globally non-uniformly blurred in space. The calculation formula is as follows:

[0076] ;

[0077] In the formula, I b is the blurred image data processed by the non-uniform blur degradation model, M is the total number of local blur regions, m is the local blur region index, w m is the weight of the m-th local blur region, I s is the reconstructed target image, K m is the m-th local blur kernel function, is the noise bias term;

[0078] The improved U-shaped network includes a multi-level downsampling encoder, a decoder, and a joint loss function;

[0079] The multi-level downsampling encoder specifically adopts a five-level downsampling structure, and through a 3×3 convolution and a motion feature splicing layer, receives and fuses the features in the physically constrained deblurring prediction data;

[0080] The decoder introduces deformable convolution to adapt to motion distortion, and optimizes the attention feature perception through a standard skip connection design;

[0081] The calculation formula of the improved U-shaped network is:

[0082] ;

[0083] where E l is the output of the l-th layer encoder, E l (·) is the operation function of the l-th layer encoder, Conv 3×3 is the 3×3 convolution operation function, F l-1 is the output feature of the (l - 1)-th layer encoder, M t is the motion feature data in the physically constrained deblurring prediction data, is the motion feature mapping operation, is the feature fusion operator, is the reconstructed image output by the decoder, D1(·) is the upsampling function in the first layer decoder, D5(·) is the upsampling function in the fifth layer decoder, DCN(·) is the deformable convolution network function, E5 is the output of the fifth layer encoder, A(·) is the attention enhancement operation function, E4 is the output of the fourth layer encoder, and E1 is the output of the first layer encoder;

[0084] The joint loss function specifically adopts a weighted combination of the L1 reconstruction loss and the motion consistency loss;

[0085] Step S24: Motion joint deblurring, specifically, performing edge sharpening on the output of the deblurring optimized image to obtain deblurring optimized image data.

[0086] By performing the above operations, in the existing image deblurring optimization methods, there are technical problems such as inaccurate modeling of complex motion blur in the non-steady flight state of drones, weak edge structure restoration ability of traditional degradation models, and lack of physical rationality in end-to-end network training. This solution creatively adopts a neural fuzzy kernel prediction method combined with physical constraints and inertial measurement unit guidance for motion joint deblurring optimization, realizes physical consistency modeling of the fuzzy kernel prediction and image reconstruction processes, inverts the six-degree-of-freedom flight trajectory through IMU data, combines Kalman filter smoothing, fuzzy kernel network prediction and improved U-shaped network reconstruction, effectively alleviates the problems of spatial distortion and edge ghosting caused by high-dynamic scenes, and enhances the image edge sharpness and texture detail fidelity.

[0087] Example 4, refer to Figure 1 、 Figure 2 and Figure 4 , this example is based on the above example. In step S3, the multi-source data registration is used to achieve pixel-level alignment of data. Specifically, based on the deblurred and optimized image data, an improved registration method combining elastic transformation with cross-spectral feature attention matching is adopted for multi-source data registration to obtain pixel-aligned enhanced data, including the following steps:

[0088] Step S31: Cross-modal feature extraction. Specifically, based on the visible light image data in the deblurred and optimized image data and the multi-spectral data in the optimized data of the low-altitude UAV photography images, through lidar point cloud projection, reference data of feature depth maps and feature intensity maps are generated, and through constructing a three-branch feature extraction network, cross-modal feature extraction is carried out, and through feature fusion, multi-source data registration feature data is obtained;

[0089] The three-branch feature extraction network includes a visible light branch, a multi-spectral branch, and a point cloud feature extraction branch;

[0090] The feature fusion specifically uses a multi-layer perceptron for feature fusion of the three types of modalities;

[0091] Step S32: Graph structure matching. Specifically, based on the multi-source data registration feature data, an improved linear iterative clustering algorithm is adopted for pixel segmentation, and through the construction of graph nodes and node features, pixel node graph structure data is obtained, and through adopting a 3-layer graph attention network, node similarity calculation is carried out to obtain a cross-modal registration matching pair set data;

[0092] The improved linear iterative clustering algorithm, based on the standard simple linear iterative clustering algorithm, introduces a multi-spectral feature and cross-modal feature double consistency optimization term for image pixel structure matching clustering optimization, and the calculation formula is:

[0093] ;

[0094] In the formula, F tar is the clustering matching target graph structure data, used to represent the result of pixel segmentation, S is the target segmentation pixel, is the total number of segmentation pixels, used to represent the total number of the graph nodes, k is the segmentation pixel index, P is the original pixel index of the image, S k is the pixel value corresponding to the kth segmentation pixel, I vis (·) is the visible light feature vector function, is the feature center of the visible light image, is the registration trade-off weight parameter, I ms(·) is a multi-spectral feature vector function, which is the feature center of the multi-spectral image;

[0095] Step S33: Elastic transformation optimization, specifically, based on the cross-modal registration matching pair set data, an improved thin plate spline transformation solution algorithm is constructed to perform elastic transformation optimization, and resampled registration feature data is obtained;

[0096] For the improved thin plate spline transformation solution algorithm, an energy function combined with a regularization term, a spectral consistency term, and a geometric constraint term is specifically used to improve the thin plate spline transformation. The calculation formula is:

[0097] ;

[0098] In the formula, E is the total energy function value. By minimizing the total energy function value, the optimal transformation solution is obtained. E reg is the regularization term, which is used to control the smoothness of the transformation. Specifically, a Laplacian smoothing function is used for calculation. is the spectral consistency weight, and E spec is the spectral consistency term, which is used to minimize the reflectance difference between the visible light and near-infrared bands. Specifically, the sum of squared feature errors is used for calculation. is the geometric constraint weight, and E geom is the geometric constraint term, which is used to stabilize the structural relationship between adjacent pixels. Specifically, the difference between adjacent pixels is used for calculation;

[0099] Step S34: Multi-source data registration, specifically, based on the resampled registration feature data, pixel-aligned enhanced data is obtained by sequentially performing local alignment mutual information checking and adaptive iterative local registration optimization.

[0100] By performing the above operations, in the existing multi-source data registration methods, there are technical problems such as insufficient cross-modal information fusion ability, low registration accuracy between images with different resolutions, and difficulty in aligning semantic structures by traditional geometric registration methods. This solution creatively adopts an elastic transformation improved registration method that combines cross-spectral feature attention matching to perform multi-source data registration, realizes the cross-modal alignment process from point cloud generation to graph structure matching, and effectively improves the enhanced spatial consistency and structural integrity after multi-source image alignment.

[0101] Example Five, refer to Figure 1 , Figure 2 and Figure 5, this embodiment is based on the above embodiment. In step S4, the scene adaptive enhancement is used to intelligently adjust the enhancement strategy according to the image scene content. Specifically, based on the pixel alignment enhanced data, a dynamic physical model coupling enhancement method combining scene perception enhancement is adopted to perform scene adaptive enhancement to obtain scene adaptive enhanced image data, including the following steps:

[0102] Step S41: Scene parsing improvement, specifically, by constructing a lightweight segmentation pre-trained network to perform scene parsing and obtain scene parsing reference data;

[0103] For the lightweight segmentation pre-trained network, an improved BiSeNetV2+ lightweight model is specifically used for scene parsing. The improved BiSeNetV2+ lightweight model performs branch feature extraction based on the pixel alignment enhanced data through parallel depthwise separable convolutions and global enhancement modules, and obtains scene parsing features through attention feature fusion, and performs scene type parsing based on the scene parsing features;

[0104] The scene types in the scene parsing reference data specifically include vegetation, buildings, water areas, and shadows;

[0105] Step S42: Regional characteristic quantization analysis, specifically, based on the specific scene types in the scene parsing reference data, construct feature indicators, and construct a multi-scale feature fusion pyramid structure based on the feature indicators to perform regional characteristic quantization analysis to obtain scene region quantization feature data;

[0106] For the multi-scale feature fusion pyramid structure, Gaussian pyramids are specifically used to calculate hierarchical features;

[0107] Preferably, Table 1 is a reference example table for the regional characteristic quantization analysis. As shown in the table, when the scene type is vegetation, the feature indicator is set as the red-edge band ratio index and calculated according to the calculation formula. In the calculation formula of the red-edge band ratio index, R nir is the reflectance of the near-infrared band, and R red is the reflectance of the red light band;

[0108] When the scene type is a building, the feature indicator is set as the edge structure complexity and calculated according to the calculation formula. In the calculation formula of the edge structure complexity, is the image gradient, Area is the area of regional pixels, and SSIM is the local structural similarity index of the image;

[0109] When the scene type is a water area, the feature indicator is set as the polarization reflection variance and calculated according to the calculation formula. In the calculation formula of the polarization reflection variance, Var(·) is the variance operation function, and I visis visible light image data, M polar is polarization mask image data;

[0110] When the scene type is shadow, set the feature index as the light attenuation coefficient and calculate according to the calculation formula. In the calculation formula of the light attenuation coefficient, is the average gray value of the shadow, is the average gray value of the unobstructed area;

[0111] Table 1 Reference Example Table for Regional Feature Quantification Analysis

[0112]

[0113] Step S43: Dynamic scene enhancement, specifically construct a classification enhancement sub-library according to the scene area quantization feature data, and perform dynamic scene enhancement based on the scene type and scene feature quantization value in the classification enhancement sub-library and the scene area quantization feature data to obtain the preliminary dynamic scene enhancement data;

[0114] Preferably, Table 2 is a reference example table of the classification enhancement sub-library. As shown in the table, when the scene type is vegetation, use red-edge band enhancement and chlorophyll inversion for scene enhancement; for the red-edge band enhancement, specifically enhance the red-edge band (705nm–740nm) in the multi-spectral image, and for the chlorophyll inversion, specifically use an index-based method for chlorophyll inversion;

[0115] When the scene type is building, use guided filter structure enhancement for scene enhancement. For the guided filter structure enhancement, specifically use an edge-preserving filtering algorithm based on a local linear model for guided filter structure enhancement;

[0116] When the scene type is water area, use polarization feature enhancement and specular reflection suppression for scene enhancement. For the polarization feature enhancement, specifically enhance the contrast by collecting polarization images, and for the specular reflection suppression, specifically use a method for separating the brightness reflected light based on a Retinex filter for specular reflection suppression;

[0117] When the scene type is shadow, use physical light compensation and adversarial generation repair for scene enhancement. For the physical light compensation, specifically use a brightness normalization method for enhancement, and for the adversarial generation repair, use a pre-trained CycleGAN architecture for shadow repair;

[0118] Table 2 Reference Example Table of Classification Enhancement Sub-library

[0119]

[0120] Step S44: Scene Adaptive Image Enhancement. Specifically, based on the scene, the dynamic preliminary enhanced data is enhanced. Edge consistency processing and bilateral filtering smooth transition operations are adopted for comprehensive adaptive enhancement of the image, and scene adaptive enhanced image data is obtained.

[0121] By performing the above operations, in the existing scene enhancement methods, there are technical problems such as poor generalization of enhancement strategies, rough recognition of scene content, and insensitivity of enhancement operations to edge structure and regional brightness control. This solution creatively adopts a dynamic physical model coupling enhancement method combined with scene perception enhancement for scene adaptive enhancement, obtains scene adaptive enhanced image data, realizes the on-demand dynamic adaptation of enhancement strategies, effectively avoids the problems of "over-enhancement" and "artifact introduction" in traditional enhancement methods, and comprehensively improves the visual realism and expression efficiency of the image.

[0122] Embodiment Six, refer to Figure 1 and Figure 2 In this embodiment, based on the above embodiment, in step S5, the low-altitude photography image enhancement is used to integrate the previous processing results and output the final enhanced image. Specifically, based on the deblurring optimized image data, the pixel alignment enhanced data, and the scene adaptive enhanced image data, comprehensive enhancement of the low-altitude photography image is performed to obtain unmanned aerial vehicle (UAV) low-altitude photography comprehensive enhanced image data.

[0123] Embodiment Seven, refer to Figure 1 and Figure 2 An unmanned aerial vehicle (UAV) low-altitude photography image enhancement system provided by the present invention includes a data preparation module, a deblurring optimization module, a multi-source data registration module, a scene adaptive enhancement module, and a low-altitude photography image enhancement module;

[0124] The data preparation module is used for data preparation. Through data preparation, UAV low-altitude photography image optimized data is obtained, and the UAV low-altitude photography image optimized data is sent to the deblurring optimization module;

[0125] The deblurring optimization module is used for motion joint deblurring optimization. Through motion joint deblurring optimization, deblurring optimized image data is obtained, and the deblurring optimized image data is sent to the multi-source data registration module and the low-altitude photography image enhancement module;

[0126] The multi-source data registration module is used for multi-source data registration. Through multi-source data registration, pixel alignment enhanced data is obtained, and the pixel alignment enhanced data is sent to the scene adaptive enhancement module and the low-altitude photography image enhancement module;

[0127] The scene adaptive enhancement module is used for scene adaptive enhancement. Through scene adaptive enhancement, scene adaptively enhanced image data is obtained, and the scene adaptively enhanced image data is sent to the low-altitude photography image enhancement module;

[0128] The low-altitude photography image enhancement module is used for low-altitude photography image enhancement. Through low-altitude photography image enhancement, the comprehensive enhanced image data of drone low-altitude photography is obtained.

[0129] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0130] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

[0131] The above describes the present invention and its implementation manners. Such description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. All in all, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the purpose of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A method for enhancing low-altitude photography images of an unmanned aerial vehicle, characterized in that: The method includes the following steps: Step S1: Data preparation to obtain optimized data of UAV low-altitude photography images; Step S2: Motion joint deblurring optimization. Adopt a neural fuzzy kernel prediction method combined with physical constraints and inertial measurement unit guidance to perform motion joint deblurring optimization to obtain deblurred optimized image data, including the following steps: Step S21: Motion trajectory modeling. Specifically, collect the angular velocity and linear acceleration data of the UAV through the inertial measurement unit, eliminate the velocity data noise through Kalman filtering, construct a six-degree-of-freedom motion equation, perform differential trajectory modeling of the UAV motion, and obtain smooth optimized motion state feature data through parametric fitting; Step S22: Fuzzy kernel prediction. Specifically, based on the smooth optimized motion state feature data, construct a long short-term memory fuzzy kernel network fusion model, and perform fuzzy kernel prediction in combination with physical constraint loss to obtain physically constrained deblurring prediction data; The long short-term memory fuzzy kernel network fusion model includes a motion encoder, a spatial attention mechanism block, and a fuzzy physical constraint loss function; The motion encoder specifically adopts a standard bidirectional long short-term memory neural network to extract the temporal sequence of UAV motion features based on the smooth optimized motion state feature data; The fuzzy physical constraint loss function introduces a motion energy conservation loss term as the model loss function; Step S23: Physical constraint reconstruction. Specifically, based on the physically constrained deblurring prediction data, construct a non-uniform blur degradation model to perform physical constraint reconstruction to obtain physically reconstructed feature intervention data, and through constructing an improved U-shaped network, train a physical constraint image reconstruction model according to the joint loss function, and in combination with the physically reconstructed feature intervention data, use the physical constraint image reconstruction model to reconstruct the deblurred optimized image to obtain the deblurred optimized image output; The non-uniform blur degradation model is used to reconstruct the structure of the image that is locally uniform but globally non-uniformly blurred in space; The improved U-shaped network includes a multi-level downsampling encoder, a decoder, and a joint loss function; The multi-level downsampling encoder specifically adopts a five-level downsampling structure, and receives and fuses the features in the physically constrained deblurring prediction data through 3×3 convolution and a motion feature splicing layer; The decoder introduces deformable convolution to adapt to motion distortion and optimizes the attention feature perception through a standard skip connection design; Step S24: Motion joint deblurring. Specifically, perform edge sharpening on the deblurred optimized image output to obtain deblurred optimized image data; Step S3: Multi-source data registration. Adopt an improved registration method combined with elastic transformation of cross-spectral feature attention matching to perform multi-source data registration to obtain pixel-aligned enhanced data, including the following steps: Step S31: Cross-modal feature extraction; Step S32: Graph structure matching; Step S33: Elastic transformation optimization; Step S34: Multi-source data registration; Step S4: Scene Adaptive Enhancement. A dynamic physical model coupling enhancement method combined with scene perception enhancement is adopted for scene adaptive enhancement to obtain scene adaptive enhanced image data, including the following steps: Step S41: Improvement of Scene Parsing; Step S42: Quantitative Analysis of Regional Characteristics; Step S43: Dynamic Scene Enhancement; Step S44: Scene Adaptive Image Enhancement; Step S5: Enhancement of Low-altitude Aerial Photography Images to obtain comprehensive enhanced image data of drone low-altitude aerial photography.

2. The method for enhancing an unmanned aerial vehicle low-altitude photography image according to claim 1, wherein: In Step S1, the data preparation is used to collect and preliminarily preprocess the original low-altitude aerial photography images. Specifically, the original low-altitude aerial photography image data is obtained through multi-source image acquisition by drones, and through time synchronization, spatial alignment, radiometric correction, and data index structuring in sequence, the data is preliminarily preprocessed to obtain optimized data of drone low-altitude aerial photography images.

3. The method for enhancing low-altitude photography images of an unmanned aerial vehicle according to claim 2, wherein: In Step S3, the multi-source data registration is used to achieve pixel-level alignment of the data. Specifically, based on the deblurred and optimized image data, an improved registration method of elastic transformation combined with cross-spectral feature attention matching is adopted for multi-source data registration to obtain pixel-aligned enhanced data, including the following steps: Step S31: Cross-modal Feature Extraction. Specifically, based on the visible light image data in the deblurred and optimized image data and the multi-spectral data in the optimized data of drone low-altitude aerial photography images, through lidar point cloud projection, reference data of feature depth maps and feature intensity maps are generated, and through constructing a three-branch feature extraction network, cross-modal feature extraction is carried out, and through feature fusion, multi-source data registration feature data is obtained; Step S32: Graph Structure Matching. Specifically, based on the multi-source data registration feature data, an improved linear iterative clustering algorithm is adopted for pixel segmentation, and through the construction of graph nodes and node features, pixel node graph structure data is obtained, and through adopting a 3-layer graph attention network, node similarity calculation is carried out to obtain a set of cross-modal registration matching pair data; The improved linear iterative clustering algorithm optimizes the image pixel structure matching clustering by introducing a multi-spectral feature + cross-modal feature double consistency optimization term on the basis of the standard simple linear iterative clustering algorithm; Step S33: Elastic Transformation Optimization. Specifically, based on the set of cross-modal registration matching pair data, an improved thin plate spline transformation solution algorithm is constructed for elastic transformation optimization to obtain resampled registration feature data; Step S34: Multi-source Data Registration. Specifically, based on the resampled registration feature data, through local alignment mutual information check and adaptive iterative local registration optimization in sequence, pixel-aligned enhanced data is obtained.

4. A method for enhancing low-altitude photography images of an unmanned aerial vehicle according to claim 3, characterized in that: In Step S4, the scene adaptive enhancement is used to intelligently adjust the enhancement strategy according to the image scene content. Specifically, based on the pixel-aligned enhanced data, a dynamic physical model coupling enhancement method combined with scene perception enhancement is adopted for scene adaptive enhancement to obtain scene adaptive enhanced image data, including the following steps: Step S41: Improvement of Scene Parsing. Specifically, through constructing a lightweight segmentation pre-training network, scene parsing is carried out to obtain scene parsing reference data; Step S42: Quantitative analysis of regional characteristics. Specifically, according to the specific scene type in the scene analysis reference data, construct feature indicators, and construct a multi-scale feature fusion pyramid structure based on the feature indicators to perform quantitative analysis of regional characteristics and obtain scene region quantitative feature data; Step S43: Dynamic scene enhancement. Specifically, construct a classification enhancement sub-library based on the scene region quantitative feature data, and perform dynamic scene enhancement based on the classification enhancement sub-library and the scene type and scene feature quantitative values in the scene region quantitative feature data to obtain preliminary dynamic scene enhancement data; Step S44: Scene adaptive image enhancement. Specifically, based on the preliminary dynamic scene enhancement data, adopt edge consistency processing and bilateral filtering smooth transition operations to perform comprehensive adaptive enhancement of the image and obtain scene adaptive enhanced image data.

5. A method for enhancing low-altitude photography images of an unmanned aerial vehicle according to claim 4, characterized in that: In step S5, the low-altitude aerial photography image enhancement is used to integrate the previous processing results to output the final enhanced image. Specifically, based on the de-blurred optimized image data, the pixel alignment enhanced data, and the scene adaptive enhanced image data, perform comprehensive enhancement of the low-altitude aerial photography image to obtain unmanned aerial vehicle low-altitude aerial photography comprehensive enhanced image data.

6. A low-altitude UAV photography image enhancement system for implementing a low-altitude UAV photography image enhancement method according to any one of claims 1-5, characterized in that: It includes a data preparation module, a de-blurring optimization module, a multi-source data registration module, a scene adaptive enhancement module, and a low-altitude aerial photography image enhancement module.

7. The UAV low-altitude photography image enhancement system according to claim 6, characterized in that: The data preparation module is used for data preparation. Through data preparation, obtain unmanned aerial vehicle low-altitude aerial photography image optimization data and send the unmanned aerial vehicle low-altitude aerial photography image optimization data to the de-blurring optimization module; The de-blurring optimization module is used for motion joint de-blurring optimization. Through motion joint de-blurring optimization, obtain de-blurred optimized image data and send the de-blurred optimized image data to the multi-source data registration module and the low-altitude aerial photography image enhancement module; The multi-source data registration module is used for multi-source data registration. Through multi-source data registration, obtain pixel alignment enhanced data and send the pixel alignment enhanced data to the scene adaptive enhancement module and the low-altitude aerial photography image enhancement module; The scene adaptive enhancement module is used for scene adaptive enhancement. Through scene adaptive enhancement, obtain scene adaptive enhanced image data and send the scene adaptive enhanced image data to the low-altitude aerial photography image enhancement module; The low-altitude aerial photography image enhancement module is used for low-altitude aerial photography image enhancement. Through low-altitude aerial photography image enhancement, obtain unmanned aerial vehicle low-altitude aerial photography comprehensive enhanced image data.

Images