A method and system for image enhancement of a drone

Abstract

The application discloses a kind of unmanned aerial vehicle image enhancement method and system.Therein, the method includes: obtaining first fuzzy image and its corresponding semantic prompt text;The structural feature extraction is carried out to first fuzzy image, and the semantic feature extraction is carried out to semantic prompt text, to obtain prior information;Prior information is injected into denoising model using cross attention mechanism;First fuzzy image is input into the denoising model injected with prior information to carry out multiple rounds of training and obtain target denoising model;Second fuzzy image is obtained, which is subjected to frequency domain transformation and input into frequency domain residual error model for multiple rounds of training to obtain target frequency domain residual error model;The fuzzy image to be detected is input into target denoising model for denoising to obtain spatial clear image;The fuzzy image to be detected is input into target frequency domain residual error model for prediction to obtain frequency domain clear image;Spatial clear image and frequency domain clear image are fused to obtain high-resolution image.The method can obtain higher quality clear image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method and system for enhancing images of unmanned aerial vehicles (UAVs). Background Technology

[0002] Existing long-distance surveillance images often suffer from problems such as extremely low resolution, blurred texture, severe loss of detail, complex background, and high noise levels.

[0003] Traditional super-resolution methods (such as SRCNN and ESRGAN) mainly rely on spatial domain information and do not make sufficient use of the frequency domain features of low-resolution images. They are not good at restoring "weak texture and small objects" and are prone to artifacts and distortions under complex backgrounds and noise interference. In addition, existing methods lack prior constraints on the structure and semantics of the target, resulting in deviations between the reconstructed image and the real scene.

[0004] Therefore, there is an urgent need to provide a method and system for UAV image enhancement that integrates structural priors, semantic cues, and frequency domain compensation. Summary of the Invention

[0005] This invention provides a method and system for enhancing images of unmanned aerial vehicles (UAVs) to address the problem of low image resolution in existing technologies.

[0006] To achieve the above objectives, the present invention provides a method for enhancing images of unmanned aerial vehicles (UAVs). The method includes: S1, acquiring a first blurred image and its corresponding semantic prompt text; extracting structural features from the first blurred image and semantic features from the semantic prompt text to obtain prior information; S2, injecting the prior information into a denoising model using a cross-attention mechanism; inputting the first blurred image into the denoising model with injected prior information for multiple rounds of training to obtain a target denoising model; S3, acquiring a second blurred image, performing frequency domain transformation on the second blurred image, and inputting it into a frequency domain residual model for multiple rounds of training to obtain a target frequency domain residual model; S4, inputting the blurred image to be detected into the target denoising model for denoising to obtain a clear image in the spatial domain; inputting the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain; S5, fusing the clear image in the spatial domain and the clear image in the frequency domain to obtain a high-resolution image.

[0007] Optionally, before step S1, the method includes: acquiring an original sharp image, and progressively adding Gaussian noise to the original sharp image at a preset diffusion step number T to obtain a corresponding blurred image sequence. , …, }; where, the A blurred image is obtained by adding Gaussian noise to an original sharp image. for Blurred image obtained by adding Gaussian noise for A blurred image obtained by adding Gaussian noise.

[0008] Optionally, S1 includes: S11, randomly selecting an image from the blurred image sequence as the first blurred image. S11: Obtain the semantic prompt text corresponding to the first blurred image; S12: Extract the edges of the first blurred image using an edge detector to obtain a contour map; S13: Perform structural encoding on the contour map using a structural encoder to generate a structural embedding vector; S14: Perform text encoding on the semantic prompt text using a pre-trained text encoder to generate a semantic embedding vector; S15: Concatenate the structural embedding vector and the semantic embedding vector to form a conditional vector, and use the conditional vector as prior information.

[0009] Optionally, S2 includes: S21, injecting the prior information into the denoising model using a cross-attention mechanism; S22, processing the first blurred image... The denoising model, which is injected with prior information, is used for prediction to obtain the Gaussian noise from the previous step of the prediction. S23. Based on the predicted Gaussian noise from the previous step... and the corresponding blurred image in the blurred image sequence S24. The noise loss value is calculated by adding Gaussian noise; S25. The denoising model is updated in reverse according to the noise loss value; S11~S24 are repeated until the noise loss value fluctuates within a first preset range and then the denoising model obtained by the last update is used as the target denoising model.

[0010] Optionally, step S3 includes: S31, randomly selecting an image from the blurred image sequence as a second blurred image, performing a Fourier transform on the second blurred image to obtain a frequency domain representation; S32, inputting the frequency domain representation into a frequency domain residual model for prediction to obtain a frequency domain target feature spectrum; S33, performing an inverse Fourier transform on the frequency domain target feature spectrum to obtain a frequency domain predicted clear image; S34, calculating a frequency domain loss value based on the frequency domain predicted clear image and the original clear image; S35, updating the frequency domain residual model in reverse based on the frequency domain loss value; S36, repeating steps S31 to S35 until the frequency domain loss value fluctuates within a second preset range, and using the frequency domain residual model obtained from the last update as the target frequency domain residual model.

[0011] Optionally, the step of inputting the blurred image to be detected into the target denoising model for denoising to obtain a clear image in the spatial domain includes: extracting edges from the blurred image to be detected using an edge detector; encoding the extracted results using a structure encoder; encoding the semantic prompt text corresponding to the blurred image to be detected using a pre-trained text encoder; concatenating the structure encoding result and the text encoding result, and injecting the concatenated result as prior information into the target denoising model; and inputting the blurred image to be detected into the target denoising model with the injected prior information for a preset T-step denoising to obtain a clear image in the spatial domain.

[0012] Optionally, the step of inputting the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain includes: performing a Fourier transform on the blurred image to be detected; inputting the transform result into the target frequency domain residual model for prediction; and performing an inverse Fourier transform based on the prediction result and the transform result to obtain a clear image in the frequency domain.

[0013] On the other hand, the present invention provides an image enhancement system for unmanned aerial vehicles (UAVs). This system includes: a priori guidance unit, used to acquire a first blurred image and its corresponding semantic prompt text; extract structural features from the first blurred image and semantic features from the semantic prompt text to obtain prior information; a target denoising model training unit, used to inject the prior information into a denoising model using a cross-attention mechanism; input the first blurred image into the denoising model with injected prior information for multiple rounds of training to obtain a target denoising model; a target frequency domain residual model training unit, used to acquire a second blurred image, perform frequency domain transformation on the second blurred image, and input it into the frequency domain residual model for multiple rounds of training to obtain a target frequency domain residual model; a spatial frequency domain image prediction unit, used to input the blurred image to be detected into the target denoising model for denoising to obtain a clear spatial image; input the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear frequency domain image; and an image fusion unit, used to fuse the clear spatial image and the clear frequency domain image to obtain a high-resolution image.

[0014] Optionally, it also includes: a forward noise-adding unit, used to: acquire the original sharp image, and progressively add Gaussian noise to the original sharp image according to a preset diffusion number T, to obtain the corresponding blurred image sequence. , …, }; where, the A blurred image is obtained by adding Gaussian noise to an original sharp image. for Blurred image obtained by adding Gaussian noise for A blurred image obtained by adding Gaussian noise.

[0015] Optionally, the prior guidance unit includes: an acquisition subunit, configured to randomly select an image from the blurred image sequence as the first blurred image. The system comprises the following subunits: a semantic prompt text corresponding to a first blurred image; an edge extraction subunit, used to extract edges from the first blurred image using an edge detector to obtain a contour map; a structure encoding subunit, used to perform structure encoding on the contour map using a structure encoder to generate a structure embedding vector; a text encoding subunit, used to perform text encoding on the semantic prompt text using a pre-trained text encoder to generate a semantic embedding vector; and a concatenation subunit, used to concatenate the structure embedding vector and the semantic embedding vector into a conditional vector, using the conditional vector as prior information.

[0016] The beneficial effects of this invention are:

[0017] This invention provides a method and system for enhancing UAV images. The method extracts structural features from blurred images and, combined with semantic cue text, injects these features into a denoising model via cross-attention. This allows the denoising model to focus on restoring target regions, thereby improving the accuracy of identifying and reconstructing small targets such as UAVs. The hybrid spatial and frequency domain reconstruction combines spatial detail with frequency residual compensation, significantly improving texture clarity and resulting in higher-quality, clearer images. Attached Figure Description

[0018] Figure 1 This is a flowchart of an image enhancement method for unmanned aerial vehicles (UAVs) provided in an embodiment of the present invention;

[0019] Figure 2 This is a flowchart of generating prior information provided in an embodiment of the present invention;

[0020] Figure 3 This is a flowchart of the target denoising model provided in an embodiment of the present invention;

[0021] Figure 4 This is a flowchart of generating a target frequency domain residual model provided in an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of the structure of an image enhancement system for unmanned aerial vehicles provided in an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of the prior guidance unit provided in an embodiment of the present invention;

[0024] Figure 7 This is a schematic diagram of the structure of the target denoising model training unit provided in an embodiment of the present invention;

[0025] Figure 8This is a schematic diagram of the structure of the target frequency domain residual model training unit provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0027] Figure 1 This invention provides a method for enhancing images of unmanned aerial vehicles (UAVs), such as... Figure 1 As shown, the method includes:

[0028] S1. Obtain a first blurred image and its corresponding semantic prompt text; extract structural features from the first blurred image and extract semantic features from the semantic prompt text to obtain prior information;

[0029] In an optional implementation, the following is included before S1:

[0030] Obtain the original clear image, and then gradually add Gaussian noise to the original clear image according to a preset diffusion number T to obtain the corresponding blurred image sequence. , …, }; where, the A blurred image is obtained by adding Gaussian noise to an original sharp image. for Blurred image obtained by adding Gaussian noise for A blurred image obtained by adding Gaussian noise.

[0031] Specifically, the original clear image It can be derived from clear images taken during long-distance monitoring or patrols; based on the original clear images, a sequence of blurred images in the diffusion process is constructed to provide training samples covering different noise intensities for subsequent model training.

[0032] The original clear image is set to a size of H×W×3, where H and W represent the height and width of the image, respectively, and the value "3" represents the three color channels (red, green, and blue) of the color image. A preset diffusion step number T is used (e.g., T=1000, but not limited to this).

[0033] In one specific implementation, Gaussian noise is added to the original clear image as a first step. To obtain a blurred image For blurred images The second step is to add Gaussian noise. To obtain a blurred image For blurred images The third step is to add Gaussian noise. To obtain a blurred image And so on, for blurred images Add Gaussian noise in step T. To obtain a blurred image That is, Gaussian noise is gradually added to the original clear image according to a preset diffusion number T. , …, }, thus obtaining the corresponding blurred image sequence { , …, }

[0034] In one optional implementation, Figure 2 This is a flowchart of generating prior information provided in an embodiment of the present invention; such as Figure 2 As shown, S1 includes:

[0035] S11. Randomly select one image from the blurred image sequence as the first blurred image. Obtain the semantic prompt text corresponding to the first blurred image;

[0036] In one alternative implementation, the semantic prompt text may include, but is not limited to, natural language descriptions of target category, image quality, or detail requirements, such as: "drone," "blurry image," "unclear details," "unsharp outline," etc.

[0037] Semantic prompt text can be manually entered, i.e., the operator selects the first blurred image and enters descriptive text.

[0038] S12. Use an edge detector to extract the edges of the first blurred image to obtain a contour map;

[0039] Specifically, the edge detector is a Canny edge detector, and its formula is:

[0040]

[0041] in, The first blurred image, For Canny edge detectors, This is a outline drawing.

[0042] The Canny edge detector can effectively extract the contour information of objects in the first blurred image, and obtain a contour map with the same size as the first blurred image.

[0043] S13. The contour map is structurally encoded using a structural encoder to generate a structural embedding vector;

[0044] Specifically, the structural encoder is a shallow Transformer structured encoder; it internally contains several self-attention modules for feature modeling of edge information in the contour map. Its formula is:

[0045]

[0046] in, For structural embedding vectors, For structural encoders, This is a outline drawing.

[0047] In one specific embodiment, the structural encoder may include four attention modules, each of which can perform feature association calculations on contour information within different receptive fields, thereby obtaining a more stable geometric structure representation.

[0048] After processing by the structure encoder, the output is a set of high-dimensional feature vector sequences, namely structure embedding vectors, which are used to represent the edge contour features and geometric features of the target in the first blurred image.

[0049] S14. The semantic prompt text is encoded using a pre-trained text encoder to generate a semantic embedding vector;

[0050] Specifically, the pre-trained text encoder is a pre-trained clip text encoder, which adopts the ViT-L / 14 architecture; its formula is:

[0051]

[0052] in, For semantic embedding vectors, For text encoders, This is semantic prompt text.

[0053] High-dimensional semantic embedding vectors are obtained through a pre-trained text encoder. This can introduce cross-modal semantic priors for subsequent image denoising and enhancement processes, enabling the model to pay more attention to target-related structures and details when processing blurry or low-resolution drone images, thereby improving the reconstruction accuracy and visual effect of the final image.

[0054] S15. Concatenate the structural embedding vector and the semantic embedding vector into a condition vector, and use the condition vector as prior information.

[0055] Specifically, the structural embedding vector obtained through edge detection and shallow Transformer is concatenated with the semantic embedding vector obtained through pre-trained text encoder along the feature dimension to generate a unified conditional vector.

[0056] The dimension of the structure embedding vector is The semantic embedding vector has a dimension of 1×d. The semantic embedding vector is copied N times and expanded to... By concatenating the structural embedding vector with the expanded semantic embedding vector, a conditional vector is obtained. (Dimensions are 2N×d).

[0057] The conditional vector is used as prior information injected into the denoising network to guide the network to pay attention to both the structural and semantic features of the image during the image reconstruction process, thereby improving the accuracy of target detail restoration.

[0058] S2. The prior information is injected into the denoising model using a cross-attention mechanism; the first blurred image is input into the denoising model with injected prior information for multiple rounds of training to obtain the target denoising model;

[0059] In one optional implementation, Figure 3 This is a flowchart of the target denoising model provided in the embodiments of the present invention, such as... Figure 3 As shown, S2 includes:

[0060] S21. The prior information is injected into the denoising model using a cross-attention mechanism;

[0061] Specifically, the prior information is injected into each intermediate feature map of the denoising model using a cross-attention mechanism. The denoising model can be a U-Net, SCUNet, or DDPM denoising structure, containing 8 downsampling / upsampling blocks.

[0062] S22, the first blurred image The denoising model, which is injected with prior information, is used for prediction to obtain the Gaussian noise from the previous step of the prediction. ;

[0063] Specifically, the first blurred image The prediction is performed in the denoising model that has been injected with prior information, and the Gaussian noise at step t-1 is obtained. First blurred image Remove Gaussian noise at step t-1 of the prediction This will give us the blurred image predicted at step t-1. .

[0064] S23. Based on the predicted Gaussian noise from the previous step. and the corresponding blurred image in the blurred image sequence Added Gaussian noise (i.e. The noise loss value is calculated.

[0065] Specifically, the noise loss value is calculated according to the following formula:

[0066]

[0067] in, This represents the noise loss value. The corresponding blurred image in the blurred image sequence Added Gaussian noise , The previous step's Gaussian noise is used for prediction, and N is the blurred image. The number of pixels.

[0068] S24. Update the denoising model in reverse based on the noise loss value;

[0069] Specifically, the weights of the denoising model are updated in reverse based on the noise loss value.

[0070] S25. Repeat S11~S24 until the noise loss value fluctuates within a first preset range, and use the denoising model obtained from the last update as the target denoising model.

[0071] Specifically, repeat steps S11~S24, traversing different randomly sampled first blurred images until the noise loss value is within a first preset range ( Training is stopped when there is fluctuation within the range, and the denoised model updated last is used as the target denoised model.

[0072] S3. Obtain the second blurred image, perform frequency domain transformation on the second blurred image, and input it into the frequency domain residual model for multiple rounds of training to obtain the target frequency domain residual model.

[0073] In one optional implementation, Figure 4 This is a flowchart of generating a target frequency domain residual model provided in an embodiment of the present invention; as shown below. Figure 4 As shown, S3 includes:

[0074] S31. Randomly select one image from the blurred image sequence as the second blurred image, and perform a Fourier transform on the second blurred image to obtain its frequency domain representation. ;

[0075] Preferably, the Fourier transform can be performed using the Fast Fourier Transform (FFT) algorithm to obtain the amplitude spectrum and phase spectrum of the second blurred image in the frequency domain;

[0076] The frequency domain representation It is mainly used to describe the spectral information of the second blurred image so as to compensate and enhance high-frequency details in the frequency domain.

[0077] S32. Input the frequency domain representation into the frequency domain residual model for prediction to obtain the frequency domain target feature spectrum. ;

[0078] The frequency domain residual model can be a U-Net or FreeU structure. In a preferred embodiment, the frequency domain residual model adopts a U-Net-based neural network structure, including four convolutional layers. Each convolutional layer contains units such as convolution, batch normalization (BN), and nonlinear activation (ReLU). The prediction objective of the model is to extract key frequency domain features of the target, remove fuzzy frequency domain noise, and obtain the frequency domain target feature spectrum. .

[0079] S33. Perform an inverse Fourier transform on the frequency domain target feature spectrum to obtain a clear frequency domain prediction image;

[0080] The predicted frequency domain target feature spectrum Performing an inverse Fourier transform (IFFT) restores the image to the spatial domain, yielding the corresponding sharp image predicted in the frequency domain. .

[0081] S34. Calculate the frequency domain loss value based on the frequency domain predicted sharp image and the original sharp image;

[0082] Specifically, the frequency domain loss value is calculated according to the following formula:

[0083]

[0084] Among them, the This represents the frequency domain loss value. Indicates the original, clear image. This indicates that the image is sharp in the frequency domain. This indicates the number of pixels in the original sharp image or the frequency domain predicted sharp image.

[0085] S35. Update the frequency domain residual model in reverse based on the frequency domain loss value;

[0086] Specifically, based on the frequency domain loss value The backpropagation algorithm is used to update the parameters of the frequency domain residual model using gradients, thereby gradually improving the prediction accuracy.

[0087] S36. Repeat S31 to S35 until the frequency domain loss value fluctuates within a second preset range, and use the frequency domain residual model obtained from the last update as the target frequency domain residual model.

[0088] Repeat steps S31 to S35 to learn different second blurred images until the frequency domain loss value is reached. Second preset range ( Training is stopped when fluctuations occur within the specified range. The frequency domain residual model updated last is used as the target frequency domain residual model.

[0089] S4. Input the blurred image to be detected into the target denoising model for denoising to obtain a clear image in the spatial domain; input the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain;

[0090] In an optional implementation, the step of inputting the blurred image to be detected into the target denoising model for denoising to obtain a clear spatial image includes:

[0091] The blurred image to be detected is used to extract edges using an edge detector;

[0092] The extracted results are then structurally encoded using a structural encoder.

[0093] The semantic prompt text corresponding to the blurred image to be detected is encoded using a pre-trained text encoder;

[0094] The structural encoding result and the text encoding result are concatenated, and the concatenated result is injected as prior information into the target denoising model;

[0095] The blurred image to be detected is input into the target denoising model, which is injected with prior information, and then subjected to a preset T-step denoising process to obtain a clear image in the spatial domain.

[0096] Specifically, the prior information acquisition method of its reasoning process is similar to S11~S15 of this application, except that the first blurred image is replaced with the blurred image to be detected, and the model uses a pre-trained target denoising model.

[0097] The blurred image to be detected is input into the target denoising model after injecting prior information, and a preset T-step reverse denoising process is performed, that is, the Gaussian noise component to be denoised is predicted step by step. The denoised image from the previous step is calculated based on the denoised image from the current step and the denoised Gaussian noise components; the formula is:

[0098]

[0099] in, This is the denoised image from the previous step. For the denoised image of the current step, For diffusion parameters, The predicted Gaussian noise component.

[0100] When t=T For the blurred image to be detected, input the blurred image to be detected into the target denoising model after injecting prior information to predict the Gaussian noise component. Denoise the blurred image to be detected and the Gaussian noise component to obtain the denoised image of step T-1. Repeat the above steps until t=1. A clear image of the airspace.

[0101] In an optional implementation, the step of inputting the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain includes:

[0102] Perform a Fourier transform on the blurred image to be detected;

[0103] The transformation result is input into the target frequency domain residual model for prediction;

[0104] A clear image in the frequency domain is obtained by performing an inverse Fourier transform based on the prediction and transformation results.

[0105] Specifically, its reasoning process is similar to S31~S33 of this application, except that the second blurred image is replaced with the blurred image to be detected, and the model uses the pre-trained target frequency domain residual model.

[0106] S5. The clear image in the spatial domain and the clear image in the frequency domain are fused to obtain a high-resolution image.

[0107] Specifically, the formula is as follows:

[0108]

[0109] in, For high-resolution images, For clear images in the airspace, For a clear image in the frequency domain.

[0110] Through the above steps, the blurred image (low-resolution image) to be detected can be reconstructed into a high-resolution image with clear details and semantic consistency, significantly improving the accuracy of target detection.

[0111] Figure 5 This is a schematic diagram of the structure of a drone image enhancement system provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the system includes:

[0112] The prior guidance unit 201 is used to acquire a first blurred image and its corresponding semantic prompt text; to extract structural features from the first blurred image and semantic features from the semantic prompt text to obtain prior information;

[0113] In one optional implementation, Figure 6 This is a schematic diagram of the priori guidance unit provided in an embodiment of the present invention; as shown below. Figure 6 As shown, the priori guidance unit 201 includes:

[0114] Acquisition subunit 2011 is used to randomly select an image from the blurred image sequence as the first blurred image. Obtain the semantic prompt text corresponding to the first blurred image;

[0115] The edge extraction subunit 2012 is used to extract the edges of the first blurred image using an edge detector to obtain a contour map;

[0116] The structure encoding subunit 2013 is used to perform structure encoding on the contour map by a structure encoder to generate a structure embedding vector;

[0117] The text encoding subunit 2014 is used to encode the semantic prompt text using a pre-trained text encoder to generate a semantic embedding vector;

[0118] The splicing subunit 2015 is used to splice the structural embedding vector and the semantic embedding vector into a condition vector, and use the condition vector as prior information.

[0119] The target denoising model training unit 202 is used to inject the prior information into the denoising model using a cross-attention mechanism; and to input the first blurred image into the denoising model with injected prior information for multiple rounds of training to obtain the target denoising model.

[0120] In one optional implementation, Figure 7 This is a schematic diagram of the structure of the target denoising model training unit provided in an embodiment of the present invention; as shown below. Figure 7 As shown, the target denoising model training unit 202 includes:

[0121] Injection subunit 2021 is used to inject the prior information into the denoising model using a cross-attention mechanism;

[0122] The first prediction subunit 2022 is used to predict the first blurred image. The denoising model, which is injected with prior information, is used for prediction to obtain the Gaussian noise from the previous step of the prediction. ;

[0123] The first calculation subunit 2023 is used to calculate the predicted Gaussian noise from the previous step. and the corresponding blurred image in the blurred image sequence The noise loss value is calculated by adding Gaussian noise;

[0124] The first backpropagation subunit 2024 is used to backpropagate the denoising model based on the noise loss value.

[0125] The first repeated training subunit 2025 is used to repeat the prior guidance unit 201, injection subunit 2021, first prediction subunit 2022, first calculation subunit 2023, and first backpropagation subunit 2024 until the noise loss value fluctuates within a first preset range, and the denoising model obtained from the last update is used as the target denoising model.

[0126] The target frequency domain residual model training unit 203 is used to acquire the second blurred image, perform frequency domain transformation on the second blurred image, and input it into the frequency domain residual model for multiple rounds of training to obtain the target frequency domain residual model.

[0127] In one optional implementation, Figure 8 This is a schematic diagram of the structure of the target frequency domain residual model training unit provided in an embodiment of the present invention; as shown below. Figure 8 As shown, the target frequency domain residual model training unit 203 includes:

[0128] Transform subunit 2031 is used to randomly select an image from the blurred image sequence as a second blurred image, and perform a Fourier transform on the second blurred image to obtain a frequency domain representation;

[0129] The second prediction subunit 2032 is used to input the frequency domain representation into the frequency domain residual model for prediction to obtain the frequency domain target feature spectrum;

[0130] The inverse transform subunit 2033 is used to perform an inverse Fourier transform on the frequency domain target feature spectrum to obtain a clear frequency domain prediction image;

[0131] The second calculation subunit 2034 is used to calculate the frequency domain loss value based on the frequency domain predicted sharp image and the original sharp image;

[0132] The second backpropagation subunit 2035 is used to update the frequency domain residual model in reverse according to the frequency domain loss value;

[0133] The second repeated training subunit 2036 is used to repeat the transformation subunit 2031, the second prediction subunit 2032, the inverse transformation subunit 2033, the second calculation subunit 2034, and the second backpropagation subunit 2035 until the frequency domain loss value fluctuates within a second preset range, and the frequency domain residual model obtained from the last update is used as the target frequency domain residual model.

[0134] The spatial and frequency domain image prediction unit 204 is used to input the blurred image to be detected into the target denoising model for denoising to obtain a clear image in the spatial domain; and to input the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain.

[0135] The image fusion unit 205 is used to fuse the clear image in the spatial domain and the clear image in the frequency domain to obtain a high-resolution image.

[0136] The beneficial effects of this invention are:

[0137] This invention provides a method and system for enhancing UAV images. The method introduces prior information on edge contours through structure-guided branching, effectively avoiding target shape distortion. Combined with semantic embedding vectors generated from semantic prompt text, it guides the model to maintain semantic invariance in the restoration of extremely small targets, solving the problem of restoring degraded regions. The hybrid spatial and frequency domain reconstruction combines spatial details with frequency domain residual compensation, significantly improving texture clarity. The method of this invention has strong scene adaptability and is suitable for long-distance UAV monitoring scenarios in complex backgrounds, providing high-quality images for target detection.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing images of unmanned aerial vehicles (UAVs), characterized in that, include: S1. Obtain the first blurred image and its corresponding semantic prompt text; Structural features are extracted from the first blurred image, and semantic features are extracted from the semantic prompt text to obtain prior information; S2. The prior information is injected into the denoising model using a cross-attention mechanism; The first blurred image is input into the denoising model injected with prior information and trained for multiple rounds to obtain the target denoising model; S3. Obtain the second blurred image, perform frequency domain transformation on the second blurred image, and input it into the frequency domain residual model for multiple rounds of training to obtain the target frequency domain residual model. S4. Input the blurred image to be detected into the target denoising model for denoising to obtain a clear spatial image; The blurred image to be detected is input into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain; S5. The clear spatial image and the clear frequency domain image are fused to obtain a high-resolution image; Before S1, the following are included: Obtain the original clear image, and then gradually add Gaussian noise to the original clear image according to a preset diffusion number T to obtain the corresponding blurred image sequence. , …, }; where, the A blurred image is obtained by adding Gaussian noise to an original sharp image. for Blurred image obtained by adding Gaussian noise for Blurred image obtained by adding Gaussian noise; S1 includes: S11. Randomly select one image from the blurred image sequence as the first blurred image. Obtain the semantic prompt text corresponding to the first blurred image; S12. Use an edge detector to extract the edges of the first blurred image to obtain a contour map; S13. The contour map is structurally encoded using a structural encoder to generate a structural embedding vector; S14. The semantic prompt text is encoded using a pre-trained text encoder to generate a semantic embedding vector; S15. Concatenate the structural embedding vector and the semantic embedding vector into a condition vector, and use the condition vector as prior information.

2. The method according to claim 1, characterized in that, S2 includes: S21. The prior information is injected into the denoising model using a cross-attention mechanism; S22, the first blurred image The denoising model, which is injected with prior information, is used for prediction to obtain the Gaussian noise from the previous step of the prediction. ; S23. Based on the predicted Gaussian noise from the previous step. and the corresponding blurred image in the blurred image sequence The noise loss value is calculated by adding Gaussian noise; S24. Update the denoising model in reverse based on the noise loss value; S25. Repeat S11~S24 until the noise loss value fluctuates within a first preset range, and use the denoising model obtained from the last update as the target denoising model.

3. The method according to claim 1, characterized in that, S3 includes: S31. Randomly select one image from the blurred image sequence as the second blurred image, and perform Fourier transform on the second blurred image to obtain its frequency domain representation; S32. Input the frequency domain representation into the frequency domain residual model for prediction to obtain the frequency domain target feature spectrum; S33. Perform an inverse Fourier transform on the frequency domain target feature spectrum to obtain a clear frequency domain prediction image; S34. Calculate the frequency domain loss value based on the frequency domain predicted sharp image and the original sharp image; S35. Update the frequency domain residual model in reverse based on the frequency domain loss value; S36. Repeat S31 to S35 until the frequency domain loss value fluctuates within a second preset range, and use the frequency domain residual model obtained from the last update as the target frequency domain residual model.

4. The method according to claim 1, characterized in that, The step of inputting the blurred image to be detected into the target denoising model for denoising to obtain a clear image in the spatial domain includes: The blurred image to be detected is used to extract edges using an edge detector; The extracted results are then structurally encoded using a structural encoder. The semantic prompt text corresponding to the blurred image to be detected is encoded using a pre-trained text encoder; The structural encoding result and the text encoding result are concatenated, and the concatenated result is injected as prior information into the target denoising model; The blurred image to be detected is input into the target denoising model, which is injected with prior information, and then subjected to a preset T-step denoising process to obtain a clear image in the spatial domain.

5. The method according to claim 1, characterized in that, The step of inputting the blurred image to be detected into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain includes: Perform a Fourier transform on the blurred image to be detected; The transformation result is input into the target frequency domain residual model for prediction; A clear image in the frequency domain is obtained by performing an inverse Fourier transform based on the prediction and transformation results.

6. An image enhancement system for unmanned aerial vehicles (UAVs), characterized in that, include: A priori guidance unit is used to acquire the first blurred image and its corresponding semantic prompt text; Structural features are extracted from the first blurred image, and semantic features are extracted from the semantic prompt text to obtain prior information; The target denoising model training unit is used to inject the prior information into the denoising model using a cross-attention mechanism; the first blurred image is input into the denoising model with injected prior information for multiple rounds of training to obtain the target denoising model; The target frequency domain residual model training unit is used to acquire the second blurred image, perform frequency domain transformation on the second blurred image, and input it into the frequency domain residual model for multiple rounds of training to obtain the target frequency domain residual model. The spatial and frequency domain image prediction unit is used to input the blurred image to be detected into the target denoising model for denoising, so as to obtain a clear spatial image. The blurred image to be detected is input into the target frequency domain residual model for prediction to obtain a clear image in the frequency domain; An image fusion unit is used to fuse the clear image in the spatial domain and the clear image in the frequency domain to obtain a high-resolution image; It also includes: a forward noise-adding unit, used for: Obtain the original clear image, and then gradually add Gaussian noise to the original clear image according to a preset diffusion number T to obtain the corresponding blurred image sequence. , …, }; where, the A blurred image is obtained by adding Gaussian noise to an original sharp image. for Blurred image obtained by adding Gaussian noise for Blurred image obtained by adding Gaussian noise; The prior guidance unit includes: The acquisition subunit is used to randomly select an image from the blurred image sequence as the first blurred image. Obtain the semantic prompt text corresponding to the first blurred image; The edge extraction subunit is used to extract edges from the first blurred image using an edge detector to obtain a contour map. The structure encoding subunit is used to perform structure encoding on the contour map by a structure encoder to generate a structure embedding vector; The text encoding subunit is used to encode the semantic prompt text using a pre-trained text encoder to generate a semantic embedding vector; The splicing subunit is used to splice the structural embedding vector and the semantic embedding vector into a condition vector, and to use the condition vector as prior information.

Images