Moving target sharpening method in dynamic operation scene of security unmanned aerial vehicle

Abstract

The invention relates to the technical field of image processing and computer vision, and particularly discloses a method for sharpening a moving target in a dynamic operation scene of a security unmanned aerial vehicle, which comprises the following steps: constructing a clear-fuzzy paired data set close to a real imaging mechanism, designing a dynamic fuzzy removal network (DeAP) fusing a momentum contrast feature module (MCFM) and a prior fusion module (SCA), and obtaining a clear target in the dynamic operation scene of the security unmanned aerial vehicle. And a multi-loss joint optimization strategy is adopted to realize self-adaptive modeling of complex dynamic fuzziness and clearness of a moving target. According to the method, the problems of incapability of effectively distinguishing global and local fuzziness, insufficient data set authenticity, unbalanced network modeling capability and calculation efficiency, poor clearness effect and the like in the prior art are solved, the quality of the security and protection monitoring image of the unmanned aerial vehicle is remarkably improved, and reliable support is provided for law enforcement and evidence collection.

Description

Technical Field

[0001] This invention relates to a method for sharpening moving targets in dynamic operation scenarios of security drones, which can be applied to fields such as security monitoring, public safety patrol, counter-terrorism and stability maintenance, and emergency response, and belongs to the fields of image processing and computer vision technology. Background Technology

[0002] With the rapid development of artificial intelligence, the Internet of Things and aviation technology, drones have become an indispensable core tool in modern security systems due to their advantages such as high mobility, wide field of vision and flexible deployment. They have greatly expanded the coverage of traditional ground monitoring and improved the response efficiency and control capabilities of social security management.

[0003] However, in critical security scenarios such as high-speed drone tracking and complex environment inspection, image quality faces severe challenges: images captured by drone cameras inevitably suffer from severe complex motion blur. This blur is caused by the superposition of two factors: first, large-scale global motion blur caused by the drone's attitude changes during high-altitude flight, high-frequency propeller vibration, and airflow disturbance, resulting in overall jitter and blurring of the entire image; second, small-scale local non-uniform blur caused by the independent movement of ground security targets (such as a fast-running suspect or a high-speed vehicle), where the relative motion difference between the target and the background causes different regions of the image to be convolved by different blur kernels. This complex motion blur can lead to the loss of key visual information, making the surveillance footage difficult to use as effective evidence for law enforcement or evidence collection.

[0004] The existing technology also has the following significant drawbacks: 1. Dataset construction: Traditional methods are mostly based on simple convolution kernels or random degradation to synthesize training data, which cannot realistically simulate the composite dynamic blur characteristics of "global + local" superposition in UAV imaging, resulting in the training data being out of touch with the actual scene and the supervision information provided being unreliable; 2. In terms of fuzzy discrimination capability: Most existing defuzzing methods are based on single fuzzy kernel estimation, which cannot explicitly distinguish between global motion fuzziness and local non-uniform fuzziness, making it difficult to specifically clarify security targets; 3. In terms of network modeling: Traditional convolutional neural networks (CNNs) have limited receptive fields and cannot effectively model long-distance spatial dependencies in dynamic blur; although the pure Transformer structure has global modeling capabilities, its computational complexity increases with the square of the image resolution and it is not stable enough for local texture restoration. Both are difficult to balance computational efficiency and modeling effect. 4. Regarding the design of loss functions: Existing methods mostly use a single loss function (such as L1 loss or MSE loss), which can only focus on pixel-level differences and cannot simultaneously take into account structural detail restoration and blur-sharp state discrimination, resulting in problems such as texture blur and incomplete edges in the sharpening results.

[0005] Therefore, developing a moving target de-identification technology that can accurately distinguish complex fuzzy patterns, adapt to drone security scenarios, and balance efficiency and effectiveness is of great significance for improving the practical capabilities of drone security systems and ensuring public safety. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies in handling blurred composite dynamic scenes of security drones, such as the inability to effectively distinguish between global and local motion and the difficulty in sharpening key security targets, this invention provides a method for sharpening moving targets in dynamic operation scenarios of security drones. This method can adaptively learn image degradation priors and enhance the details of moving targets.

[0007] A method for declarative analysis of moving targets in dynamic operation scenarios of security drones includes the following steps: Step 1: Constructing a sharp-blurred pair dataset: High frame rate continuous sharp frame sequences are collected by security drones, and blurred images are synthesized based on the physical integral model of the camera response function. The intermediate frame of the continuous sharp frames is selected as the sharp reference image to construct a sharp-blurred pair dataset containing training and test sets. Step 2: Constructing the De-blurring Network DeAP: The De-blurring network includes a Momentum Contrast Feature Module (MCFM), a Prior Fusion Module (SCA), and a multi-level deblurring main network. The Momentum Contrast Feature Module (MCFM) is used to learn degradation perception priors for different blurring modes. The Prior Fusion Module (SCA) is used to adaptively fuse degradation perception priors with multi-scale features. The multi-level deblurring main network is based on a U-Net structure that fuses convolutional neural networks and StripedTransformers, taking into account both local details and long-range spatial dependency modeling. Specifically, the DeAP (Learning Degradation-aware Prior for Image Deblurring) network uses a U-Net encoder-decoder structure as the backbone network and integrates a momentum contrast feature module (MCFM) and a prior fusion module (SCA) to improve the model's ability to represent different types of dynamic blur.

[0008] The Momentum Contrast Feature Module (MCFM) is primarily used to learn a global prior representation of degradation perception from the input blurred image. Let the input image be... MCFM, through its contrastive learning mechanism that constructs positive and negative samples, enables the network to explicitly capture and distinguish blurry patterns from different sources, providing a discernible prior for degradation perception for the subsequent deblurring main network.

[0009] The specific steps for obtaining degradation-aware priors using the MCFM module are as follows: 1. Construct positive and negative sample pairs: from the image Two sizes are randomly cut from the middle. Image patches are processed and random data augmentation is applied to obtain query samples. and positive samples Patches cropped from other images within the same batch are defined as negative samples. Because positive samples come from the same image, their internal blur patterns are consistent; while negative samples come from different images and contain different degrees and types of blur, making them naturally distinguishable. This structure allows MCFM to automatically learn the differences between different blur patterns during training.

[0010] 2. Shared encoder characteristics: Input shared encoder , to obtain features Then, it is mapped to the degenerate perception latent space through a fully connected layer to obtain... .

[0011] 3. Obtaining Degradation Priors Using Momentum Contrastive Learning: The dataset obtained in Step 1 is first trained using the MCFM network for 100 epochs. The training objective is to maximize the feature consistency of positive sample pairs within the same blurring mode and minimize the feature similarity between different blurring modes. Finally, the degradation-aware prior output by MCFM is represented as: This prior can characterize the blur intensity information of the image, providing degradation guidance for the subsequent deblurring main network.

[0012] The prior fusion module SCA is used to obtain the degradation-aware prior from the MCFM module. Effectively embedded into features at various scales of the U-Net backbone network. The degradation prior... As image-level vectors, to adapt to the multi-scale features of different levels of U-Net, they are multi-scale expanded to construct prior representations that match the feature sizes at each level: Where H, W and C represent respectively The height, width, and number of channels. The specific fusion method is as follows: in Features of each layer of U-Net For each layer of degradation priors.

[0013] SCA uses priors as feature reweighting signals, enabling the network to perceive the degree of degradation at each layer, thereby more effectively recovering detailed textures and edge structures. This fusion mechanism achieves effective interaction between degraded priors and multi-scale features while ensuring controllable network computational complexity, balancing fusion effect and inference efficiency.

[0014] The multi-level deblurring main network is built based on the U-Net encoder-decoder structure, and its design concept originates from a re-analysis of existing image deblurring baseline networks. While traditional convolutional-based U-Net can alleviate some blurring through multi-scale feature extraction, its receptive field is limited, making it difficult to model long-range spatial dependencies in dynamic blurring. While the pure Transformer structure possesses global modeling capabilities, its computational complexity is high, and it is not stable enough for local texture restoration. To address these issues, this invention adopts a U-Net structure combining convolutional neural networks and Transformers, ensuring computational efficiency while also considering the ability to model local details and long-range dependencies.

[0015] In the main network, the input degraded image x is first passed through a 3×3 convolution. Preliminary feature extraction is performed on the input image to obtain initial shallow features hx. This layer transforms the RGB image into a higher-dimensional feature representation, capturing basic edge, texture, and color information. Simultaneously, a LeakyReLU activation function is applied to introduce non-linearity, enhancing feature representation and preventing gradient vanishing. This computational process can be represented as: The initial shallow features Prior knowledge of external degradation The input prior fusion module SCA performs pixel-level alignment and channel-level weighted integration of backbone features and degradation cues, outputting fused enhanced features. This step explicitly introduces degradation prior information, guiding the network to focus on degradation-related regions and improving sensitivity to complex degradation. This fusion process can be represented as: in This indicates the first fusion module.

[0016] Subsequently, the fused features are optimized layer by layer using three consecutive residual refinement blocks. Each residual block contains two 3×3 convolutional layers, LeakyReLU activation, and residual connections. The purpose of these residual blocks is to progressively refine the features and preserve the original information through residual connections, thus avoiding the degradation problem of deep networks. The computation process of the residual blocks can be represented as follows: .

[0017] After repeating this residual operation three times, the first-scale residual characteristics are obtained. This feature, as shallow high-resolution information, is used for skip connections in the subsequent decoder to achieve multi-scale feature aggregation. The overall formula for this calculation process is expressed as: in Indicates the first A double convolutional structure within each residual.

[0018] Similarly, the second-scale processing involves processing the residual features of the first scale. Downsampling convolution is applied and activated, and then the mid-layer degradation-aware prior is processed by the prior fusion module SCA. By incorporating the features and repeating the same residual refinement operation as the first scale three more times, the second-scale residual features can be obtained. This scale primarily achieves effective capture of regional degradation patterns by expanding the receptive field and injecting mid-level degradation priors.

[0019] A bottleneck layer is set at the bottom of the Unet backbone network, and a StripedTransformer is introduced into this bottleneck layer as the core feature modeling module. The StripedTransformer adopts a striped self-attention mechanism, calculating attention weights along the horizontal and vertical directions of the image respectively, thereby decomposing the two-dimensional global self-attention into a one-dimensional attention calculation with restricted direction. This significantly reduces computational complexity while enabling information interaction across a large spatial region.

[0020] Compared to traditional deblurring methods that rely solely on convolution operators, this mechanism overcomes the limitation of a fixed receptive field, effectively capturing the correlation between distant pixels. Compared to the standard two-dimensional self-attention Transformer structure, it avoids the problem of computational complexity increasing with the square of image resolution, maintaining global modeling capabilities while ensuring computational efficiency. Through this structure, the network can more accurately model the directional motion trajectories and long-distance pixel dependencies commonly found in dynamic blurring. It is suitable for motion blurring with strong directional consistency introduced by platform motion disturbances or high-speed movement of ground targets in UAV security scenarios, thereby improving the deblurring effect in complex dynamic scenes.

[0021] To further illustrate the implementation of the bottleneck layer, let the feature representation input to the third encoding layer be: , The Striped Transformer first generates Query, Key, and Value from the input features, and then performs striped self-attention computation along the horizontal and vertical directions, respectively.

[0022] Horizontal strip attention is defined as: in This represents a one-dimensional strip feature unfolded along the horizontal direction. is the scaling factor for the feature dimension. Vertical strip attention is defined as: in This represents a one-dimensional strip feature that unfolds along the vertical direction.

[0023] Finally, the Striped Transformer fuses the attention results from both directions to obtain the bottleneck layer output features. .

[0024] The decoding end of the deblurring backbone network adopts a multi-level structure corresponding to the encoding end, which is used to restore the spatial resolution of the feature maps step by step and reconstruct a clear image. During the decoding process, multi-scale features from the encoding end are introduced into the decoding end through skip connections to supplement high-frequency detail information and enhance structural consistency.

[0025] The second-level decoder is structurally similar to the corresponding encoder, but the original residual module ResBlock is replaced with a Striped Transformer block, while the other modules remain unchanged. By introducing a striped self-attention mechanism into this decoding layer, the network further enhances its ability to model long-distance spatial dependencies and directional fuzzy structures during the feature reconstruction stage, thereby more effectively eliminating dynamic fuzzy residues.

[0026] The first-level decoder and the first-level encoder adopt a symmetrical design structure, and their module composition is consistent with the corresponding encoding layer. This is used to refine texture and edge information in a high-resolution feature space.

[0027] Furthermore, the first-level decoder and the second-level decoder respectively use skip connections to transmit the features of the corresponding layers from the encoder. and A decoding process is introduced to achieve multi-scale fusion of encoded and decoded features, avoiding the loss of detailed information caused by multiple downsampling. Apart from the differences mentioned above, the remaining decoding layers maintain a symmetrical design with the encoding end in terms of module composition, feature flow, and processing methods, giving the overall network good hierarchical consistency and stability in structure.

[0028] Step 3: Design the joint loss function to optimize the network: A two-stage training strategy is adopted. In the pre-training stage, the momentum contrast feature module (MCFM) is optimized through the InfoNCE loss function. In the joint training stage, the joint loss function consisting of InfoNCE loss, Charbonnier loss, edge loss and contrast loss is used to optimize the entire DeAP network end-to-end. During the pre-training phase, image patches of size 256×256 are randomly cropped from the original training images as training samples to pre-train the Momentum Contrast Feature Module (MCFM). The training phase lasts for 3000 training epochs. The first 100 epochs focus solely on pre-training the MCFM, keeping the other network parameters unchanged. After these 100 epochs, the MCFM and the main network are jointly trained. The InfoNCE loss function based on momentum contrast learning is used during pre-training, and its calculation formula is as follows: in, This represents the query feature vector. Represents the feature vector of positive samples. Represents the feature vector of negative samples. The temperature hyperparameter is set to 0.07 in this embodiment. This represents the number of negative samples. The aforementioned loss function guides the momentum contrast feature module to distinguish between different types of blurring degradation modes.

[0029] After pre-training, the entire de-blurred network undergoes end-to-end joint training. In this stage, a hybrid loss function is used to optimize the network; the overall calculation formula is as follows: in, Indicates InfoNCE loss, The weighting coefficient between the two is 0.1 in this embodiment. , and These represent Charbonnier loss, edge loss, and contrast loss, respectively. and The corresponding weighting coefficients are set to 0.05 and 0.005 respectively in this embodiment.

[0030] Charbonnier loss is used to measure the pixel-level difference between a deblurred image and a truly sharp image; it is an improvement over traditional... The smooth approximation of the loss can improve the overall signal-to-noise ratio of the restored image while ensuring numerical stability.

[0031] Edge loss is used to measure the structural difference between the deblurred image and the real clear image in the edge region. By constraining the image gradient information, the network is guided to pay more attention to the contour, boundary and detailed structure of the image during the restoration process, thereby effectively mitigating the destruction of structural information by dynamic blur.

[0032] The contrastive loss is constructed based on the perceptual feature space. By simultaneously inputting the deblurred image, the true sharp image, and the corresponding blurry image, it constrains the deblurring result to be closer to the sharp image and further away from the blurry image in the feature space. This loss term helps enhance the network's ability to distinguish between blurry and sharp states, enabling the network to obtain more discriminative deblurred feature representations in complex dynamic scenes.

[0033] Step 4: Network performance testing and evaluation: Using the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR), the trained DeAP network is tested using a test set, and the optimal de-blurring model is output.

[0034] The frame rate of the camera module of the security drone in step one ≥240fps, the number of consecutive sharp frames used to synthesize a blurry image. M The value range is 7-13 frames, and M Equivalent exposure time T satisfy The process of synthesizing the blurred image is as follows: the blurred image in the linear domain is composed of continuous... M The time-averaged approximation of the clear frame is then mapped to the observation domain via the camera response function to obtain the final blurred image.

[0035] Specifically, the security drone performs inspections along a preset route within the target area, and its onboard camera module captures video sequences at a high frame rate. The frame rate of the camera module is denoted as... =240fps, exposure time per frame is And satisfy The time interval between adjacent frames is Let the observation frames continuously collected within a certain time period be... ,in The number of consecutive frames used to generate a blurred image. Based on the physical integral model, the generated linear domain blurred image... Approximate by the time average of consecutive instantaneous sharp frames: The linear domain result is passed through the camera response function Mapping back to the observation domain yields the final blurred image. : For the blurred image Paired sharp reference images (ground-truth sharp images) are defined as constituting... Intermediate frames of a continuous frame sequence ,Right now: The timestamp of this intermediate frame is located at the center of the simulated exposure time interval, thus aligning with the temporal domain of the blurred image, making it suitable as a supervisory label for blur-sharp pairing. Meanwhile, the... Equivalent exposure time to the target They have an approximate relationship, that is ,in It can be designed according to the required fuzziness intensity. In the embodiment, take... =240fps, if the expected generation is equivalent to If s is blurred, then it can be selected. Frame size; to generate a stronger blur, the frame size can be increased accordingly. (e.g., 7–13 frames).

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: The realism of the dataset is significantly improved: the training data is synthesized based on the physical integral model of high frame rate continuous clear frame sequence and camera response function, which accurately simulates the real degradation process of "global + local" composite dynamic blur of UAV. The supervision information provided is closer to the actual scene than traditional synthesis methods, laying a reliable foundation for model training. Outstanding ability to distinguish between complex fuzzy patterns: The MCFM module explicitly captures the degradation features of different fuzzy patterns through momentum contrast learning, and realizes discriminative prior modeling of global and local fuzziness. It effectively solves the technical problem that existing methods cannot distinguish between complex fuzzy patterns, and provides precise guidance for targeted defuzzification. Balancing network modeling efficiency and effectiveness: The multi-level U-Net network integrates CNN and Striped Transformer. CNN ensures the ability to model local details, while Striped Transformer decomposes two-dimensional attention into one-dimensional computation. While breaking through the limitations of fixed receptive field and effectively modeling long-distance spatial dependencies, it significantly reduces computational complexity and adapts to the real-time processing needs of drone security scenarios. Strong adaptive feature fusion capability: The SCA module achieves adaptive fusion of degradation perception prior and multi-scale features through multi-scale expansion and channel reweighting mechanism, which improves the network's ability to perceive different degrees of degradation and enhances the restoration effect of detailed texture and edge structure. Comprehensive optimization of sharpness: The joint loss function takes into account pixel-level reconstruction quality (Charbonnier loss), structural detail restoration (edge ​​loss), and blur-sharp state discrimination (contrast loss, InfoNCE loss), significantly improving the sharpness of moving targets and enabling the restored image to meet the stringent requirements of law enforcement evidence collection. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the momentum comparison feature module (MCFM) in a method for declarative analysis of moving targets in dynamic operation scenarios of security drones according to the present invention. Figure 2 This is a schematic diagram of the multi-scale extension of the degradation perception prior in the method for declarative motion targets in dynamic operation scenarios of security drones according to the present invention. Figure 3 This is a schematic diagram of the prior fusion module (SCA) in a method for declarative analysis of moving targets in dynamic operation scenarios of security drones according to the present invention; Figure 4 This is a structural block diagram of the multi-level deblurring main network in a method for clarifying moving targets in dynamic operation scenarios of security drones according to the present invention; Figure 5 This is an overall framework diagram of the De-blurring Network (DeAP) in the method for clarifying moving targets in dynamic operation scenarios of security drones according to the present invention; Figure 6 This is a flowchart of a method for clarifying moving targets in dynamic operation scenarios of security drones according to the present invention.

[0039] Figure 7 The figure shows a qualitative comparison of two methods on typical samples: DeAP (de-dynamic blurring method) and DeblurGAN (an end-to-end de-blurring method based solely on traditional convolutional neural networks).

[0040] Figure 8Figures showing the experimental results of ablation experiments conducted with and without the MCFM module. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Reference Figure 5 As shown, DeAP is used to sharpen a blurred security scene image, resulting in a clear image. (Refer to...) Figure 6 The flowchart shown illustrates a specific implementation method of the present invention. A method for declarating moving targets in a dynamic operation scenario of a security drone includes the following steps: Step 1: Constructing a sharp-blurred pair dataset: High frame rate continuous sharp frame sequences are collected by security drones, and blurred images are synthesized based on the physical integral model of the camera response function. The intermediate frame of the continuous sharp frames is selected as the sharp reference image to construct a sharp-blurred pair dataset containing training and test sets. Traditional blur synthesis methods often employ simple convolutional kernels or random degradation, failing to realistically simulate the complex dynamic blur of UAVs. This patent utilizes a self-built dataset, synthesizing training data through a high-frame-rate continuous clear frame sequence and a physical integral model of the camera response function. Its synthesis mechanism approximates the blur degradation process in real UAV dynamic imaging, providing more realistic supervisory information than traditional synthesis methods.

[0043] This invention employs a self-built dataset. Specifically, a security drone performs an inspection task within a target area along a preset inspection route, and continuously acquires video sequences of the target area at a high frame rate using its onboard camera module. Let the frame rate of the camera module be... In this embodiment =240fps. The exposure time per frame is And satisfy Less than the time interval between two adjacent frames ,in The sequence of continuously acquired observation video frames within a certain time period is denoted as... ,in M This represents the number of consecutive frames used to synthesize a motion-blurred image. Based on the physical-time integral model of the camera imaging process, and ignoring noise and quantization errors, the blurred image in the linear domain... It can be approximated by the time average of the above M clear images over the exposure time, and its mathematical expression is: Subsequently, the linear domain blurring result is processed through the camera response function. Mapped to blurred image B in the observation domain: For the sharp reference image corresponding to the blurred image B, this embodiment selects the middle frame from the continuous frame sequence constituting the blurred image: Because the timestamp of this intermediate frame is located at the center of the simulated exposure time interval, maintaining temporal alignment with the blurred image, it can serve as a supervisory label for the blurred-sharp image pair. Furthermore, the number of consecutive frames M and the equivalent exposure time T satisfy the following relationship: Thus, it can be adjusted M The value of can flexibly control the blur intensity of the synthesized blurred image. In this embodiment, when the frame rate =240fps. If you want to generate a motion-blurred image with an equivalent exposure time of T=1 / 30s, you can select... Frames; when a stronger motion blur effect is needed, the value range of M can be changed accordingly, for example, set to 7–13 frames. The blurred images and sharp images obtained in the above manner are paired as samples to form the training dataset and test dataset for network training and performance evaluation.

[0044] Step 2: Constructing the De-blurring Network DeAP: The De-blurring network includes a Momentum Contrast Feature Module (MCFM), a Prior Fusion Module (SCA), and a multi-level deblurring main network. The Momentum Contrast Feature Module (MCFM) is used to learn degradation perception priors for different blurring modes. The Prior Fusion Module (SCA) is used to adaptively fuse degradation perception priors with multi-scale features. The multi-level deblurring main network is based on a U-Net structure that fuses convolutional neural networks and StripedTransformers, taking into account both local details and long-range spatial dependency modeling. The DeAP (Learning Degradation-aware Prior for Image Deblurring) network adopts a U-Net encoder-decoder structure as the backbone network and integrates a momentum contrast feature module (MCFM) and a prior fusion module (SCA) to improve the model's ability to represent different types of dynamic blur.

[0045] The implementation process of the momentum comparison feature module in step two is as follows: The system obtains query samples and positive samples by cropping from the input blurred image and performing data augmentation, and uses cropped blocks from other images in the batch as negative samples. The query samples, negative samples, and positive samples are input into a shared encoder to extract sample features and map them to the degradation-aware latent space. Based on momentum contrastive learning, the system maximizes the feature consistency of positive sample pairs and minimizes the feature similarity between negative samples, thereby generating a discriminative degradation-aware prior. The degradation-aware prior is then provided to the subsequent deblurring main network to guide the image restoration process.

[0046] To address the challenge of distinguishing between complex dynamic ambiguities, such as those caused by the superposition of global motion of a UAV and local motion of a ground target, this invention proposes a Momentum Contrast Feature Module (MCFM). MCFM can explicitly learn the degradation features of different ambiguity modes, enabling discriminative prior modeling of global and local ambiguities, and providing degradation guidance for subsequent deambiguity networks.

[0047] The Momentum Contrast Feature Module (MCFM) is primarily used to learn a global prior representation for degradation perception from an input blurred image. Its overall structure is as follows: Figure 1 As shown. In this embodiment, the degradation-aware prior learning module MCFM constructs positive and negative sample pairs based on contrastive learning to constrain the degradation features of the input image, enabling the network to explicitly distinguish blur patterns from different sources in the feature space, thereby generating a discriminative degradation-aware prior, and providing the prior to the subsequent deblurring main network to guide the image restoration process.

[0048] To obtain degradation-aware priors using MCFM, positive and negative sample pairs need to be constructed first. Two image patches of size H×W are randomly cropped from the input image x, and random data augmentation operations are applied to these image patches to obtain the query samples. and positive samples Meanwhile, image patches cropped from other different images within the same training batch are defined as negative samples. Because positive samples come from the same image, their internal blur patterns are consistent; while negative samples come from different images and contain different degrees and types of blur, making them naturally distinguishable. This structure allows MCFM to automatically learn the differences between different blur patterns during training.

[0049] The query sample was then... Positive samples and negative samples The input is fed into the shared encoder f(·) for feature extraction, and the corresponding feature representations are obtained respectively. Subsequently, it is mapped to the degenerate perception latent space via a fully connected layer (FC) to obtain... .

[0050] The MCFM is pre-trained using the dataset obtained in step one, preferably with 100 pre-training rounds. During training, the optimization objective is to maximize the feature consistency between positive sample pairs under the same fuzzy degradation mode and minimize the feature similarity between different fuzzy degradation modes. The InfoNCE loss function is introduced to update the network parameters. Finally, the MCFM module outputs the degradation-aware prior, which can be expressed as: This degradation-aware prior can characterize the blur intensity and degradation characteristics of the input image and provide effective degradation guidance information for the subsequent deblurring main network.

[0051] The implementation process of the prior fusion module in step two is as follows: the degradation perception prior is expanded at multiple scales to construct a prior representation that matches the feature size of each layer of the multi-level deblurring main network; the prior at each scale is fused with the network features of the corresponding layer by using a channel reweighting method to output the fused enhanced features.

[0052] A priori fusion module (SCA) was designed to achieve adaptive fusion of degradation-aware priors and multi-scale features, thereby improving the network's ability to perceive different degrees of degradation.

[0053] The prior fusion module SCA is used to integrate the degradation-aware priors. The prior acquisition methods and specific SCA structures embedded in the features at various scales of the U-Net backbone network are as follows: Figure 2 Appendix Figure 3 As shown. To adapt to the multi-scale features of different levels of U-Net, [the following is done / then]... Perform multi-scale expansion to construct prior representations that match the feature sizes at each level: Where H, W and C represent respectively The height, width, and number of channels. The specific fusion method is as follows: in Features of each layer of U-Net For each layer of degradation priors.

[0054] SCA achieves effective interaction between degenerate priors and multi-scale features while ensuring controllable network computational complexity, thus balancing fusion effect and inference efficiency.

[0055] In step two, the encoder of the multi-level deblurring main network includes a first encoding layer, a second encoding layer, and a bottleneck layer, and the decoder includes a second decoding layer, a first decoding layer, and an output layer. The bottleneck layer introduces a StripedTransformer module to decompose the two-dimensional global self-attention into one-dimensional strip attention calculations in the horizontal and vertical directions. The decoder introduces the corresponding level features of the encoder through skip connections, and the residual module at the second-level decoder end is replaced with a StripedTransformer module.

[0056] A multi-level U-Net network structure combining convolutional neural networks and Striped Transformers is proposed, which balances computational efficiency with local detail modeling and long-range spatial dependency modeling. The striped self-attention mechanism decomposes two-dimensional global attention into one-dimensional attention in the horizontal and vertical directions, significantly reducing computational complexity. It is suitable for handling motion blur with strong directional consistency and improves the deblurring capability in complex dynamic scenes.

[0057] Figure 4 The specific structure of the multi-level deblurring main network is shown, which is built based on a U-Net-type encoder-decoder structure. This invention combines convolutional neural networks with Transformer modules within a U-shaped backbone framework, ensuring overall computational efficiency while also considering local detail modeling and long-range dependency modeling capabilities, thus making it more suitable for complex dynamic blurring scenarios.

[0058] In the main network, the input degraded image x first undergoes preliminary feature extraction through a first convolutional layer to obtain initial shallow features hx. This convolutional layer maps the original RGB image to a high-dimensional feature space to extract basic edge, texture, and color information. Simultaneously, a LeakyReLU activation function is introduced to enhance nonlinear expressiveness and alleviate the gradient vanishing problem. This calculation process can be represented as: Subsequently, the shallow features hx are combined with the degradation perception prior. Inputting SCA (Self-Assessment and Acquisition) introduces prior degradation information into the backbone network, guiding the network to focus on degradation-related regions. This fusion process can be represented as: in This indicates the prior fusion module.

[0059] Subsequently, the fused features are optimized layer by layer using three consecutive residual refinement blocks. Each residual block contains two 3×3 convolutional layers, LeakyReLU activation, and residual connections. The purpose of these residual blocks is to progressively refine the features and preserve the original information through residual connections, thus avoiding the degradation problem of deep networks. The computation process of the residual blocks can be represented as follows: After repeating the residual refinement operation three times, the first-scale residual characteristics are obtained. This feature is preserved as a high-resolution shallow feature and used in skip connections during the subsequent decoding stage to achieve multi-scale information fusion. The overall formula for this calculation process is expressed as: in Indicates the first A double convolutional structure within each residual.

[0060] Similarly, the second-scale processing involves processing the residual features of the first scale. Downsampling convolution is applied and activated, and then the mid-layer degradation-aware prior is processed by the prior fusion module SCA. By incorporating the features and repeating the same residual refinement operation as the first scale three more times, the second-scale residual features can be obtained. .

[0061] A bottleneck layer is set at the bottom of the U-Net backbone network, and a StripedTransformer is introduced into this bottleneck layer as the core feature modeling module. The StripedTransformer adopts a striped self-attention mechanism, calculating attention weights along the horizontal and vertical directions of the image respectively, thereby decomposing the traditional two-dimensional global self-attention into a one-dimensional attention calculation with restricted direction. This significantly reduces computational complexity while enabling information interaction across a large spatial region.

[0062] To further illustrate the specific implementation of the bottleneck layer, let the feature representation input to the third encoding layer be F3. The Striped Transformer first generates Query, Key, and Value representations for the input features, and then performs striped self-attention computation along the horizontal and vertical directions, respectively.

[0063] Horizontal strip attention is defined as: in This represents a one-dimensional strip feature unfolded along the horizontal direction. is the scaling factor for the feature dimension. Vertical strip attention is defined as: in This represents a one-dimensional strip feature that unfolds along the vertical direction.

[0064] Finally, Stripformer fuses the attention results from both directions to obtain the bottleneck layer output features. .

[0065] The decoding end of the deblurring backbone network adopts a multi-level structure corresponding to the encoding end, which is used to restore the spatial resolution of the feature maps step by step and reconstruct a clear image. During the decoding process, multi-scale features from the encoding end are introduced into the decoding end through skip connections to supplement high-frequency detail information and enhance structural consistency.

[0066] The second-level decoding end is basically the same as the corresponding encoding end in terms of overall structure, but the original residual module ResBlock is replaced with a Striped Transformer block, while the other modules remain unchanged.

[0067] The first-level decoder and the first-level encoder adopt a symmetrical design, with their module composition consistent with the corresponding encoding layer. This is used to further refine texture and edge details in the high-resolution feature space. Furthermore, the first-level and second-level decoders respectively introduce features from the corresponding layers of the encoding end into the decoding process through skip connections, achieving multi-scale fusion of encoded and decoded features and avoiding the loss of detail information caused by multiple downsampling. Apart from the above differences, the remaining decoding layers maintain a symmetrical design with the encoding end in terms of module composition, feature propagation paths, and processing methods.

[0068] Step 3: Design the joint loss function to optimize the network: A two-stage training strategy is adopted. In the pre-training stage, the momentum contrast feature module (MCFM) is optimized through the InfoNCE loss function. In the joint training stage, the joint loss function consisting of InfoNCE loss, Charbonnier loss, edge loss and contrast loss is used to optimize the entire DeAP network end-to-end. This invention proposes a joint loss function consisting of Charbonnier loss, edge loss, contrast loss, and InfoNCE loss, which takes into account pixel-level reconstruction quality, structural detail restoration, and blur-to-sharp state discrimination ability, and significantly improves the sharpness of moving targets.

[0069] This invention employs a two-stage training strategy to optimize the de-blurring network, with a total training duration of 3000 rounds. The first 100 rounds are dedicated to pre-training the MCFM (Multi-Channel Dynamics Foundation), followed by joint training of the MCFM and the main network. During the pre-training phase, image patches of size 256×256 are randomly cropped from the original training images as training samples, using the InfoNCE loss function. in, This represents the query feature vector. Represents the feature vector of positive samples. Represents the feature vector of negative samples. The temperature hyperparameter is set to 0.07 in this embodiment. This represents the number of negative samples. The aforementioned loss function guides the momentum contrast feature module to distinguish between different types of blurring degradation modes.

[0070] During the joint training phase, a joint loss function is used to optimize the network. The overall calculation formula is as follows: in , and These represent the Charbonnier loss, edge loss, and contrast loss, respectively. Weight coefficients. , and The values ​​are 0.05, 0.005, and 0.1, respectively.

[0071] Charbonnier loss is used to measure the pixel-level difference between a deblurred image and a truly sharp image; it is an improvement over traditional... The smooth approximation of the loss can improve the overall signal-to-noise ratio of the restored image while ensuring numerical stability.

[0072] Edge loss is used to measure the structural difference between the deblurred image and the real clear image in the edge region. By constraining the image gradient information, the network is guided to pay more attention to the contour, boundary and detailed structure of the image during the restoration process, thereby effectively mitigating the destruction of structural information by dynamic blur.

[0073] The contrastive loss is constructed based on the perceptual feature space. By simultaneously inputting the deblurred image, the true sharp image, and the corresponding blurry image, it constrains the deblurring result to be closer to the sharp image and further away from the blurry image in the feature space. This loss term helps enhance the network's ability to distinguish between blurry and sharp states, enabling the network to obtain more discriminative deblurred feature representations in complex dynamic scenes.

[0074] Step 4: Network performance testing and evaluation: Using the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR), the trained DeAP network is tested using a test set, and the optimal de-blurring model is output.

[0075] To verify the effectiveness of the method of this invention, the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) were selected as quantitative evaluation indicators to measure the degree of difference between the restored image and the corresponding real clear image. Specifically, the proposed deblurring method DeAP was quantitatively compared with the end-to-end deblurring method DeblurGAN, which is based solely on traditional convolutional neural networks. Experimental results show that DeAP achieved an average PSNR of 31.16 and an SSIM of 0.902 on the test set, while DeblurGAN achieved an average PSNR of 29.39 and an SSIM of 0.879, indicating that the proposed method has a better restoration effect on dynamically blurred images. Furthermore, Figure 7 Qualitative comparison results of the two methods on typical samples are presented.

[0076] Furthermore, to verify the impact of the Momentum Contrast Feature Module (MCFM) on the overall deblurring performance, ablation experiments were conducted with and without the MCFM module, under the same network structure and training strategy. The experimental results are as follows: Figure 8 As shown in the figure, it can be seen that after introducing the MCFM module, the network achieves better restoration results in complex dynamic blurring scenarios. This indicates that the MCFM module can provide effective degradation-aware prior support for the deblurring main network, thereby improving the overall deblurring performance.

[0077] In summary, the present invention has the following significant technical effects: This invention utilizes a security drone to capture continuous, instantaneous, clear frames at a high frame rate and constructs clear-blurred paired samples based on the camera response function. This ensures that the generated blurred images are highly consistent with the actual drone shooting process in terms of both temporal dimension and imaging mechanism. Compared to existing dataset construction methods based on simple convolutional kernels or random degradation synthesis, the dataset of this invention can realistically reflect the composite dynamic blurring characteristics generated by the superposition of the drone platform motion and the independent motion of ground targets, providing more reliable supervision information for subsequent model training.

[0078] This invention constructs a De-blurring network (DeAP) for drone security scenarios, possessing adaptive modeling capabilities for complex fuzziness. The De-blurring network employs a U-Net structure that integrates convolutional neural networks and Transformers, ensuring computational efficiency while also considering local detail modeling and long-range spatial dependency modeling capabilities. Through multi-level feature extraction and progressive reconstruction mechanisms, the network effectively alleviates the limitations of the receptive field in traditional convolutional networks and the high computational complexity of pure Transformer networks, thereby improving the deblurring stability and robustness in complex dynamic scenarios.

[0079] This invention designs a momentum contrast feature module (MCFM) to explicitly distinguish different dynamic blurring modes. Addressing the common problem in UAV security scenarios of a combined superposition of "global blurring caused by UAV motion" and "local non-uniform blurring caused by ground target motion," this invention introduces a momentum contrast feature module based on momentum contrast learning. By constructing appropriate positive and negative sample pairs and introducing InfoNCE loss, MCFM can automatically learn and distinguish different blur degradation modes in the feature space, providing prior information on degradation for the subsequent deblurring process and effectively overcoming the difficulty of explicitly modeling composite blurring in existing methods.

[0080] This invention proposes a prior fusion module (SCA) to integrate degradation-aware priors into a multi-level network. Addressing the characteristics of degradation priors being image-level vectors while the deblurring backbone network is a multi-scale feature structure, this invention designs and introduces a simplified channel attention mechanism to embed the degradation-aware priors into the features of each layer of the U-Net through channel reweighting. This fusion method, while ensuring controllable computational complexity, effectively integrates the multi-scale degradation prior information extracted by the MCFM module into the corresponding feature layers of the main deblurring network, ultimately achieving a comprehensive optimization of deblurring effect and inference efficiency.

[0081] This invention employs a joint loss function comprised of Charbonnier loss, edge loss, contrast loss, and InfoNCE loss. While maintaining the overall signal-to-noise ratio of the deblurring result, it strengthens the constraints on image structural details and the differences between blurred and sharp states. Specifically, the contrast loss introduces a dual contrast between sharp and blurred images, guiding the network to differentiate between different degradation states in the feature space. This enables the model to obtain more discriminative deblurring feature representations in complex dynamic scenes, thereby improving the sharpening effect on moving targets in drone security scenarios.

[0082] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A method for declaring moving targets in dynamic operation scenarios of security drones, characterized in that: Includes the following steps: Step 1: Constructing a sharp-blurred pair dataset: High frame rate continuous sharp frame sequences are collected by security drones, and blurred images are synthesized based on the physical integral model of the camera response function. The intermediate frame of the continuous sharp frames is selected as the sharp reference image to construct a sharp-blurred pair dataset containing training and test sets. Step 2: Constructing the De-blurring Network DeAP: The De-blurring network includes a momentum contrast feature module, a prior fusion module, and a multi-level deblurring main network; the momentum contrast feature module is used to learn degradation perception priors for different blurring modes, the prior fusion module is used to adaptively fuse degradation perception priors with multi-scale features, and the multi-level deblurring main network is based on a U-Net structure that fuses convolutional neural networks and Striped Transformers, taking into account both local details and long-range spatial dependency modeling; Step 3: Design a joint loss function to optimize the network: A two-stage training strategy is adopted. In the pre-training stage, the momentum contrast feature module is optimized through the InfoNCE loss function. In the joint training stage, a joint loss function consisting of InfoNCE loss, Charbonnier loss, edge loss and contrast loss is used to optimize the entire DeAP network end-to-end. Step 4: Network performance testing and evaluation: Using structural similarity index and peak signal-to-noise ratio, the trained DeAP network is tested using a test set, and the optimal de-blurring model is output.

2. The method for clarifying moving targets in dynamic operation scenarios of security drones according to claim 1, characterized in that: The frame rate of the camera module of the security drone in step one ≥240fps, the number of consecutive sharp frames used to synthesize a blurry image. M The value range is 7-13 frames, and M Equivalent exposure time T satisfy The process of synthesizing the blurred image is as follows: the blurred image in the linear domain is composed of continuous... M The time-averaged approximation of the clear frame is then mapped to the observation domain via the camera response function to obtain the final blurred image.

3. The method for declaring moving targets in dynamic operation scenarios of security drones according to claim 1, characterized in that: The implementation process of the momentum comparison feature module in step two is as follows: The system obtains query samples and positive samples by cropping from the input blurred image and performing data augmentation, and uses cropped blocks from other images in the batch as negative samples. The query samples, negative samples, and positive samples are input into a shared encoder to extract sample features and map them to the degradation-aware latent space. Based on momentum contrastive learning, the system maximizes the feature consistency of positive sample pairs and minimizes the feature similarity between negative samples, thereby generating a discriminative degradation-aware prior. The degradation-aware prior is then provided to the subsequent deblurring main network to guide the image restoration process.

4. The method for clarifying moving targets in dynamic operation scenarios of security drones according to claim 1, characterized in that: The implementation process of the prior fusion module in step two is as follows: the degradation perception prior is expanded at multiple scales to construct a prior representation that matches the feature size of each layer of the multi-level deblurring main network; the prior at each scale is fused with the network features of the corresponding layer by using a channel reweighting method to output the fused enhanced features.

5. The method for clarifying moving targets in dynamic operation scenarios of security drones according to claim 1, characterized in that: In step two, the encoder of the multi-level deblurring main network includes a first encoding layer, a second encoding layer, and a bottleneck layer, and the decoder includes a second decoding layer, a first decoding layer, and an output layer. The bottleneck layer introduces a StripedTransformer module to decompose the two-dimensional global self-attention into one-dimensional strip attention calculations in the horizontal and vertical directions. The decoder introduces the corresponding level features of the encoder through skip connections, and the residual module at the second-level decoder end is replaced with a StripedTransformer module.

6. The method for sharpening moving targets in a dynamic operation scenario of a security drone according to claim 5, characterized in that: The formula for calculating the horizontal strip attention of the Striped Transformer module is defined as follows: in This represents a one-dimensional strip feature unfolded along the horizontal direction. The scaling factor is the feature dimension; the formula for calculating the vertical strip attention of the Striped Transformer module is defined as follows: in This represents a one-dimensional strip feature that unfolds along the vertical direction.

Images