A blind image motion blur removal method based on unsupervised learning

Abstract

This invention belongs to the field of image deblurring technology, specifically relating to a blind image motion blur removal method based on unsupervised learning. First, an unpaired dataset consisting of sharp images and motion-blurred images is created. Second, a generator part of a generative adversarial network (GAN) is constructed: using the motion-blurred images from the unpaired dataset as input, a generator is designed based on the frequency domain characteristics of the motion-blurred images, and the generator is used to generate sharp images. Then, an unsupervised learning framework is designed, incorporating a multi-scale Gaussian filter to eliminate interference from the unpaired training image data in pixel-level contrastive learning. A multi-scale PatchGAN discriminator is used as the decision maker of the adversarial network, using the generated sharp image as input and comparing it with a real sharp image to evaluate the authenticity of the generated sharp image. Finally, a loss function is designed to constrain the aforementioned generator and discriminator to ensure the image deblurring effect. This method integrates the advantages of deep learning and prior methods, and experiments show that it performs superiorly on the test set.

Description

Technical Field

[0001] This invention belongs to the field of image deblurring technology, and mainly relates to a blind image motion blur removal method based on unsupervised learning. Background Technology

[0002] With the continuous improvement of automation in the energetic materials industry, machine vision-based online monitoring technology has become a research hotspot due to its advantages of non-contact operation and high timeliness. However, the effectiveness of these technologies largely depends on image quality. Image blurring is a common problem affecting image quality, caused by a variety of factors, including motion blur caused by the movement of materials during the mixing process of energetic materials.

[0003] For the degradation process of motion-blurred images, image deblurring methods can be divided into non-blind and blind image deblurring methods based on whether the blur kernel is known. Existing blind image deblurring methods can be broadly classified into two categories: optimization model-based methods and deep learning-based methods. Deep learning-based methods adaptively learn the features of sharp images from large-scale image data using convolutional neural networks. This method can be further divided into supervised and unsupervised learning. Supervised learning requires a large amount of paired sharp-blurred training image data, learning a direct mapping from blurry to sharp through convolutional neural networks. However, obtaining such paired data is very difficult in real-world scenarios, often relying on synthesized blurry images, which can easily lead to overfitting the model to a manually generated blur kernel and insufficient generalization ability. In contrast, unsupervised learning only requires unpaired blurry or sharp images, without manual annotation. It can transform blurry images into the distribution of sharp images by learning the inherent laws of image distribution. Unsupervised learning is a commonly used unsupervised learning method.

[0004] The application with application number "202110215750.0" discloses "a blurry image restoration method based on unsupervised generative adversarial networks." This method uses two generators to model the problem of blurry image restoration and the blurring degradation process of clear images, respectively, and introduces a discriminator to cooperate with the corresponding generators for adversarial training. Simultaneously, L2 pixel reconstruction loss and perceptual loss are introduced to model the target loss function.

[0005] The application with application number "202311693318.8" discloses "an unsupervised, lightweight, real-time blind image deblurring method". This method integrates basic elements and residual elements into the generative adversarial network architecture, and proposes a new encoder-decoder structure in combination with lightweight structural design to optimize the generative adversarial network structure. At the same time, it introduces contrastive learning to obtain the latent representation of samples and evaluate similarity, thereby enhancing the generative adversarial network's ability to learn the mapping from blurred images to clear images.

[0006] The above documents share the following common problems: Generative Adversarial Networks (GANs) are prone to pattern collapse and convergence difficulties during training. Furthermore, Document 1 suffers from the problem that combining L2 pixel loss and perceptual loss can lead to over-smoothing and loss of detail in the restored image; while Document 2's lightweight design improves efficiency, the inherent computational overhead of adversarial training remains high. Under complex non-uniform motion blur, cyclic consistency constraints struggle to fully preserve structural information, resulting in artifacts or distortion; and it is highly dependent on the distribution of training data, limiting its cross-domain generalization ability. Summary of the Invention

[0007] This invention provides a blind image motion blur removal method based on unsupervised learning to solve the problems of pattern collapse and convergence difficulties in existing schemes, as well as excessive smoothing of details, structural distortion under complex motion blur, and limited cross-domain generalization ability.

[0008] To achieve the above objectives, the technical solution provided by this invention is: a blind image motion blur removal method based on unsupervised learning, comprising the following steps:

[0009] Step 1: Create an unpaired dataset of real-world images consisting of sharp and motion-blurred images;

[0010] Step 2: Build the generator part of the generative adversarial network to obtain a clear image: Using motion-blurred images in the unpaired dataset as input, design a generator based on the frequency domain characteristics of the motion-blurred images, and use the generator to obtain a clear image;

[0011] Step 3: Design an unsupervised learning framework. In pixel-level contrastive learning, add a multi-scale Gaussian filter to eliminate interference from unpaired training image data. Use a multi-scale PatchGAN decision maker as the decision maker for the adversarial network. The generated clear image is taken as input and compared with the real clear image. The decision maker is used to evaluate the authenticity of the generated clear image.

[0012] Step 4: Design loss functions to constrain and optimize the generators and discriminators designed in Steps 2 and 3 respectively.

[0013] Furthermore, in step two above, the generator's processing flow is as follows: First, the input motion-blurred image is preprocessed to obtain high-frequency and low-frequency components aggregated by the AFAM module; second, these components are used as inputs to the high-frequency component feature extraction network and the low-frequency component feature extraction network in the frequency domain processing network, respectively, for feature extraction; then, the extracted features are aggregated again by the AFAM module; finally, motion blur is removed in the spatial domain by the spatial perception fusion network to generate a clear image.

[0014] Furthermore, in step two above, the fuzzy kernel estimation network design in the preprocessing stage of the generator adopts a multi-scale fusion strategy; firstly, a large-size convolutional kernel is used as the initial processing layer to capture the overall motion trajectory and global fuzzy direction; then, a feature extraction network is constructed based on the residual structure; finally, multi-scale fuzzy kernels are generated and fused through 1×1 convolution and dimensionality transformation operations.

[0015] Furthermore, in step two above, the generator is designed with a multi-scale adaptive feature fusion (AFFM) module and a hybrid scale feedforward network (MSFN) is introduced as a low-frequency component feature extraction network for the frequency domain processing network and a skip connection for the spatial perception fusion network.

[0016] Furthermore, the AFFM module takes the output of each encoder and the k-th level upsampling as input, and uses convolutional layers to combine multi-scale features. Specifically, after feature concatenation and 1×1 convolution, channel attention and spatial attention are applied sequentially through the CBAM module, and then a 3×3 dilated convolution operation is performed. The tensor after convolution is input into the MSFN network to obtain the k+1 level upsampling input.

[0017] Furthermore, in step three above, the unsupervised learning framework first designates the unpaired sharp image and motion-blurred image as positive and negative samples, respectively; then, it uses the VGG-16 network pre-trained on the ImageNet dataset as the embedding space for feature extraction; finally, the multi-scale PatchGAN discriminator uses the extracted feature information of the positive and negative samples to improve the generator's motion blur removal effect through pixel-level contrast learning guided by multi-scale Gaussians.

[0018] Pixel-level contrastive learning representation :

[0019] (1)

[0020] To address the interference caused by inherent differences in content and color information in unpaired data, a multi-scale Gaussian filter is introduced into the unsupervised learning framework design; the multi-scale Gaussian pixel-level contrast perception loss is represented as... :

[0021] (2)

[0022] in, This represents the first [unclear] extracted from a VGG-16 network pre-trained on the ImageNet dataset. Features from hidden layers; the outputs of the 2nd, 3rd, and 5th max-pooling layers were selected as features; This indicates that it has a kernel size of Gaussian filter operator, Indicates the first The weight of the level; and These represent sharp and motion-blurred images randomly selected from an unpaired dataset, respectively. This is the weighting factor.

[0023] Furthermore, step four above is divided into two stages:

[0024] (1) Introduce a loss function into the designed generative adversarial network;

[0025] (2) With minimizing the loss function as the optimization objective, the generator and discriminator are trained alternately and iteratively.

[0026] Furthermore, in step four (1) above, the loss function is expressed as:

[0027] (3)

[0028] in, For GAN loss, For multi-scale Gaussian pixel-level contrast perceptual loss, To prevent the loss of principal component consistency due to severe degradation; and This represents hyperparameters.

[0029] Furthermore, the aforementioned re-degradation-induced principal component consistency loss is expressed as: :

[0030] (4)

[0031] in, For L1 norm loss, This is the regularization loss; and Indicates hyperparameters;

[0032] (5)

[0033] in, Input a motion-blurred image; The fuzzy kernel is obtained by fuzzy kernel estimation during preprocessing. To estimate the fuzzy kernel The Wiener filter is applied to the initial deblurred image to initially remove motion blur.

[0034] Since L1 norm loss may introduce high-frequency artifacts, regularization loss is introduced to encourage smooth transitions between adjacent pixel values; this process is expressed as:

[0035] (6)

[0036] in, This is to obtain a pseudo-blurred image after re-degradation through convolution operations.

[0037] Compared with existing methods, the beneficial effects of the present invention are:

[0038] (1) In the generator preprocessing stage, this invention constructs a fuzzy kernel estimation network for motion fuzziness. This network is specially designed with a learnable multi-scale fuzzy kernel estimation network. It captures the global fuzzy direction through large-size convolutional kernels, extracts deep features through residual structure, and fuses multi-scale information through 1×1 convolution, thus realizing explicit modeling of motion fuzzy kernels. In order to address the adversarial training of generative adversarial networks, this invention constructs a multi-scale PatchGAN discriminator. While taking into account both global structure and local details, it generates multi-scale fuzzy kernels to actively represent the non-uniformity of spatial changes. Furthermore, a dual-branch frequency domain analysis framework is designed to process high-frequency and low-frequency information separately, forming a low-frequency component processing network based on residual learning and a spatial perception fusion network. The AFFM module and MSFN module are used as skip connections to efficiently fuse cross-layer features and finely model multi-scale information.

[0039] (2) This invention designs a pixel-level self-contrast learning method based on multi-scale Gaussian guidance. Through contrastive learning, the restored image is brought closer to the clear image (positive sample) while being pushed away from the motion-blurred image (negative sample), thereby improving the motion blur removal effect. This method ensures that the generated clear image is not easily affected by the inherent information (such as content and color) of the motion-blurred / clear image while maintaining a good deblurring effect.

[0040] (3) This invention proposes a new unsupervised contrastive learning framework, which avoids the dependence of traditional methods on synthetic data and effectively alleviates the domain shift problem by directly using unpaired real blurred and clear images for training.

[0041] (4) This invention first explicitly models motion-blurred images through a multi-scale blur kernel estimation network to obtain multi-scale blur kernels and preliminary features; then, cross-layer feature fusion is enhanced by a dual-branch frequency domain analysis framework and AFFM and MSFN modules, and then fed into a multi-scale PatchGAN discriminator to obtain global and local adversarial feedback; at the same time, a re-degradation guided principal component consistency (RPC) loss is designed, which uses the estimated blur kernel to re-degrade the clear image, constrains the consistency of principal component distribution between the pseudo-blurred image and the original blurred image, and combines multi-scale Gaussian guided pixel-level self-comparison learning to jointly optimize the generator and discriminator, so as to achieve high-quality blind restoration of non-uniform motion blur. Attached Figure Description

[0042] Figure 1 This is a flowchart of the present invention;

[0043] Figure 2 This is a schematic diagram of the generative adversarial network constructed according to the present invention;

[0044] Figure 3 This is a schematic diagram of the overall framework of the frequency domain processing module in this invention;

[0045] Figure 4 This is a schematic diagram of an unsupervised contrastive learning framework proposed in this invention;

[0046] Figure 5 The image shows a comparison of the effects of different unsupervised methods on the test dataset of this invention. In the image, (a) is the input blurred image, (b) is the image with motion blur removed by the USDF method, (c) is the image with motion blur removed by the PCLGAN method, (d) is the image with motion blur removed by the UAUD method, (e) is the image with motion blur removed by this invention, and (f) is the clear image. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] This invention is further described in detail with reference to specific implementation methods and accompanying drawings. The specific steps are as follows:

[0049] Step 1: Create an unpaired dataset of real-world images consisting of sharp and motion-blurred images;

[0050] To address the motion blur issue arising during image acquisition in the process of energetic material mixing, and to ensure the effectiveness of motion blur image restoration in this industrial inspection scenario, this invention creates an energetic material mixing image dataset to support the experimental training of relevant restoration method models, while also adapting to the actual engineering application needs of detection in energetic material mixing scenarios.

[0051] The dataset includes a training set and a test set, covering mixing state data with various modes such as global mixing uniformity and local component aggregation under different mixing times and material mixing ratios. This effectively simulates complex scenarios in actual energetic material mixing processes. The training set and test set are unpaired image data and paired image data, respectively. Both the unpaired training dataset and the paired test dataset include motion-blurred image sets and sharp image sets, denoted as {...} }and{ }

[0052] By constructing an experimental scenario simulating the mixing process of energetic materials, frames are extracted from the video, and then regions of interest are extracted from the video frames to obtain motion-blurred images and clear images of different mixing states of the mixing system, and the image sizes are standardized.

[0053] To ensure that the deviation of paired image pairs at corresponding pixels in the test set remains within a reasonable range, the interval between adjacent video frame sequences of motion paths is set to 10 frames when extracting images from the video. To supplement the mixed image data of global and local motion blur and thus better test the network model, blurred images are generated from the clear images obtained by frame extraction based on random trajectories. The random trajectory generation is achieved by constructing the trajectory of objects in the image using a Markov random process. Based on the position and velocity of the previous point, Gaussian perturbation, impulse perturbation, and inertial components, the position of the next point is randomly determined, and trajectory points are continuously generated iteratively to form a complete trajectory.

[0054] Step 2: Constructing the generator part of the generative adversarial network: Using motion-blurred images from the unpaired dataset as input, a generator is designed based on the frequency domain characteristics of the motion-blurred images. This generator is then used to produce sharp images. Specifically, this includes the following sub-steps:

[0055] 2.1 Construct a generative adversarial network and design a generator specifically for the frequency domain characteristics of motion-blurred images:

[0056] In this invention, the generative adversarial network constructed is described in [reference needed]. Figure 2 The generator includes a preprocessing module, a frequency domain processing module, and a spatial awareness fusion module. The generator's processing flow is as follows: input a motion-blurred image... First, the high-frequency components after polymerization are obtained through preprocessing. and low frequency components They serve as high-frequency component feature extraction networks in the frequency domain processing network, respectively. and low-frequency component feature extraction network The input is then processed to obtain high-frequency components. and low frequency components After feature aggregation via the AFAM module, the feature is finally passed through a spatial perception fusion network. Further motion blur removal in the spatial domain yields the final sharp image. .

[0057] See Figure 2The preprocessing module uses large-size convolutional kernels as the initial processing layer, leveraging their wide receptive field to capture global blur direction information. Subsequently, a feature extraction network is constructed using depthwise separable convolutions and residual block structures. Finally, multi-scale blur kernels are generated and fused through 1×1 convolutions and dimensionality transformations. To address the training instability issue of GANs, spectral normalization is further introduced into the residual blocks to effectively alleviate the gradient explosion problem.

[0058] The preprocessing module's workflow includes the following sub-steps:

[0059] First, for the input motion-blurred image Perform fuzzy kernel estimation;

[0060] Secondly, the input image and the obtained blur kernel are input into a Wiener filter to obtain an image with preliminary motion blur removal. ;

[0061] Then, guided filtering is used to... Processing is performed to obtain smooth low-frequency information. and high frequency components The specific process is as follows:

[0062] (1)

[0063] in, This indicates that a filtering operation is being guided. It is the filtration radius, and Control its fuzziness level. These represent the input image and the guide image, respectively. High-frequency components. for:

[0064] (2)

[0065] After the above operations, the following can be performed respectively: as well as The high-frequency and low-frequency components are separated, and then the attention feature aggregation module (AFAM) is used to aggregate the frequency domain components to obtain the aggregated high-frequency components. and low frequency components .

[0066] See Figure 3 The left half of the frequency domain component processing network: This invention designs a high-frequency component processing module based on U2Net. Low-frequency component processing module based on residual blocks This is used to process the frequency domain components aggregated by the AFAM module. Simultaneously, an AFFM module is designed and an MSFN network is introduced as a skip connection between deep and shallow features.

[0067] The workflow of the frequency domain processing module includes the following sub-steps:

[0068] First, a 4x downsampling method is used to encode the low-frequency components of the input into a low-resolution feature map.

[0069] Subsequently, six residual blocks are used to extract deeper features. The deep features output from the residual blocks are then fused with the corresponding shallow features using the attention-weighted AFFM module, and finally integrated across scales via multi-scale convolutions of the MSFN network, thus constructing skip connections between deep and shallow features. To effectively improve the stability of GAN training, a spectral normalization strategy is introduced into the residual blocks to ensure model continuity and prevent gradient explosion. Furthermore, to further improve generator performance, in... The system introduces a multi-scale feature enhancement module (self-calibrating convolution, SC Conv).

[0070] Each AFFM module takes the output of each encoder and the k-th level upsampled feature as input, and uses convolutional layers to combine multi-scale features. To further enhance the feature fusion module's ability to capture key information, a CBAM attention mechanism is introduced into the AFFM module. Specifically, after feature concatenation and 1×1 convolution, channel attention and spatial attention are applied sequentially through the CBAM module, followed by a 3×3 dilated convolution operation. The tensor after convolution is input into the MSFN network to obtain the (k+1)-th level upsampled input. The MSFN network adopts a dual-branch structure, enabling parallel extraction and fusion of multi-scale features. The input features are first normalized by LayerNorm, and then extracted through 3×3 and 5×5 depthwise separable convolutional branches, respectively. Multiple concatenation and convolution operations further fuse information at different scales. Finally, feature integration is achieved through a 1×1 convolution.

[0071] Finally, a 4x upsampling operation and a 7×7 convolutional layer are used to output the processed low-frequency components. .

[0072] The spatial perception fusion module See Figure 3 The right half is the spatial perception fusion network. This module adopts a classic encoder-decoder architecture, with nine improved residual blocks connected in series for deep feature extraction and fusion. Each residual block contains two 3×3 convolutional layers, instance normalization, ReLU activation function, and unique spatial normalization and SC convolutional modules. Skip connections are used to add the input and output, effectively preserving spatial details and enhancing feature representation. Furthermore, this network interacts with the frequency domain component processing network through AFFM and MSFN modules, receiving high-frequency and low-frequency component information respectively, achieving collaborative optimization of spatial and frequency domain features and improving overall perception performance.

[0073] Working process: The processed high-frequency components are fused using the AFAM module. and low frequency components Obtain the complete features and feed the complete features to Obtain the final motion blur removal result Low-frequency component processing network Spatial perception fusion network The difference is the number of residual blocks.

[0074] Step 3: An unsupervised learning framework was designed. By introducing a multi-scale Gaussian filter to optimize pixel-level contrast learning to eliminate interference from unpaired data, and using a multi-scale PatchGAN discriminator to perform adversarial evaluation between the generated image and the real clear image, the realism of the generated result was improved.

[0075] This invention designs an unsupervised learning framework to address the need to balance global structure and local texture in adversarial training of generative adversarial networks (GANs). Within this framework, adversarial training of GANs requires attention to both the global structure of the image (large receptive field) and the preservation of local edges and textures (small receptive field), employing a multi-scale PatchGAN discriminator. The large-scale discriminator, with a larger receptive field and fewer parameters, is responsible for modeling global information, while the small-scale discriminator, with a smaller receptive field and more parameters, focuses on learning detailed features. During training, the discriminators are compared with generated sharp images and randomly selected unpaired images, and spectral normalization is incorporated to enhance training stability.

[0076] The unsupervised learning framework consists of three core components: a multi-scale Gaussian sample construction module, a shared-weight feature extraction network, and a pixel-level contrast loss optimization module. The multi-scale Gaussian sample construction module generates semantically consistent positive samples and significantly different negative samples using Gaussian transform, providing the model with the data foundation to distinguish effective structures from noise interference. The shared-weight feature extraction network acts as a unified mapper, transforming anchor points and various samples from pixel space to a high-dimensional feature space, ensuring the consistency of feature measurements. The pixel-level contrast loss optimization module drives the network to automatically learn robust features that are insensitive to blur but sensitive to content under unlabeled conditions by "bringing" the distance between positive samples and anchor points and "pushing" the distance between negative samples and anchor points, thereby effectively improving the quality of image deblurring or reconstruction.

[0077] The working process of the unsupervised learning framework includes the following sub-steps:

[0078] First, unpaired sharp images and motion-blurred images are designated as positive and negative samples, respectively;

[0079] Then, the VGG-16 network pre-trained on the ImageNet dataset is used as the embedding space for feature extraction;

[0080] Finally, the discriminator uses multi-scale Gaussian-guided pixel-level contrast learning to improve the generator's motion blur removal performance by utilizing the feature information of extracted positive and negative samples.

[0081] Pixel-level contrastive learning representation :

[0082] (3)

[0083] The multi-scale Gaussian pixel-level contrast perception (GSCP) loss is represented as: :

[0084] (4)

[0085] in, This represents the first [unclear] extracted from a VGG-16 network pre-trained on the ImageNet dataset. Features from hidden layers. The outputs of the 2nd, 3rd, and 5th max-pooling layers were selected as features. This indicates that it has a kernel size of Gaussian filter operator, Indicates the first The weight of the level. and These represent sharp and motion-blurred images randomly selected from the unpaired dataset, respectively. This is the weighting factor.

[0086] Step 4: Design a loss function to constrain and optimize the generator designed in Step 2 and the discriminator designed in Step 3.

[0087] 4.1 Introducing the Designed Loss Function in Generative Adversarial Networks

[0088] The total loss function is expressed as:

[0089] (5)

[0090] in, For GAN loss, For multi-scale Gaussian pixel-level contrast perceptual loss, This leads to a loss of principal component consistency due to severe degradation. and This represents hyperparameters.

[0091] a. GAN loss

[0092] Based on research demonstrating that Least Squares GAN (LSGAN) loss is more effective than traditional GAN ​​loss, ensuring a more stable training process, LSGAN loss was adopted when training the network. The adversarial loss is defined as:

[0093] (6)

[0094] In the formula, This represents the input motion-blurred image. This represents the deblurred image generated by the generator. This indicates a clear image. Among them, and Not a blurry-sharp paired image.

[0095] b. Re-degradation leads to principal component consistency loss.

[0096] Although a designed fuzzy kernel estimation network can provide some kernel prior, the lack of real labels makes it difficult to generalize to real-world fuzzy images. This invention designs a re-degradation guided principal component consistency (RPC) loss. The basic idea is to apply the estimated fuzzy kernel to a clear image after obtaining it, generating a re-degradation pseudo-fuzzy image.

[0097] Specifically, kernel estimation networks are used for blurred images. The input is the estimated fuzzy kernel, and the output is the fuzzy kernel. Subsequently, the estimated fuzzy kernel Applied to the initial removal of motion blur from images The pseudo-blurred image after re-degradation is obtained through convolution operation. In the loss calculation stage, L1 norm loss is introduced to constrain the blurred image. With preliminary restoration of blurred images Principal component consistency. This process is defined as:

[0098] (7)

[0099] Since L1 norm loss can introduce high-frequency artifacts (such as checkerboard patterns), a regularized TVLoss is introduced to suppress high-frequency noise and checkerboard effects in the generated image and encourage smooth transitions between adjacent pixel values. This process is defined as follows:

[0100] (8)

[0101] Therefore, the re-degradation-guided principal component consistency (RPC) loss can be expressed as:

[0102] (9)

[0103] During training, the network parameters are updated based on this loss, thereby improving the accuracy of kernel estimation and providing more targeted prior information for subsequent tasks such as deblurring.

[0104] 4.2 With minimizing the loss function as the optimization objective, the generator and discriminator are trained iteratively and alternately.

[0105] The training dataset constructed in step one is used to perform forward computation on the network to obtain the loss values ​​output by the generator network and the discriminator network. Forward computation refers to inputting the blurred dataset and the clear dataset into the generator and the multi-scale discriminator described in step two, respectively, until the corresponding loss value in step four (1) is calculated. The blurred image is input into the preprocessing module in the generator to perform preliminary deblurring calculation to obtain a preliminary deblurred image. At this time, the re-degradation guided principal component consistency loss in the loss optimization module in step four (1) is calculated. Then, motion blur removal is performed on the preliminary deblurred image to obtain the final clear image. At this time, the multi-scale PatchGAN discriminator will perform discrimination calculation on the final clear image. Finally, the total loss value is calculated according to each loss weight.

[0106] The invention will be explained in detail below with specific experimental examples:

[0107] This invention was compared with the latest unsupervised methods in recent years. Performance metrics (Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)) were used to evaluate the effectiveness of each method. Table 1 shows the performance metrics comparison results of different methods.

[0108] Table 1. Comparison of performance metrics of different methods

[0109]

[0110] Table 1 shows the PSNR and SSIM results of various methods on the test dataset. It can be observed that the present invention has a significant advantage in both evaluation metrics. Specifically, the PSNR is 27.89 dB, an improvement of 1.97 dB compared to the second-best UAAD method, indicating that the proposed method has stronger capabilities in image detail restoration and noise suppression. The SSIM index is 0.811, reaching the technical threshold of 0.8, while the SSIM of the other three comparative methods does not exceed 0.77, indicating that the closer the visual effect is to the real, clear image, the better the motion blur removal performance of the proposed method.

[0111] To more intuitively compare the actual restoration effects of different unsupervised motion blur removal methods, in addition to quantitative index analysis, we further compare and analyze the visual restoration results of each method on the test set. By observing the differences in detail texture, edge contour, structural integrity, and artifact suppression between the clear reference image, the motion-blurred image, and the restored images of different unsupervised methods, we can intuitively reflect the ability of each method to remove realistic and complex motion blur and the level of detail preservation. Considering the high requirements for texture detail in energetic material mixing scenes, we focus on comparing the performance of each method in restoring texture details and eliminating blur distortion, thereby verifying the effectiveness and superiority of the proposed unsupervised deblurring method from a subjective visual perspective. See the example of motion blur restoration. Figure 5 It can be seen that while methods such as USDF, PCLGAN, and UAUD can alleviate image motion blur, their effect on restoring texture details is poor. In contrast, the image removed by this invention has clearer textures and is visually closer to a clear image. A comprehensive comparison of the metrics and motion blur restoration examples demonstrates that this invention achieves excellent results in removing motion blur.

[0112] This invention utilizes unpaired datasets for unsupervised training, saving the significant time spent collecting and creating supervised datasets. It also enables high-quality restoration of blurred images, significantly improving image quality and enhancing the ability to resolve image details. Experimental results for this invention can be found in [link to experimental results]. Figure 5 Figure (a) shows a motion-blurred image with obvious motion blur, Figure (f) shows the corresponding clear reference image, and Figures (b)-(d) show the restoration results of three mainstream unsupervised deblurring methods: USDF, PCLGAN, and UAUD, respectively. From a visual perspective, the motion-blurred image in Figure (a) exhibits severe motion blur, with blurred edges and significant loss of texture details in the energetic material particles, directly affecting the accurate detection of subsequent mixing uniformity. Comparing the three unsupervised methods, the USDF method can only alleviate some motion blur, has limited ability to remove strongly blurred areas, and still has obvious blurring at the particle edges, with insufficient detail recovery; the PCLGAN method improves overall clarity, but still has residual blur, insufficient particle outline sharpness, and slight detail distortion in some areas; the UAUD method achieves better restoration results than the previous two methods, but suffers from overly smoothed particle edges and loss of local texture details. In comparison, the motion blur removal effect of this invention is closest to the clear reference image. Referring to the motion blur removal output of this invention in Figure (e), it can be seen that not only are motion blur and blur distortion in the original image completely eliminated, but the edge contours, texture details, and regional distribution characteristics of energetic material particles are also restored. This result fully verifies the effectiveness and superiority of motion blur removal in energetic material mixing scenarios.

[0113] The above description is a specific illustration of the present invention, and not a limitation thereof. Those skilled in the art can make various equivalent technical solutions without departing from the scope of the present invention; therefore, all equivalent technical solutions should be included within the protection scope of the present invention.

Claims

1. A method for blind image motion blur removal based on unsupervised learning, characterized in that: Includes the following steps: Step 1: Create an unpaired dataset of real-world images consisting of sharp and motion-blurred images; Step 2: Build the generator part of the generative adversarial network to obtain a clear image; Using motion-blurred images from an unpaired dataset as input, a generator is designed based on the frequency domain characteristics of the motion-blurred images, and the generator is used to obtain a clear image. Step 3: Design an unsupervised learning framework. In pixel-level contrastive learning, add a multi-scale Gaussian filter to eliminate interference from unpaired training image data. Use a multi-scale PatchGAN decision maker as the decision maker for the adversarial network. The generated clear image is taken as input and compared with the real clear image. The decision maker is used to evaluate the authenticity of the generated clear image. Step 4: Design loss functions to constrain and optimize the generators and discriminators designed in Steps 2 and 3 respectively.

2. The blind image motion blur removal method based on unsupervised learning according to claim 1, characterized in that: In step two, the generator's processing flow is as follows: First, the input motion-blurred image is preprocessed to obtain high-frequency and low-frequency components aggregated by the AFAM module; second, these components are used as inputs to the high-frequency component feature extraction network and the low-frequency component feature extraction network in the frequency domain processing network for feature extraction; then, the extracted features are aggregated again by the AFAM module; finally, the motion blur is removed in the spatial domain by the spatial perception fusion network to generate a clear image.

3. The blind image motion blur removal method based on unsupervised learning according to claim 1, characterized in that: In step two, the fuzzy kernel estimation network design in the preprocessing stage of the generator adopts a multi-scale fusion strategy. First, a large-size convolutional kernel is used as the initial processing layer to capture the overall motion trajectory and global fuzzy direction. Then, a feature extraction network is constructed based on the residual structure. Finally, multi-scale fuzzy kernels are generated and fused through 1×1 convolution and dimension transformation operations.

4. The blind image motion blur removal method based on unsupervised learning according to claim 1, characterized in that: In step two, the generator is designed with a multi-scale adaptive feature fusion (AFFM) module and a hybrid scale feedforward network is introduced as a skip connection for the low-frequency component feature extraction network of the frequency domain processing network and the spatial perception fusion network.

5. The blind image motion blur removal method based on unsupervised learning according to claim 1, characterized in that: The AFFM module takes the output of each encoder and the k-th level upsampling as input, and uses convolutional layers to combine multi-scale features. Specifically, after feature concatenation and 1×1 convolution, channel attention and spatial attention are applied sequentially through the CBAM module, and then a 3×3 dilated convolution operation is performed. The tensor after convolution is input into the MSFN network to obtain the k+1 level upsampling input.

6. The blind image motion blur removal method based on unsupervised learning according to claim 1, characterized in that: In step three, the unsupervised learning framework first designates unpaired sharp images and motion-blurred images as positive and negative samples, respectively; then, it uses a VGG-16 network pre-trained on the ImageNet dataset as an embedding space for feature extraction; finally, the multi-scale PatchGAN discriminator uses the extracted feature information of positive and negative samples to improve the generator's motion blur removal effect through pixel-level contrast learning guided by multi-scale Gaussians. Pixel-level contrastive learning representation : （1） A multi-scale Gaussian filter is introduced into the unsupervised learning framework design; the multi-scale Gaussian pixel-level contrastive perceptual loss is represented as... : （2） in, This represents the first [unit / item] extracted from a VGG-16 network pre-trained on the ImageNet dataset. Features from hidden layers; the outputs of the 2nd, 3rd, and 5th max-pooling layers were selected as features; This indicates that it has a kernel size of Gaussian filter operator, Indicates the first The weight of the level; and These represent sharp and motion-blurred images randomly selected from an unpaired dataset, respectively. This is the weighting factor.

7. The blind image motion blur removal method based on unsupervised learning according to claim 6, characterized in that: Step four is divided into two stages: (1) Introduce a loss function into the designed generative adversarial network; (2) With minimizing the loss function as the optimization objective, the generator and discriminator are trained alternately and iteratively.

8. The blind image motion blur removal method based on unsupervised learning according to claim 7, characterized in that: In step four (1), the loss function is expressed as: （3） in, For GAN loss, For multi-scale Gaussian pixel-level contrast perceptual loss, To prevent the loss of principal component consistency due to severe degradation; and This represents hyperparameters.

9. A blind image motion blur removal method based on unsupervised learning according to claim 8, characterized in that: The re-degradation-guided principal component consistency loss is expressed as: : （4） in, For L1 norm loss, This is the regularization loss; and Indicates hyperparameters; （5） in, Input a motion-blurred image; The fuzzy kernel is obtained by fuzzy kernel estimation during preprocessing. To estimate the fuzzy kernel The Wiener filter is applied to the initial deblurred image to initially remove motion blur. Since L1 norm loss may introduce high-frequency artifacts, regularization loss is introduced to encourage smooth transitions between adjacent pixel values; this process is expressed as: （6） in, This is to obtain a pseudo-blurred image after re-degradation through convolution operations.

Images