Image deblurring method based on generative adversarial network
Through the image defuzzing method based on the generative adversarial network, combined with the fusion of physical constraints and multi-scale features, the accuracy problem of image defuzzing in complex industrial environments is solved, and high-quality defuzzing effect and stability of fault detection is achieved. It is suitable for complex environments such as mines, tunnels, and coal fields.
Patent Information
- Application Number
- CN202510495979.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-08-08
AI Technical Summary
In complex industrial environments, traditional image defuzzing algorithms are difficult to meet high-precision requirements, and are prone to excessive or insufficient defuzzing, loss of detail or amplification of noise, especially in environments such as mines, tunnels, and coal fields, which affects the accuracy and stability of fault detection.
The image defuzzing method based on a generative adversarial network is adopted, combined with the physical constraint module and the multi-scale feature fusion mechanism, and the alternating training of the generator and the discriminator, using an unsupervised learning framework and atmospheric scattering model, a generator with a dual-branch architecture is used for feature learning and physical modeling, and optimized with local and global discriminators.
Without the need for paired blurred/fuzzy data supervision, the quality of defuzzy images and the adaptability of the industrial environment are improved, the recognizable image and the accuracy of fault detection are enhanced, and the safety of industrial production is ensured.
Smart Images
Figure CN120451001A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine vision image processing, and in particular relates to an image deblurring method based on a generative adversarial network. Background Art
[0002] In complex environments such as mines, tunnels, and coal yards, the images collected by the monitoring system are often affected by factors such as dust, moisture, and changes in light, resulting in a decrease in imaging quality. Especially in the presence of diffuse dust, the image contrast is reduced and edge details are blurred, seriously affecting the accuracy and stability of fault detection.
[0003] Traditional image deblurring algorithms are primarily based on physical models or image enhancement techniques. However, these methods have limitations in complex industrial environments, prone to over- or under-deblurring, loss of detail, or noise amplification, making them difficult to meet the high-precision requirements of industrial visual inspection.
[0004] Deep learning technology has made significant progress in image processing. Generative adversarial networks (GANs) and visual models based on Transformers have demonstrated strong adaptability in deblurring tasks. They can effectively extract image structural information and restore high-quality, unblurred images within an unsupervised learning framework. Summary of the Invention
[0005] The purpose of the present invention is to provide an image deblurring method based on a generative adversarial network, aiming to solve the above-mentioned problems.
[0006] To achieve the above object, the technical solution adopted by the present invention is as follows: An image deblurring method based on a generative adversarial network comprises the following steps: Step 1, data acquisition: Use a binocular camera to collect real-time images of the industrial site, which are used as blurred images I for the proposed model to learn deblurring features; Step 2: Data augmentation: Use public datasets or manually selected industrial scene images to synthesize blurry data, and use unblurred images for auxiliary supervision; Step 3, discriminator pre-training: First pre-train the local discriminator and the global discriminator To learn the unblurred image distribution, local discriminator The global discriminator distinguishes true from false at the image patch level. Determine whether the blurred image J conforms to the distribution of the real unblurred image through the entire image; Step 4: Generate adversarial network training: Input the blurred image I into the generator to generate the deblurred image J, and use the physical constraint module to estimate the transmittance and atmospheric light to ensure that the deblurred image conforms to the atmospheric scattering model; alternately train the generator and the discriminator throughout the process, and fix the discriminator. and the global discriminator , optimize the generator to improve the deblurring quality; fix the generator and optimize the local discriminator and the global discriminator , improve the ability to distinguish true from false; Step 5, effect evaluation: Combine multiple loss functions to ensure that the model generates high-quality deblurred images.
[0007] The data sources are real-time images of industrial sites captured by binocular cameras and the public dataset Reside. The training dataset includes blurred images of real industrial sites, blurred images synthesized by data enhancement, and clear images of the on-site environment.
[0008] In step 3, at the initial stage of training, the true unblurred image and the deblurred image are input into the discriminator to pre-train it, learn the distribution of unblurred images, and perform initial discrimination.
[0009] In step 4, after the discriminator initially learns the feature distribution, the generator is trained and optimized by inputting a blurred image and outputting a blur-free image. The physical constraint branch ensures that the resulting image conforms to physical properties. After multiple update iterations, the capabilities of the generator and discriminator are greatly improved. Ideally, the generator can output a deblurred image with reasonable brightness, contrast, and color distribution, and the discriminator cannot distinguish whether the image is a real image or a training-generated image.
[0010] In step 5, multiple loss functions include Fighting Losses : The performance of the generator and the discriminator is continuously optimized during the game between them; Loss of physical consistency : Constrain the generated image to meet the atmospheric scattering model; Perceptual loss : Use VGG to extract high-level features, improve clarity, and make the generated image close to the real clear image in the feature space; Reconstruction losses : Improve the structural consistency between the deblurred image and the input image.
[0011] The generator has a dual-branch architecture, which is used for feature learning and physical modeling constraints respectively.
[0012] The discriminator is a dual discriminator structure, which improves the naturalness of the deblurred image and enhances the detail quality of the image.
[0013] The discriminator mainly consists of a local discriminator and a global discriminator, which are alternately trained and optimized: the local discriminator based on PanchGAN focuses on recovering details in local areas, and the global discriminator based on CNN structure focuses on judging the entire image. The entire architecture ensures that the generated image has global consistency and no artifacts.
[0014] The generator is mainly composed of the following core modules: Convolutional layer: used for preliminary feature extraction; DehazeFormer Block: Globally models the fuzzy distribution to improve the deblurring effect; Downsampling / upsampling module: multi-scale feature extraction to improve resolution; Multi-scale feature fusion module: integrates deblurring information at different scales; Physical Constraint Module: Modeling based on the atmospheric scattering model to ensure that the results conform to physical properties.
[0015] The multi-scale feature fusion module mainly consists of element summation operation, global average pooling GAP, MLP (Linear-ReLU-Linear), softmax function and splitting operation.
[0016] The DehazeFormer Block mainly consists of scaling layer normalization, reflection filling, linear layer, convolutional layer, and MLP module.
[0017] Compared to existing technologies, this paper proposes an image deblurring algorithm based on a generative adversarial network (GAN), which boasts higher deblurring quality and greater adaptability to industrial environments. This method employs an unsupervised learning framework, combining a generative adversarial network (GAN) with a physical constraint model to guide the generator to adhere to the atmospheric scattering model during the deblurring process. This improves the realism of the deblurred image and reduces color cast and artifacts. Furthermore, the generator utilizes a dehazing attention block (DehazeFormer Block) based on the Swin Transformer architecture for feature extraction. Compared to traditional CNNs, this algorithm has stronger global information modeling capabilities, enabling better recovery of details in distant areas and improving the legibility of industrial field monitoring images. Furthermore, this paper introduces a multi-scale feature fusion (MS-Fusion) mechanism, enabling full interaction between feature information at different scales to more effectively remove blur interference in complex environments. The method also combines local and global discriminators for joint optimization, resulting in improved local detail and overall consistency in the resulting deblurred image. Because an unsupervised training method is adopted, there is no need for paired fuzzy / unfuzzy data for supervised learning, which overcomes the problem of difficulty in obtaining clear images in industrial scenarios. The deblurring model of the present invention has good generalization ability in complex environments such as mines, coal yards, and tunnels, thereby effectively improving the accuracy of fault detection in industrial environments and ensuring industrial production safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a flowchart of the image deblurring algorithm based on generative adversarial networks; Figure 2 This is a framework diagram of the defuzzification algorithm model in the present invention; Figure 3 This is the multi-scale feature fusion module diagram in the generator; Figure 4 Diagram of the dehazing attention module in the generator. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0020] See Figure 1 The present invention discloses an image deblurring method based on a generative adversarial network, the method comprising the following steps: Step 1, Data Acquisition: A binocular camera is used to collect real-time images of industrial sites. These images are often affected by dust, moisture, and other factors, resulting in dust interference. This data is used as the blurred image I for the proposed model to learn deblurring features.
[0021] Step 2, data enhancement: Use public datasets (such as Reside) or manually screened clear industrial site images to synthesize blurry data, and use unblurred images for auxiliary supervision.
[0022] Step 3, discriminator pre-training: First pre-train the local discriminator and the global discriminator To learn the distribution of blur-free images, the local discriminator D1 distinguishes true from false at the image patch level, while the global discriminator D2 judges whether the blurred image J conforms to the distribution of true blur-free images through the entire image.
[0023] Step 4: Generate adversarial network training: The blurred image I is input into the generator to generate the deblurred image J. The physical constraint module is used to estimate the transmittance and atmospheric light to ensure that the deblurred image conforms to the atmospheric scattering model. During the whole process, the generator and the discriminator are trained alternately. The discriminator is fixed and the generator is optimized to improve the deblurring quality. The generator is fixed and the local discriminator is optimized. and the global discriminator , improve the ability to distinguish true from false.
[0024] Step 5, effect evaluation: Integrate multiple loss functions (such as adversarial loss, physical consistency loss, perceptual loss, and reconstruction loss) to ensure that the model generates high-quality deblurred images.
[0025] Multiple loss functions include the following four: Fighting Losses : The performance of the generator and the discriminator is continuously optimized during the game between them; Loss of physical consistency : Constrain the generated image to meet the atmospheric scattering model; Perceptual loss : Use VGG to extract high-level features, improve clarity, and make the generated image close to the real clear image in the feature space; Reconstruction losses : Improve the structural consistency between the deblurred image and the input image.
[0026] See Figure 2 , the image deblurring algorithm architecture diagram based on the generative network against GAN. The algorithm adopts an unsupervised training method and does not require paired blurred / unblurred data for supervised learning, overcoming the problem of difficulty in obtaining clear images in actual situations.
[0027] Furthermore, for the generator model, a dual-branch structure is designed, which is used for feature learning and physical modeling constraints respectively. The main core modules include: 3×3 convolutional layer: performs preliminary feature extraction on the input blurred image; DehazeFormer Block: In the feature learning branch, this module extracts local-global features from the image, models the blur distribution, and improves the deblurring effect. Down-sample / Up-sample module: Changes the image resolution during the multi-scale feature extraction process to implement image feature correlation operations; Multi-scale feature fusion (MS-Fusion) module: integrates deblurred feature information at different scales; Physical Constraint: This module is based on the atmospheric scattering model to ensure that the deblurred image results conform to physical properties.
[0028] See Figure 3 The multi-scale feature fusion module mainly consists of element summation operation, global average pooling, multi-layer perceptron (MLP (Linear-ReLU-Linear)), softmax function and split operation. The multi-scale feature fusion module can capture key information at different scales. Low-level features contain rich texture details, while high-level features contain more global information and contextual relationships. By fusing these features, a clear image can be restored more comprehensively. Secondly, the multi-scale feature fusion module performs feature weight adaptive learning through global average pooling (GAP) and multi-layer perceptron (MLP), enabling the network to dynamically adjust the fusion strategy according to the characteristics of the input image, thereby improving the robustness of the deblurring effect. Furthermore, softmax normalization is used to calculate the feature weight coefficient to ensure that the fusion of features of different scales is more reasonable, avoiding information redundancy or loss, and improving the quality of the deblurred image.
[0029] See Figure 4 The DehazeFormer Block is primarily composed of a scaling layer normalization, reflection padding, linear layer, convolutional layer, and MLP. RescaleNorm performs feature normalization, making the feature distribution more stable and helping to improve the model's training and generalization capabilities. Reflection Padding reduces edge artifacts through mirror padding, making the deblurred image more natural in the boundary area. At the same time, the linear transformation (Linear) of the linear layer and the convolutional layer (Conv) are combined for feature extraction, capturing global features and local spatial information, achieving a fusion of local and global information. Furthermore, Cropping and Affine transforms are used to optimize the feature space, increase feature contrast, and enhance deblurring capabilities.
[0030] Furthermore, another key component of the DehazeFormer Block is the MLP, which consists of a linear layer, a ReLU activation function, and another linear layer. This nonlinear transformation enhances feature representation, resulting in clearer and better contrast in the deblurred image. The entire module utilizes residual connections, which effectively mitigate the vanishing gradient problem in deep network training while preserving input features, making the final deblurring results more stable.
[0031] Furthermore, a dual discriminant structure is adopted for the discriminator structure to perform image quality evaluation from both local and global levels. The local discriminator focuses on the detailed texture and local features of the image to ensure that the deblurred image maintains clarity and naturalness within a small scale range, while the global discriminator is used to evaluate the consistency of the entire image to ensure that the deblurred image does not have problems such as color distortion and structural disorder.
[0032] The working principle and usage process of the present invention are as follows: First, blurry images of industrial sites collected in real time, blurry images synthesized by data enhancement and public clear images are used as training sets and test sets; then the discriminator is pre-trained, and the local discriminator is trained. and the global discriminator To learn the distribution of unblurred images, the local discriminator distinguishes true from false at the image patch level, while the global discriminator judges whether J conforms to the distribution of true unblurred images through the entire image; then the entire network is trained, the blurred image I is input into the generator to generate the deblurred image J, and the physical constraint module is used to estimate the transmittance and atmospheric light to ensure that the deblurred image conforms to the atmospheric scattering model. In the whole process, the generator and discriminator are trained alternately, the discriminator is fixed, the generator is optimized, and the deblurring quality is improved; the generator is fixed and the local discriminator is optimized and the global discriminator Finally, we use a combination of multiple loss functions (such as adversarial loss, physical consistency loss, perceptual loss, and reconstruction loss) to evaluate the effect and ensure that the model generates high-quality deblurred images.
[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. An image deblurring method based on a generative adversarial network, characterized by: The following steps are involved: Step 1, data acquisition: Use a binocular camera to collect real-time images of the industrial site, which are used as blurred images I for the proposed model to learn deblurring features; Step 2: Data augmentation: Use public datasets or manually selected industrial scene images to synthesize blurry data, and use unblurred images for auxiliary supervision; Step 3, discriminator pre-training: First pre-train the local discriminator and the global discriminator To learn the unblurred image distribution, local discriminator The global discriminator distinguishes true from false at the image patch level. Determine whether the blurred image J conforms to the distribution of the real unblurred image through the entire image; Step 4: Generate adversarial network training: Input the blurred image I into the generator to generate the deblurred image J, and use the physical constraint module to estimate the transmittance and atmospheric light to ensure that the deblurred image conforms to the atmospheric scattering model; alternately train the generator and the discriminator throughout the process, and fix the discriminator. and the global discriminator , optimize the generator to improve the deblurring quality; fix the generator and optimize the local discriminator and the global discriminator , improve the ability to distinguish true from false; Step 5, effect evaluation: Combine multiple loss functions to ensure that the model generates high-quality deblurred images.
2. The image deblurring method based on a generative adversarial network according to claim 1, wherein: The data sources are real-time images of industrial sites captured by binocular cameras and the public dataset Reside. The training dataset includes blurred images of real industrial sites, blurred images synthesized by data enhancement, and clear images of the on-site environment.
3. The image deblurring method based on a generative adversarial network according to claim 1, wherein: In step 3, at the initial stage of training, the true unblurred image and the deblurred image are input into the discriminator to pre-train it, learn the distribution of unblurred images, and perform initial discrimination.
4. The image deblurring method based on a generative adversarial network according to claim 1, wherein: In step 4, after the discriminator initially learns the feature distribution, the generator is trained and optimized by inputting a blurred image and outputting a blur-free image. The physical constraint branch ensures that the resulting image conforms to physical properties. After multiple update iterations, the capabilities of the generator and discriminator are greatly improved. Ideally, the generator can output a deblurred image with reasonable brightness, contrast, and color distribution, and the discriminator cannot distinguish whether the image is a real image or a training-generated image.
5. The image deblurring method based on a generative adversarial network according to claim 1, wherein: In step 5, multiple loss functions include Fighting Losses : The performance of the generator and the discriminator is continuously optimized during the game between them; Loss of physical consistency : Constrain the generated image to meet the atmospheric scattering model; Perceptual loss : Use VGG to extract high-level features, improve clarity, and make the generated image close to the real clear image in the feature space; Reconstruction losses : Improve the structural consistency between the deblurred image and the input image.
6. The image deblurring method based on a generative adversarial network according to claim 1, wherein: The generator has a dual-branch architecture, which is used for feature learning and physical modeling constraints respectively.
7. The image deblurring method based on a generative adversarial network according to claim 1, wherein: The discriminator is a dual discriminator structure, which improves the naturalness of the deblurred image and enhances the detail quality of the image.
8. The image deblurring method based on a generative adversarial network according to claim 3, wherein: The discriminator mainly consists of a local discriminator and a global discriminator, which are alternately trained and optimized: the local discriminator based on PanchGAN focuses on recovering details in local areas, and the global discriminator based on CNN structure focuses on judging the entire image. The entire architecture ensures that the generated image has global consistency and no artifacts.
9. The image deblurring method based on a generative adversarial network according to claim 4, wherein: The generator mainly consists of the following core modules composition: Convolutional layer: used for preliminary feature extraction; Dehazing attention block: globally models blur distribution to improve deblurring effect; Downsampling / upsampling module: multi-scale feature extraction to improve resolution; Multi-scale feature fusion module: integrates deblurring information at different scales; Physical Constraint Module: Modeling based on the atmospheric scattering model to ensure that the results conform to physical properties.
10. The image deblurring method based on a generative adversarial network according to claim 9, characterized in that: The multi-scale feature fusion module mainly consists of element summation operation, global average pooling GAP, multi-layer perceptron MLP, softmax function and split operation; the defogging attention module mainly consists of scaling layer normalization, reflection filling, linear layer, convolution layer and MLP module.