Closed-loop control image deblurring method for guiding inverse process solution through fuzzy mechanism

Abstract

The invention discloses a closed-loop control image deblurring method for guiding inverse process solution through a blurring mechanism, and relates to the field of deep learning and image deblurring calculation. The method aims at solving the problem that an existing algorithm depends on pairwise fuzzy-clear image pairs and is difficult to adapt to real image deblurring. The method comprises a mechanism prior guidance module, a deep coding and decoding network, a fuzzy mapping space converter, a fuzzy kernel extractor and a loss cost calculation module. A blurring kernel and a clear image initial value are estimated through a blurring generation principle, a blurred image is reconstructed through a deep coding and decoding network, double-closed-loop cross learning is realized by using a loss cost result, and regularization constraint is introduced to improve optimization stability. According to the method, a large amount of training data is not needed, high-quality deblurring can be achieved in a few-sample scene through closed-loop optimization and a frequency domain information retention mechanism, the method is particularly suitable for high-speed motion blurred scenes such as vehicle-mounted image collection, and the accuracy of subsequent image recognition and analysis is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning and image deblurring computation, and in particular to a closed-loop controlled image deblurring method that uses a fuzzing mechanism-guided inverse process solution. Background Technology

[0002] Environmental interference and motion blur can degrade the quality of acquired images, especially in high-speed motion scenarios. Blurred images captured by image sensors can severely impact subsequent image recognition and analysis tasks. Therefore, image deblurring research is of great significance. When image sensors operate at high speeds (such as in vehicle-mounted image acquisition), the camera's movement within the exposure time leads to blurring, further impairing image processing tasks like target detection. Thus, research on sharpening high-speed moving images is urgently needed. Because the camera's speed variation during acquisition introduces non-uniformity in image blurring (i.e., the blur kernel changes over time), image deblurring is an inverse problem, which is ill-conditioned. To address this special type of blurring, such as that caused by high-speed motion, we innovatively propose an inverse deblurring theory based on the principle of blur generation.

[0003] The forward problem of an image studies the evolutionary process and distribution pattern of things by inferring the effect from the cause, while the inverse problem infers the cause from the effect, exploring the internal laws or external influences of things based on observed phenomena. Because the formation process of the result is complex and mixed with various noises, the solution to the inverse problem is ill-posed, meaning the solution is discontinuous and depends on the measurement data. The core strategy for solving the inverse problem is to introduce regularization methods, that is, to use various prior information to transform the problem in an appropriate form, thereby making the solution to the inverse problem exist, be unique, and be stable.

[0004] Currently, there is no widely applicable image deblurring method. Image degradation is characterized by diversity, coupling, and nonlinearity. Existing research mostly makes specific assumptions for corresponding blur types. The key challenge is how to adaptively perceive blur characteristics based on the formation mechanism of high-speed motion blur—that is, how to analyze motion blur at its source and then perform deblurring based on its formation mechanism. Closed-loop control is a control method that corrects based on the feedback of the controlled object's output. The forward generation and reverse deblurring processes in image deblurring embody this feedback-based adjustment concept. By using this approach to solve the inverse problem of image deblurring in a closed-loop manner, image quality can be continuously optimized until it reaches its optimum. Summary of the Invention

[0005] The purpose of this invention is to provide a closed-loop controlled image deblurring method that uses a fuzziness mechanism-guided inverse process solution to overcome the shortcomings of existing deep learning-based deblurring algorithms that lack paired fuzzy-clear image data during network training, thereby achieving high-quality image deblurring with limited sample data.

[0006] The technical solution of this invention is a closed-loop control image deblurring method guided by fuzzy mechanism and employing inverse process solving. This method comprises the following five model architecture components: a mechanism prior guidance module, a deep encoding / decoding network, and a fuzzy mapping space transformer. Fuzzy kernel extractor And a loss cost calculation module. The composition and function of each module are described below: Module 1: Mechanism Prior Guidance Module Design. This module consists of an initial blur kernel estimation module, an initial sharp image estimation model based on deconvolution operations, and its selection switch group, used to estimate the blur kernel from the original blurred image. The initial value of the fuzzy kernel is estimated based on the fuzzy generation principle. and its corresponding clear image initial value .

[0007] Module 2: Deep Encoder-Decoder Network. This module is the core of the adversarial learning network, mainly composed of UNet network modules with skip connections. In the network structure, the encoder uses downsampling techniques to reduce the spatial size of the feature map while increasing its channel depth, preserving rich image feature information while compressing image data; an embedded wavelet residual module is used to preserve the frequency domain information in the image features, avoiding the loss of key details caused by the downsampling operation of standard convolution; the decoder recovers the compressed feature map through upsampling operations and maps it to the output layer. Skip connections are used in the encoding and decoding layers of the same dimension, which can enhance information flow, alleviate gradient vanishing, and accelerate network training speed.

[0008] Module 3: Fuzzy Mapping Space Transformer This module consists of a deep feature space mapping system and a retrieval interface. The deep feature space mapping system stores the binary mapping relationship between the blurred image, the blur kernel, and the sharp image. The retrieval interface has two input interfaces (blurred image and blur kernel) and one output interface (sharp image).

[0009] Module 4: Fuzzy Kernel Extractor This module consists of a processor that generates blur kernels from blurred-sharp image pairs. After preprocessing, the input blurred-sharp image pairs are processed by a deep learning network to learn the differences between them, thus obtaining a potential blur kernel that effectively represents the relationship between them.

[0010] Module 5: Loss Cost Calculation Module. This module consists of various loss cost calculation functions, used to calculate the model loss cost and feed the cost results back to the fuzzy mapping space transformer. and fuzzy kernel extractor This achieves closed-loop feedback control. The loss cost functions of this invention include a content reconstruction loss function, a frequency reconstruction loss function, and an adversarial loss function.

[0011] The content loss function used is Charbonnier loss, defined as: in, A blurry image representing reality; Represents a composite blurred image; It is a constant term; and These represent the height and width of the image, respectively.

[0012] Frequency reconstruction loss is defined as: in, This indicates the Fast Fourier Transform operation; Represents a truly blurred image; It is a reconstructed, blurry image.

[0013] To reduce the adversarial loss caused by alternating learning in deep encoder-decoder networks, adversarial loss is introduced. The definition is as follows: in This represents the probability that a sample generated by the fuzzy mapping space transformer is identified as true. This represents the log-likelihood of the samples generated by the fuzzy mapping space transformer.

[0014] Combining the above three loss functions, the network training loss function in this paper is defined as follows: in, It is the content reconstruction loss function; It is the frequency reconstruction loss function; It is about combating losses; These are the weighting coefficients for frequency loss. These are weighting coefficients that counteract loss, and they are all adjustable hyperparameters.

[0015] In the deblurring process, introducing regularization constraints to the loss function can enhance the stability of the optimization process and improve the quality of the restored image. This involves adjusting the blur kernel parameters. L2 norm regularization constraints are used. In the estimation of potentially sharp images, this paper introduces a hyper-Laplacian prior as a regularization constraint to promote the sparsity of the solution and suppress the interference of noise and outliers. After introducing this prior term, the objective loss function can be expressed as: in, and These are the derivative operators in the horizontal and vertical directions, respectively; It is an adjustable hyperparameter.

[0016] The specific steps of the closed-loop control image deblurring method guided by fuzzy mechanism and solved by inverse process according to the present invention are as follows: Step 1: Input the blurred image. Guided by the prior mechanism of Module 1, construct the fuzzy kernel extraction function and the fuzzy mapping space transformation function according to the principle of fuzzy generation. Then, the initial values ​​of the fuzzy kernel are calculated sequentially using the prior model of the mechanism. and its corresponding clear image initial value .

[0017] Step 2: Select the prior module for the open / closed connection mechanism, and set the initial value of the fuzzy kernel. and clear image initial value The inputs are respectively fed into module 3, the fuzzy mapping space transformer. And Module 4 Fuzzy Kernel Extractor The corresponding clear image and blur kernel are obtained by mapping and operation with the blurred image, respectively.

[0018] Step 3: Apply fuzzy mapping space transform Output sharp image and blur kernel extractor The output blur kernel is input into the module 2 deep encoder-decoder network, and the reconstructed blur image is obtained through deep learning.

[0019] Step 4: Input the reconstructed blurred image and the input blurred image into module 5, the loss cost calculation module, calculate the model loss cost through the loss cost function, and start the closed-loop cross-learning mode.

[0020] Step 5 ( Figure 1 (Mid-loop module 1): Switch the input selection switch 1 of the fuzzy kernel extractor to the feedback channel, input the loss cost result to the fuzzy kernel extractor, and adjust the fuzzy kernel extractor. After the network parameters are learned, repeat steps 3, 4, and 5 until the loss cost is minimized.

[0021] Step 6 ( Figure 1 (Middle closed loop 2 module): Switch the input selection switch 2 of the fuzzy mapping space transformer to the feedback channel, input the loss cost result to the fuzzy mapping space transformer, and update the fuzzy mapping space transformer. After learning the feature mapping relationship parameters, repeat steps 3, 4 and 6 until the loss cost is minimized.

[0022] Step 7: Repeat steps 5 and 6 alternately until the loss cost stabilizes or the model accuracy requirement is met, then stop the calculation and output the corresponding clear image.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Overcoming the limitation of few sample data: This invention does not rely on a large number of pairs of blurred-clear images for network training. Through the initial value estimation and closed-loop optimization mechanism guided by the blurring mechanism, high-quality image deblurring can be achieved in scenarios with few sample data, solving the problem of strong dependence on training data in existing deep learning deblurring algorithms.

[0024] 2. Closed-loop control achieves dynamic optimization: Through a dual closed-loop feedback mechanism (fuzzy kernel extractor closed loop + fuzzy mapping space transformer closed loop), the network parameters and feature mapping relationship are continuously adjusted using the loss cost results to achieve dynamic iterative optimization of the deblurring process, ensuring that the output image quality gradually approaches the optimal state.

[0025] 3. Regularization enhances stability and anti-interference capability: L2 norm regularization is applied to the fuzzy kernel parameters, and hyper-Laplacian prior constraints are introduced for the potentially clear image, which effectively enhances the stability of the optimization process, suppresses noise and outlier interference, and improves the robustness of the restored image.

[0026] 4. Precisely preserves image features: The wavelet residual module embedded in the deep encoder-decoder network can effectively preserve the image frequency domain information and avoid the loss of key details caused by downsampling; the skip connection design between the encoder and decoder enhances information flow, alleviates the gradient vanishing problem, accelerates network training, and improves deblurring accuracy.

[0027] 5. Adaptive to complex fuzzy scenes: The model is built based on the fuzzy generation mechanism. It adaptively perceives fuzzy features through fuzzy kernel extraction function and fuzzy mapping space transformation function. It is especially suitable for non-uniform fuzzy scenes caused by high-speed motion, and solves the limitations of existing methods for specific fuzzy types. Attached Figure Description

[0028] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is the network architecture of the defuzzified closed-loop control model in an embodiment of the present invention. Detailed Implementation

[0030] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific embodiments. However, the scope of protection of this invention is not limited to the content described.

[0031] Example 1: This example presents a method for deblurring closed-loop control images using a fuzzy mechanism-guided inverse process solution. The network architecture of the deblurred closed-loop control model is as follows: Figure 1 As shown, the specific steps are as follows: Step 1: Based on the generation mechanism of images acquired by the camera under high-speed motion, the blur kernel extraction function and the blur mapping space transformation function are instantiated as follows: Constructing the edge matrix ,in The input blurred image is input through the mechanism prior guidance module in module 1, and the initial value of the blur kernel is calculated sequentially according to the aforementioned mechanism prior model. and its corresponding clear image initial value .

[0032] Step 2: Instantiate the fuzzy mapping space transformer and fuzzy kernel extractor Choose the CNN deep learning model, with the following model parameters: The fuzzy mapping space transformer uses a simple 5-layer dual-input residual network with dual input ports for both the blurred image and the convolution kernel, and the convolution output is the corresponding sharp image; the fuzz kernel extractor uses a simple 5-layer dual-input residual network with dual input ports for both the blurred image and the sharp image, and the convolution output is the corresponding fuzz kernel.

[0033] We will select the prior module for the open-loop connection mechanism and set the initial value of the fuzzy kernel. and clear image initial value The inputs are respectively fed into module 3, the fuzzy mapping space transformer. And Module 4 Fuzzy Kernel Extractor The corresponding clear image and blur kernel are obtained by mapping and operation with the blurred image, respectively.

[0034] Step 3: Instantiate the deep encoder-decoder network, with the following model parameters: The encoder consists of five convolutional layers and five wavelet residual modules, with each network layer adding a LeakyReLU activation function layer. The convolutional layers use a 4×4 kernel with a stride of 2 for downsampling; the wavelet residual modules contain one wavelet transform, two convolutional layers with 1×1 kernels, and one wavelet inverter. The decoder performs upsampling step-by-step through transposed convolutional layers in a manner matching the encoder, and then makes skip connections with the corresponding layer features of the encoder to form a deep encoder-decoder network.

[0035] Fuzzy mapping space transformer Output sharp image and blur kernel extractor The output blur kernel is input into the module 2 depth encoder-decoder network, and the reconstructed blur image is obtained through network learning.

[0036] Step 4: Instantiate the loss cost model and set the loss cost function parameters as follows: when the number of iterations is less than 1 / 2 of the preset number of iterations, , , When the number of iterations is greater than 1 / 2 of the preset number of iterations, , , Frequency loss weighting coefficient Loss resistance weighting coefficient .

[0037] The reconstructed blurred image and the input blurred image are input together into module 5, the loss cost calculation module. The model loss cost is calculated through the loss cost function, and the closed-loop cross-learning mode is started.

[0038] Step 5 ( Figure 1 (Mid-loop module 1): Switch the input selection switch 1 of the fuzzy kernel extractor to the feedback channel, input the loss cost result to the fuzzy kernel extractor, and adjust the fuzzy kernel extractor. After the network parameters are learned, repeat steps 3, 4, and 5 until the loss cost is minimized.

[0039] Step 6 ( Figure 1 (Middle closed loop 2 module): Switch the input selection switch 2 of the fuzzy mapping space transformer to the feedback channel, input the loss cost result to the fuzzy mapping space transformer, and update the fuzzy mapping space transformer. After learning the feature mapping relationship parameters, repeat steps 3, 4 and 6 until the loss cost is minimized.

[0040] Step 7: Repeat steps 5 and 6 alternately until the loss cost stabilizes or the model accuracy requirement is met, then stop the calculation and output the corresponding clear image.

[0041] The main technical features, basic principles, and related advantages of the present invention have been described above. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the concept or basic characteristics of the invention. Therefore, the above-described embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

[0042] Furthermore, it should be understood that although this specification describes various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A closed-loop control image deblurring method of blur mechanism guided inverse process solving, characterized in that, Includes the following steps: Step 1: Input the blurred image into the mechanism prior guidance module, and construct the blurred kernel extraction function and the blurred mapping space transformation function according to the blurred generation principle: ， ， Constructing the edge matrix ,in , ， The initial value of the fuzzy kernel is calculated. and the corresponding clear image initial value ; Step 2: Set the initial value of the fuzzy kernel using the selection switch. Input fuzzy mapping space transformer The initial value of the clear image Input Fuzzy Kernel Extractor The blurred images are then mapped and processed to obtain corresponding candidate clear images and candidate blur kernels. Step 3: Input the candidate sharp image and candidate blur kernel into a deep encoder-decoder network to reconstruct the blur image through deep learning; Step 4: Input the reconstructed blurred image and the input blurred image into the loss cost calculation module, calculate the model loss cost through the preset loss cost function, and start the closed-loop cross-learning mode; Step 5: Switch the input selection switch of the fuzzy kernel extractor to the feedback channel, and input the loss cost result into the fuzzy kernel extractor. Adjust its network parameters and repeat steps 3, 4 and 5 until the loss is minimized; Step 6: Switch the input selection switch of the fuzzy mapping space transformer to the feedback channel, and input the loss cost result into the fuzzy mapping space transformer. And update its feature mapping relationship parameters, repeating steps 3, 4 and 6 until the loss cost is minimized; Step 7: Repeat steps 5 and 6 until the loss cost stabilizes or reaches the preset model accuracy requirement, and output the final clear image; Among them, the fuzzy kernel parameters The L2 norm is used for regularization constraints, and the hyper-Laplacian prior is introduced as a regularization constraint for the estimation of potentially sharp images.

2. The method according to claim 1, characterized in that, The mechanism prior guidance module consists of an initial blur kernel estimation module, an initial sharp image estimation model based on deconvolution operation, and a selection switch group, used to estimate the blur kernel from the original blur image. The initial value of the fuzzy kernel is estimated based on the fuzzy generation principle. and its corresponding clear image initial value The selection switch group is used to control the output path of the initial value of the blur kernel and the initial value of the sharp image.

3. The method according to claim 1, characterized in that, The deep encoding / decoding network is a UNet network with skip connections, including an encoder, an embedded wavelet residual module, and a decoder. The encoder reduces the spatial size of the feature map and increases the channel depth through downsampling technology. The wavelet residual module is used to preserve the frequency domain information of the image. The decoder recovers the feature map through upsampling operation and maps it to the output layer. The same dimension layers of the encoder and decoder achieve information exchange through skip connections.

4. The method according to claim 1, characterized in that, The fuzzy mapping space transformer It consists of a deep feature space mapping system and a retrieval interface; the deep feature space mapping system stores the binary mapping relationship between the blurred image, the blurred kernel, and the sharp image; the retrieval interface includes a blurred image input interface, a blurred kernel input interface, and a sharp image output interface.

5. The method according to claim 1, characterized in that, The fuzzy kernel extractor It is a 5-layer dual-input residual network with a blurred image input port and a sharp image input port. After preprocessing the input blurred-sharp image pairs, it learns the difference between the two through deep learning and outputs a blurred kernel that represents their relationship.

6. The method according to claim 1, characterized in that, The loss cost calculation module includes content reconstruction loss function, frequency reconstruction loss function, and adversarial loss function, which are used to calculate the model loss cost and feed the cost results back to the fuzzy mapping space transformer. And fuzzy kernel extractor This enables closed-loop feedback control.

7. The method according to claim 6, characterized in that: The content loss function used is Charbonnier loss, defined as: ， in, A blurry image representing reality; Represents a composite blurred image; It is a constant term; and These are the height and width of the image, respectively; Frequency reconstruction loss is defined as: ， in, This indicates the Fast Fourier Transform operation; Represents a truly blurred image; It is a reconstructed, blurry image; To reduce the adversarial loss caused by alternating learning in deep encoder-decoder networks, an adversarial loss function is introduced. The definition is as follows: ， in This represents the probability that a sample generated by the fuzzy mapping space transformer is identified as true. This represents the log-likelihood of samples generated by the fuzzy mapping space transformer. Combining the above three loss functions, the overall loss function for network training is defined as: ， in, It is the content reconstruction loss function; It is the frequency reconstruction loss function; It is about combating losses; These are the weighting coefficients for frequency loss. It is a weighting coefficient to counteract losses.

8. The method according to claim 7, characterized in that, The constraint terms corresponding to the hyperLaplace prior satisfy: ， in, and These are the derivative operators in the horizontal and vertical directions, respectively; It is an adjustable hyperparameter.

9. The method according to claim 3, characterized in that, The encoder includes five convolutional layers and five wavelet residual modules. Each network layer has a LeakyReLU activation function layer. The convolutional layers use a 4×4 kernel and a stride of 2 for downsampling. The wavelet residual module includes one wavelet transform, two 1×1 kernel convolutional layers, and one wavelet inverter. The decoder upsamples progressively through transposed convolutional layers and makes skip connections with the corresponding layer features of the encoder.

10. The method according to claim 7, characterized in that, The parameters of the loss cost function satisfy the following condition: when the number of iterations is less than 1 / 2 of the preset number of iterations, , , ; When the number of iterations is greater than 1 / 2 of the preset number of iterations, , , Frequency loss weighting coefficient Loss resistance weighting coefficient .

Images