Method, device and equipment for removing motion blur of optical images based on blur kernel correction

The fuzzy kernel correction network converts motion blur images into Gaussian blur images, and combines the degaussian blur network for defuzzing processing, which solves the generalization performance of the existing blind demotion blur methods and achieves better demotion Fuzzy effect and generalization ability.

CN118691504BActive Publication Date: 2024-11-19NAT UNIV OF DEFENSE TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411171781.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-19
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

The existing blind demotion fuzzing method based on deep learning has insufficient generalization performance, especially when dealing with unknown fuzzy scenarios, and the limitations of the training data set and overfitting problems lead to poor generalization of the model.

Method used

Using the optical image demotion method based on fuzzy kernel correction, the motion blur image is converted into Gaussian blur image through the fuzzy kernel correction network, and the defuzzy network is used for defuzzy processing, and the feature extraction unit and the fuzzy kernel correction unit are used to generate blur The kernel corrects the kernel, performs convolution operations on the image, and sums it with the Gaussian fuzzy kernel library to achieve defuzzing.

Benefits of technology

The generalization performance of the demotion blur method is improved, and the demotion blur performance can be shown in practical applications, avoiding the performance degradation of unknown blurred images, and simplifying the modeling and removal process of the fuzzy core.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118691504B_ABST
    Figure CN118691504B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device and apparatus for deblurring an optical image based on blur kernel correction, which first uses a blur kernel correction network to perform blur kernel correction on an optical image, converts an optical image with motion blur into a Gaussian blur image, and then deblurs the Gaussian blur image to obtain a deblurred optical image. The method has good generalization and has high deblurring performance in practical applications.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of image deblurring, and in particular to a method, device and apparatus for deblurring optical images based on blur kernel correction. Background Art

[0002] With the development of image processing technology, deblurring technology has emerged. Deblurring technology can effectively restore image blur caused by camera shake, object motion, etc., making the image clearer and more accurate. This technology has played an important role in various fields, such as security monitoring, medical imaging, satellite remote sensing, etc., improving the quality of images and the accuracy of information acquisition.

[0003] An important application scenario of deblurring technology is deblurring of optical images. Optical image deblurring technology aims to restore image blur caused by camera lens shake during exposure or high-speed motion of the object being photographed. Deblurring technology models the blurring process as follows:

[0004] ;

[0005] Among them, motion blurred image By clear image Convolution blur kernel Add Gaussian noise Existing deblurring methods are mainly divided into traditional methods based on modeling and methods based on deep learning. Traditional methods are non-blind deblurring methods, which require the blur kernel to be obtained in advance through direct prediction and other methods. , given the known blur kernel The clear image is solved based on modeling. The deep learning-based methods include blind deblurring methods and non-blind deblurring methods. The non-blind deblurring method based on deep learning is similar to the traditional method, but the clear image is solved through a neural network. The blind deblurring method based on deep learning has a blur kernel. End-to-end solution for converting motion blurred images to clear images is achieved under unknown conditions.

[0006] Non-blind deblurring methods require a pre-blur kernel Therefore, it is difficult to apply in real scenes where the blur kernel is complex and difficult to solve. The blind deblurring method based on deep learning is the mainstream of the current deblurring method. Existing blind deblurring methods based on deep learning are often supervised training through paired motion blur-clear image pairs. However, when the neural network model obtained by the supervised training method is used for deblurring, it often has the problem of poor generalization performance, that is, it has good deblurring performance for the blur existing in the training data, but its performance is greatly reduced for the deblurring of unknown blur. Summary of the invention

[0007] Based on this, it is necessary to provide an optical image motion deblurring method, device and equipment based on blur kernel correction with good generalization performance to address the above technical problems.

[0008] An optical image motion blur removal method based on blur kernel correction, the method comprising:

[0009] Acquiring an optical image with motion blur;

[0010] The optical image is subjected to blur kernel correction by using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector. The optical image is subjected to a convolution operation by using the blur kernel correction kernel to obtain the Gaussian blurred image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0011] The Gaussian blurred image is deblurred to obtain a deblurred optical image.

[0012] In one embodiment, in the blur kernel correction unit:

[0013] The length of the feature vector is equal to the number of Gaussian blur kernels in the Gaussian blur kernel library.

[0014] In one embodiment, the feature extraction unit is constructed by a convolutional neural network and an attention mechanism.

[0015] In one embodiment, when training the blur kernel correction network:

[0016] Acquire a sample set, wherein the sample set includes a plurality of motion blurred sample images and corresponding clear sample images, and perform a convolution operation on each of the clear sample images with a Gaussian blur kernel to obtain a corresponding Gaussian blurred label image;

[0017] The motion blurred sample image and the corresponding Gaussian blurred label image are used as a set of training sample pairs, and a training sample set is constructed according to multiple sets of training sample pairs;

[0018] Inputting a set of training sample pairs in the training sample set into the blur kernel correction network to obtain a Gaussian blurred prediction image corresponding to the motion blurred sample image;

[0019] The loss function is calculated according to the Gaussian blurred prediction image and the Gaussian blurred label image, and the parameters in the blur kernel correction network are updated according to the calculation results until convergence, so as to obtain a trained blur kernel correction network.

[0020] In one embodiment, a de-Gaussian blur network is used to deblur the Gaussian blurred image.

[0021] In one embodiment, a motion blur removal network is constructed based on the blur kernel correction network and the Gaussian blur removal network;

[0022] The optical image with motion blur is input into the de-motion blur network to obtain the de-blurred optical image.

[0023] In one embodiment, when training the de-motion blur network:

[0024] Firstly, the blur kernel correction network is pre-trained to obtain a pre-trained blur kernel correction network;

[0025] The pre-trained blur kernel correction network and the Gaussian blur removal network are then trained to obtain a trained motion blur removal network.

[0026] The present application also provides an optical image motion blur removal device based on blur kernel correction, the device comprising:

[0027] A blurred image acquisition module, used for acquiring an optical image with motion blur;

[0028] A Gaussian blur conversion module, used for performing blur kernel correction on the optical image by using a blur kernel correction network, and converting the optical image with motion blur into a Gaussian blur image, wherein the blur kernel correction network includes a feature extraction unit and a blur kernel correction unit, wherein the feature extraction unit performs feature extraction on the optical image to obtain a feature vector related to the motion blur kernel, wherein the blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector, and performs a convolution operation on the optical image by using the blur kernel correction kernel to obtain the Gaussian blur image, wherein in the blur kernel correction unit, according to the feature vector, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel;

[0029] The deblurring enhancement module is used to deblur the Gaussian blurred image to obtain a deblurred optical image.

[0030] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Acquiring an optical image with motion blur;

[0032] The optical image is subjected to blur kernel correction by using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector. The optical image is subjected to a convolution operation by using the blur kernel correction kernel to obtain the Gaussian blurred image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0033] The Gaussian blurred image is deblurred to obtain a deblurred optical image.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0035] Acquiring an optical image with motion blur;

[0036] The optical image is subjected to blur kernel correction by using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector. The optical image is subjected to a convolution operation by using the blur kernel correction kernel to obtain the Gaussian blurred image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0037] The Gaussian blurred image is deblurred to obtain a deblurred optical image.

[0038] The above-mentioned optical image de-motion blurring method, device and apparatus based on blur kernel correction firstly uses a blur kernel correction network to perform blur kernel correction on the optical image, converts the optical image with motion blur into a Gaussian blur image, and then deblurs the Gaussian blur image to obtain a deblurred optical image. The method has good generalization and has high de-motion blurring performance in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1It is a schematic diagram of a flow chart of an optical image motion blur removal method based on blur kernel correction in one embodiment;

[0040] Figure 2 A schematic diagram of the structure of a fuzzy kernel correction network in one embodiment;

[0041] Figure 3 is a schematic diagram of blur kernels included in a Gaussian blur kernel dictionary in one embodiment;

[0042] Figure 4 A schematic diagram of a network structure of a feature extraction unit in one embodiment;

[0043] Figure 5 A schematic diagram of a training process of a motion blur removal network in one embodiment;

[0044] Figure 6 This is a schematic diagram of an image after de-motion blurring using this method in an experiment;

[0045] Figure 7 is a structural block diagram of an optical image motion blur removal device based on blur kernel correction in one embodiment;

[0046] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] When deblurring motion-blurred optical images, motion blur can be caused by a variety of factors, including camera shake, object motion, and simultaneous camera and object motion. Each factor may cause blur in different directions, speeds, and degrees. This diversity makes it difficult to cover all possible motion blur scenarios during the training process of learning-based deblurring methods, resulting in poor performance of the model when processing new blurred images outside the training set. At the same time, the blur kernel of motion blur is usually complex and difficult to accurately model. The blur kernel depends not only on the direction and speed of motion, but also on the lighting and object texture at the time of shooting. If the blur kernel in the training data set cannot fully represent various situations in the real world, the generalization of the model will be limited. In addition, due to the limitations of the data set and overfitting problems during the training process, the trained neural network will have generalization problems.

[0049] In response to the above problems, Figure 1As shown, a method for deblurring an optical image based on blur kernel correction is provided, comprising the following steps:

[0050] Step S100, acquiring an optical image with motion blur.

[0051] Step S110, using a blur kernel correction network to perform blur kernel correction on the optical image, and converting the optical image with motion blur into a Gaussian blurred image.

[0052] Step S120, deblurring the Gaussian blurred image to obtain a deblurred optical image.

[0053] In this embodiment, the existing deblurring methods often have the problem of poor generalization performance. The deblurring performance is good for the blur existing in the data used during training, but the deblurring performance for blurred images with unknown blur is greatly reduced. By converting motion blur into Gaussian blur, the mapping of blurred images with unknown blur to blurred images with known blur is achieved. Compared with directly inputting the unknown blurred image into the deblurring module, inputting the converted Gaussian blurred image into the deblurring module can obtain a better deblurring effect.

[0054] In step S100 , the blur existing in the optical image to be deblurred is caused by motion.

[0055] In step S110, the optical image is subjected to blur kernel correction using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. In this way, the motion blur with complex blur kernel, uncertain directionality and unpredictable properties is converted into a Gaussian blur with simple blur kernel, symmetry and easy modeling characteristics, and the motion blur that is difficult to remove is converted into a Gaussian blur that is relatively simple to deblur, thereby achieving the purpose of removing motion blur.

[0056] like Figure 2 As shown, the blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit then generates a corresponding blur kernel correction kernel based on the feature vector. The blur kernel correction kernel is used to perform a convolution operation on the optical image to obtain a Gaussian blurred image.

[0057] In this embodiment, in the blur kernel correction unit: according to the feature vector, the Gaussian blur kernel in the Gaussian blur kernel library, i.e., the Gaussian blur kernel dictionary, is assigned a value, and the assigned Gaussian blur kernels are summed to obtain a blur kernel correction kernel. The blur kernels included in the Gaussian blur kernel dictionary are as follows: Figure 3 As shown, the length of the feature vector is equal to the number of Gaussian blur kernels in the Gaussian blur kernel dictionary.

[0058] Specifically, the feature vector is assigned one-to-one to each blur kernel in the Gaussian blur kernel dictionary in sequence, and the mapping from the motion blurred image to the Gaussian blurred image is fitted by the linear combination of the blur kernels stored in the Gaussian blur kernel dictionary, that is, the blur correction kernel is generated by summing all the blur kernels after the feature vector assignment.

[0059] In this embodiment, the feature extraction unit is constructed by a convolutional neural network and an attention mechanism.

[0060] Furthermore, the feature extraction unit is implemented using a neural network, and its network structure is as follows: Figure 4 As shown in the figure. The motion blurred image is feature extracted through multi-layer convolution pooling, and finally the blur information feature vector is output through the fully connected layer. Since the blur in the motion blurred image is often non-uniform, and the attention mechanism can help the neural network model perceive the change of blur in the image and focus the model on the area with severe blur, the attention mechanism module is added in each convolution pooling layer to improve the performance of the neural network in extracting blur information.

[0061] In this embodiment, when the blur kernel correction network is trained: first, a sample set is obtained, which includes multiple motion blurred sample images and corresponding clear sample images, each clear sample image is convolved with a Gaussian blur kernel to obtain a corresponding Gaussian blurred label image, the motion blurred sample image and the corresponding Gaussian blurred label image are used as a group of training sample pairs, a training sample set is constructed based on multiple groups of training sample pairs, a group of training sample pairs in the training sample set is input into the blur kernel correction network, and a Gaussian blurred prediction image corresponding to the motion blurred sample image is obtained, a loss function is calculated based on the Gaussian blurred prediction image and the Gaussian blurred label image, and the parameters in the blur kernel correction network are updated according to the calculation results until convergence, so as to obtain a trained blur kernel correction network.

[0062] In this embodiment, a Gaussian blurred image synthesized from a clear sample image is used as a true value label to guide the training process of the blur kernel correction network, so that the trained blur kernel correction network is capable of converting a motion blurred image into a Gaussian blurred image.

[0063] Specifically, the loss function used to train the blur kernel correction network is obtained by calculating the minimum mean square error and peak signal-to-noise ratio between the Gaussian corrected image and the synthetic Gaussian image.

[0064] In step S120, the Gaussian blurred image can be deblurred by using traditional image processing methods or deep learning-based methods, including but not limited to Wiener filtering method, inverse filtering method, blind deconvolution method, regularized deconvolution method, convolutional neural network, generative adversarial neural network, variational autoencoder, composite model, etc.

[0065] Preferably, a de-Gaussian blur network is used to deblur the Gaussian blurred image, wherein the de-Gaussian blur network may be, but is not limited to, any one of DeepDeblur, MANet or NAFNet.

[0066] Furthermore, when a deep learning-based method is used to deblur a Gaussian blurred image, an overall network of a motion deblurring network can be constructed based on the blur kernel correction network and the de-Gaussian blurring network to achieve end-to-end motion deblurring. The optical image with motion blur is input into the motion deblurring network to obtain a deblurred optical image.

[0067] In this embodiment, when training the de-motion blur network: first pre-train the blur kernel correction network to obtain the pre-trained blur kernel correction network, and then train the pre-trained blur kernel correction network and the de-Gaussian blur network to obtain the trained de-motion blur network.

[0068] Specifically, the process of pre-training the blur kernel correction network can refer to the above-mentioned process of training the blur kernel correction network, which will not be described in detail here.

[0069] In this embodiment, when the entire de-motion blurring network is trained, that is, when the pre-trained blur kernel correction network and the de-Gaussian blurring network are trained simultaneously, a training sample set is first constructed, and the training sample set includes multiple groups of training samples, each of which includes a motion blurred sample image, a clear sample image, and a Gaussian blurred label image generated by convolution of the clear sample image and the Gaussian blur kernel. The motion blurred sample image is input into the de-motion blurring network, and the pre-trained blur kernel correction network first converts the motion blurred sample image into a Gaussian blurred prediction image, and then the de-Gaussian blurring network de-Gaussian blurs the Gaussian blurred prediction image to obtain a clear prediction image.

[0070] Furthermore, the total loss function is calculated based on the Gaussian blurred prediction image, the clear prediction image, the corresponding clear sample image and the Gaussian blurred label image. After the gradient calculation is performed based on the total loss function, the parameters in the de-motion blur network are reversely corrected according to the gradient calculation result to achieve the update of the de-motion blur network.

[0071] In this embodiment, the total loss function includes a Gaussian corrected reconstruction loss function and a clear reconstruction loss function, wherein the Gaussian corrected reconstruction loss function is calculated based on a Gaussian blurred prediction image and a Gaussian blurred label image, and the clear reconstruction loss function is calculated based on a Gaussian blurred prediction image and a clear prediction image.

[0072] Specifically, the total loss function is expressed as:

[0073] ;

[0074] In the above formula, and are weight parameters respectively. is the Gaussian modified reconstruction loss function, Reconstruct the loss function for clarity.

[0075] like Figure 5 The figure shows a schematic diagram of the training process of the de-motion blur network.

[0076] In this paper, the effectiveness of this method is also demonstrated through experimental results. Figure 6 , which is a schematic diagram of a Gaussian blurred image obtained by inputting a motion blurred image into a blur kernel correction unit, that is, an image after de-motion blurring.

[0077] The above optical image deblurring method based on blur kernel correction first uses a blur kernel correction network to perform blur kernel correction on the optical image, converts the optical image with motion blur into a Gaussian blur image, and then deblurs the Gaussian blur image to obtain a deblurred optical image. By combining this method with the existing deblurring method, the positionally blurred non-deblurred image is first preprocessed by a blur correction unit to obtain a Gaussian corrected image with known blur, and then the blur corrected image is input into the subsequent deblurring network, which can avoid the disadvantage that the deblurring network cannot process unknown blurred images, improve the deblurring performance of the deblurring network and expand the generalization performance. This method has good generalization and has high deblurring performance in practical applications.

[0078] In the process of performing Gaussian correction on a motion blurred image to obtain a corresponding Gaussian blurred image, and then deblurring the Gaussian blurred image to obtain a clear image, blur correction and deblurring are not paired. The Gaussian blurred image can be input into any deblurring network to achieve image deblurring. Compared with motion blurred images, Gaussian blurred images have simpler blur distributions. Compared with directly inputting motion blurred images into the deblurring network, inputting Gaussian blurred images into the deblurring network can achieve better deblurring effects. The above operations can expand the generalization performance of any deblurring network in a plug-and-play manner.

[0079] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0080] In one embodiment, Figure 7 As shown, an optical image motion blur removal device based on blur kernel correction is provided, comprising: a blur image acquisition module 200, a Gaussian blur conversion module 210 and a deblur enhancement module 220, wherein:

[0081] A blurred image acquisition module 200, used to acquire an optical image with motion blur;

[0082] A Gaussian blur conversion module 210 is used to perform blur kernel correction on the optical image using a blur kernel correction network, and convert the optical image with motion blur into a Gaussian blur image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit performs feature extraction on the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector, and performs a convolution operation on the optical image using the blur kernel correction kernel to obtain the Gaussian blur image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0083] The deblurring enhancement module 220 is used to deblur the Gaussian blurred image to obtain a deblurred optical image.

[0084] For the specific definition of the optical image de-motion blur device based on blur kernel correction, please refer to the definition of the optical image de-motion blur method based on blur kernel correction mentioned above, which will not be repeated here. Each module in the above-mentioned optical image de-motion blur device based on blur kernel correction can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0085] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for de-motion blurring an optical image based on blur kernel correction is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0086] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0088] Acquiring an optical image with motion blur;

[0089] The optical image is subjected to blur kernel correction by using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector. The optical image is subjected to a convolution operation by using the blur kernel correction kernel to obtain the Gaussian blurred image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0090] The Gaussian blurred image is deblurred to obtain a deblurred optical image.

[0091] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0092] Acquiring an optical image with motion blur;

[0093] The optical image is subjected to blur kernel correction by using a blur kernel correction network, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit extracts features from the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit generates a corresponding blur kernel correction kernel according to the feature vector. The optical image is subjected to a convolution operation by using the blur kernel correction kernel to obtain the Gaussian blurred image. In the blur kernel correction unit, a Gaussian blur kernel in a Gaussian blur kernel library is assigned a value according to the feature vector, and the assigned Gaussian blur kernels are summed to obtain the blur kernel correction kernel.

[0094] The Gaussian blurred image is deblurred to obtain a deblurred optical image.

[0095] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0096] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An optical image deblurring method based on blur kernel correction, characterized in that: The method comprises: Acquiring an optical image with motion blur; A blur kernel correction network is used to perform blur kernel correction on the optical image, and the optical image with motion blur is converted into a Gaussian blurred image. The blur kernel correction network includes a feature extraction unit and a blur kernel correction unit. The feature extraction unit performs feature extraction on the optical image to obtain a feature vector related to the motion blur kernel. The blur kernel correction unit assigns a Gaussian blur kernel in a Gaussian blur kernel dictionary according to the feature vector, and sums the assigned Gaussian blur kernels to obtain the blur kernel correction kernel. The blur kernel correction kernel is used to perform a convolution operation on the optical image to obtain the Gaussian blurred image. The length of the feature vector is equal to the number of Gaussian blur kernels in the Gaussian blur kernel dictionary. When assigning a Gaussian blur kernel in the Gaussian blur kernel dictionary, the feature vector is assigned one-to-one to each blur kernel in the Gaussian blur kernel dictionary in sequence. The Gaussian blurred image is deblurred to obtain a deblurred optical image.

2. The optical image de-motion blurring method according to claim 1, characterized in that: The feature extraction unit is constructed by a convolutional neural network and an attention mechanism.

3. The optical image de-motion blurring method according to claim 1 or 2, characterized in that: When training the blur kernel correction network: Acquire a sample set, wherein the sample set includes a plurality of motion blurred sample images and corresponding clear sample images, and perform a convolution operation on each of the clear sample images with a Gaussian blur kernel to obtain a corresponding Gaussian blurred label image; The motion blurred sample image and the corresponding Gaussian blurred label image are used as a set of training sample pairs, and a training sample set is constructed according to multiple sets of training sample pairs; Inputting a set of training sample pairs in the training sample set into the blur kernel correction network to obtain a Gaussian blurred prediction image corresponding to the motion blurred sample image; The loss function is calculated according to the Gaussian blurred prediction image and the Gaussian blurred label image, and the parameters in the blur kernel correction network are updated according to the calculation results until convergence, so as to obtain a trained blur kernel correction network.

4. The optical image de-motion blurring method according to claim 3, characterized in that: A de-Gaussian blur network is used to deblur the Gaussian blurred image.

5. The optical image de-motion blurring method according to claim 4, characterized in that: Constructing a motion blur removal network according to the blur kernel correction network and the Gaussian blur removal network; The optical image with motion blur is input into the de-motion blur network to obtain the de-blurred optical image.

6. The optical image de-motion blurring method according to claim 5, characterized in that: When training the de-motion blur network: Firstly, the blur kernel correction network is pre-trained to obtain a pre-trained blur kernel correction network; The pre-trained blur kernel correction network and the Gaussian blur removal network are then trained to obtain a trained motion blur removal network.

7. An optical image motion blur removal device based on blur kernel correction, characterized in that: The device comprises: A blurred image acquisition module, used for acquiring an optical image with motion blur; A Gaussian blur conversion module, used for performing blur kernel correction on the optical image by using a blur kernel correction network, and converting the optical image with motion blur into a Gaussian blur image, wherein the blur kernel correction network includes a feature extraction unit and a blur kernel correction unit, wherein the feature extraction unit performs feature extraction on the optical image to obtain a feature vector related to the motion blur kernel, wherein the blur kernel correction unit assigns a Gaussian blur kernel in a Gaussian blur kernel dictionary according to the feature vector, and sums the assigned Gaussian blur kernels to obtain the blur kernel correction kernel, and performs a convolution operation on the optical image by using the blur kernel correction kernel to obtain the Gaussian blur image, wherein the length of the feature vector is equal to the number of Gaussian blur kernels in the Gaussian blur kernel dictionary, and when assigning a Gaussian blur kernel in the Gaussian blur kernel dictionary, the feature vector is assigned one-to-one to each blur kernel in the Gaussian blur kernel dictionary in sequence; The deblurring enhancement module is used to deblur the Gaussian blurred image to obtain a deblurred optical image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.