Image deblurring method for improving blurred image recognition rate

Abstract

The invention discloses an image deblurring method for improving the blurred image recognition rate, and belongs to the technical field of image processing. The method is low in calculation complexity and high in generalization ability, image deblurring and target detection are cascaded, the requirement of a machine for image perception can be considered, and therefore the blurred image target detection effect is enhanced. Wherein the loss function of the whole network is composed of the frequency reconstruction loss and the content perception loss of the deblurred image generated by the deblurring network and the perception Yolo loss of the target detection network. The target detection accuracy is improved while the image deblurring effect is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an image deblurring method for improving the recognition rate of blurred images. Background Art

[0002] Image deblurring is an important image preprocessing technology, and its main purpose is to restore a clear image from a blurred image for subsequent computer vision tasks. Most traditional image deblurring methods utilize prior knowledge, such as gradient prior, patch prior, etc., to obtain the blur kernel that blurs the image. Then, operations such as deconvolution are performed on the blurred image using the calculated blur kernel to obtain a clear image. In recent years, due to the rapid development of deep learning technology, neural network technology has been widely applied to image processing. Many effective end-to-end deep learning deblurring methods have also been proposed in image deblurring. Due to reasons such as object movement, camera shake, and defocus, the blur in blurred images is mostly non-uniform. Different regions in the image contain blurs with different directions and sizes, which brings great difficulties and challenges to the image deblurring work. Although many image deblurring algorithms have been proposed currently, some algorithms have problems such as loss of image details and poor restoration of edge textures while removing the blur.

[0003] Object detection algorithms are an important computer vision task, and significant achievements have been made with the development of deep learning. However, when faced with blurred images, simply using object detection algorithms cannot achieve effective detection results. Using object detection algorithms to detect objects in blurred images not only reduces the detection accuracy but also has problems such as missed detection and false detection. The blurred image textures and smooth edge information in blurred images are not conducive to the implementation of object detection. During the image acquisition process, due to the movement of the target object, camera shake, etc., image blur will inevitably occur. The research on object detection in blurred images not only has important theoretical significance but also shows broad prospects in practical applications. In security monitoring, it can quickly identify suspicious targets from blurred videos; in the field of autonomous driving, it can analyze blurred road images in real time to ensure driving safety; in medical imaging, it can extract key information from blurred medical images to assist in diagnosis. Therefore, it is necessary to propose an object detection method for blurred images to eliminate the influence of blur on the object detection performance, thereby ensuring the accuracy of object detection in blurred images.

[0004] Regarding the processing of blurred images, researchers have proposed many solutions to improve the deblurring effect of images. Generally speaking, deep learning-based image deblurring frameworks can be divided into methods based on CNN, GAN, RNN, Transformer, Diffusion, etc. However, in the process of processing blurred images, the importance of frequency information, especially high-frequency information, is often overlooked, resulting in the loss of edge details in the restored images. In addition, since the blur in the image is non-uniform, this poses a great challenge to image deblurring. Non-uniform blur limits the deblurring performance of many methods.

[0005] Regarding the object detection of blurred images, some scholars have also proposed many solutions. These methods aim to improve the clarity of the images, and by enhancing the clarity of the images, the accuracy of image object detection is further improved. In these methods, the image deblurring technology is generally used as a data preprocessing technology, and object detection is used as a downstream task. These methods are committed to studying how to improve the PSNR of the deblurred images, rather than studying the restoration of the feature maps required in object detection. Image deblurring technology can be beneficial to people's visual perception and the image perception of video surveillance devices. However, for the object detection task of machine learning, it cannot be guaranteed that the better the human eye perception, the better the object detection effect. At present, some studies have also proposed algorithms for object detection of blurred images, but few algorithms combine image deblurring with object detection algorithms. Summary of the Invention

[0006] In view of the above technical problems, the present invention proposes an image deblurring solution to improve the recognition rate of blurred images.

[0007] The first aspect of the present invention discloses an image deblurring method for improving the recognition rate of blurred images, and the method includes:

[0008] Step S1, obtaining an image pair, where the image pair includes a blurred image and a clear image corresponding to the blurred image, and the image pair is used to train a network, and the network includes a deblurring network and an object detection network;

[0009] Step S2, inputting the blurred image into the deblurring network, and the deblurring network performs deblurring processing on the blurred image to obtain a deblurred image, and calculates a frequency reconstruction loss and a content-aware loss according to the deblurred image and the clear image;

[0010] Step S3, inputting the deblurred image and the clear image into the object detection network, and the object detection network performs object detection on the deblurred image to output an object detection image, and calculates a perceptual Yolo loss between the object detection image and the clear image;

[0011] Step S4: Use the frequency reconstruction loss, content perception loss, and Yolo perception loss as the overall loss of the network, and feedback it to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network, and using the trained network for image deblurring processing.

[0012] According to the method of the first aspect of the present invention, the deblurring network includes an encoder, a blur information perception module, a frequency information enhancement module, and a decoder; wherein:

[0013] The encoder consists of convolutional layers. The blurred image is input into the encoder for feature extraction to obtain the feature map of the blurred image.

[0014] The blur information perception module uses strip convolution to collect and perceive blur information, and uses a residual structure to obtain the overall blur information; wherein, the feature map of the blurred image extracted by the encoder is input into the blur information perception module, and strip convolution and activation functions are used to perceive the blur in different regions and different directions of the image.

[0015] The frequency information enhancement module uses wavelet transform to transform the feature map of the blurred image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information. The high-frequency information is convolved and enhanced through convolution functions and activation functions, and the texture details are restored by combining spatial domain and frequency domain information.

[0016] Fuse the two sets of enhanced features output by the blur information perception module and the frequency information enhancement module, and obtain the deblurred image through the decoder.

[0017] According to the method of the first aspect of the present invention, the object detection network is the Yolo-v8 network. The deblurred image is input into the object detection network, and the object detection image is obtained through object detection, and the perceived Yolo loss is calculated.

[0018] According to the method of the first aspect of the present invention, an image pair is obtained. The size of the preprocessed image is 3×H×W, where H×W is the size of the input image, 3 is the number of channels of the image, and after passing through the encoder, the number of feature channels of the image is expanded to C.

[0019] According to the method of the first aspect of the present invention, the image feature F with a size of C×H×W after passing through the encoder is input into the blur information perception module for feature enhancement. The blur information perception module is composed of convolutional layers and strip convolution. The calculation process is as follows: First, perform global average pooling on the image feature F, and then use strip convolution to obtain blur information to obtain the corresponding feature F 1 and F 2 :

[0020] F 1 =Conv 3×1(Avg(F))Conv 1×3 (Avg(F))

[0021] F 2 =Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F))

[0022] where Conv represents convolution and Avg represents average pooling;

[0023] At the same time, feature extraction is performed on the image feature F to obtain the global information feature F 3 :

[0024] F 3 =Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0025] where ReLU represents the activation function and BN represents normalization;

[0026] The feature F 1 , F 2 and F 3 are fused, and weighted mapping is performed through the Sigmoid function to obtain the enhanced feature F_w:

[0027] F_w = F * Sigmoid(concate((F 1 , F 2 , F 3 ), dim = 0))

[0028] where concate represents concatenation, dim represents dimension, and the features F 1 , F 2 and F 3 are concatenated and fused on the first dimension of the tensor.

[0029] According to the method of the first aspect of the present invention, the image feature F of size C×H×W after the encoder is input into the frequency information enhancement module for feature enhancement. The frequency information enhancement module is composed of wavelet transform, convolutional layer, activation function, normalization layer, and inverse wavelet transform. The calculation process is as follows: First, wavelet transform is performed on the image feature F to obtain four sub-generation features of high-high frequency, high-low frequency, low-high frequency, and low-low frequency, which are respectively as follows:

[0030] F HH , F LH , F HL , F LL =DWT(F)

[0031] where DWT represents wavelet transform;

[0032] After performing convolutional learning on high-frequency information, the high-frequency information is fused with low-frequency information, and an enhanced frequency feature map is obtained through inverse wavelet transform. The feature map is mapped into a weight feature map M through the Sigmoid function:

[0033] M = Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL )))

[0034] where IDWT represents inverse wavelet transform;

[0035] The image feature F includes the original image content and frequency feature information. The image feature is further extracted through convolutional operations to obtain the feature map F 4 , which is multiplied by the weight feature map M to obtain the enhanced feature F_f of the high-frequency information:

[0036] F 4 = Conv(Norm(ReLU(Conv(F))))

[0037]

[0038] where Norm represents normalization.

[0039] According to the method of the first aspect of the present invention, the enhanced features F_w and F_f together with the blurred image features are input into the decoder to obtain a de-blurred image.

[0040] The second aspect of the present invention discloses an image de-blurring system for improving the recognition rate of blurred images. The system includes a processing unit, and the processing unit is configured to execute:

[0041] Obtain an image pair, where the image pair includes a blurred image and a clear image corresponding to the blurred image. The image pair is used to train a network, and the network includes a de-blurring network and an object detection network;

[0042] Input the blurred image into the de-blurring network, and the de-blurring network performs de-blurring processing on the blurred image to obtain a de-blurred image. Calculate the frequency reconstruction loss and content-aware loss based on the de-blurred image and the clear image;

[0043] Input the de-blurred image and the clear image into the object detection network, and the object detection network performs object detection on the de-blurred image to output an object detection image, and calculate the perceptual Yolo loss between the object detection image and the clear image;

[0044] The frequency reconstruction loss, content-aware loss, and Yolo-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network and using the trained network for image deblurring.

[0045] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, an image deblurring method for improving the recognition rate of blurred images described in the first aspect of the present disclosure is implemented.

[0046] A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, an image deblurring method for improving the recognition rate of blurred images described in the first aspect of the present disclosure is implemented.

[0047] In summary, the present invention has low computational complexity and strong generalization ability. By cascading image deblurring and target detection, it can take into account the requirements of the machine for image perception, thereby enhancing the effect of target detection for blurred images. The loss function of the entire network is jointly composed of the frequency reconstruction loss and content-aware loss generated by the deblurring network for the deblurred image and the Yolo-aware loss of the target detection network. While enhancing the image deblurring effect, the target detection accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 A flowchart of an image deblurring method for improving the recognition rate of blurred images;

[0050] Figure 2 An architecture diagram of a deblurring target detection network;

[0051] Figure 3 A structural diagram of a blur information perception module;

[0052] Figure 4 A structural diagram of a frequency information enhancement module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0054] Image blurring is the most common image information loss phenomenon in daily life. As an important information carrier, blurring not only affects people's visual perception but also has an adverse impact on downstream computer vision tasks. This patent proposes an image deblurring solution to improve the recognition rate of blurred images.

[0055] The first aspect of the present invention discloses an image deblurring method for improving the recognition rate of blurred images, and the method includes:

[0056] Step S1: Obtain an image pair, where the image pair includes a blurred image and a clear image corresponding to the blurred image. The image pair is used to train a network, and the network includes a deblurring network and an object detection network;

[0057] Step S2: Input the blurred image into the deblurring network, and the deblurring network performs deblurring processing on the blurred image to obtain a deblurred image. Calculate the frequency reconstruction loss and the content perception loss based on the deblurred image and the clear image;

[0058] Step S3: Input the deblurred image and the clear image into the object detection network, and the object detection network performs object detection on the deblurred image to output an object detection image, and calculate the perceptual Yolo loss between the object detection image and the clear image;

[0059] Step S4: Use the frequency reconstruction loss, the content perception loss, and the Yolo perceptual loss as the overall loss of the network, and feedback it to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network, and using the trained network for image deblurring processing.

[0060] According to the method of the first aspect of the present invention, the deblurring network includes an encoder, a blur information perception module, a frequency information enhancement module, and a decoder; where:

[0061] The encoder is composed of convolutional layers. Input the blurred image into the encoder for feature extraction to obtain the feature map of the blurred image;

[0062] The fuzzy information perception module uses strip convolution to collect and perceive fuzzy information, and uses a residual structure to obtain the overall fuzzy information. Among them, the feature map of the fuzzy image extracted by the encoder is input into the fuzzy information perception module, and strip convolution and activation functions are used to perceive the fuzziness in different regions and in different directions of the image.

[0063] The frequency information enhancement module uses wavelet transform to transform the feature map of the fuzzy image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information. The high-frequency information is enhanced by convolution through a convolution function and an activation function, and the restoration of texture details is improved by combining spatial domain and frequency domain information.

[0064] The two sets of enhanced features output by the fuzzy information perception module and the frequency information enhancement module are fused, and a de-blurred image is obtained through a decoder.

[0065] According to the method of the first aspect of the present invention, the target detection network is the Yolo-v8 network. The de-blurred image is input into the target detection network, and a target detection image is obtained through target detection, and the perception Yolo loss is calculated.

[0066] According to the method of the first aspect of the present invention, an image pair is obtained. The size of the preprocessed image is 3×H×W, where H×W is the size of the input image and 3 is the number of channels of the image. After passing through the encoder, the number of feature channels of the image is expanded to C.

[0067] According to the method of the first aspect of the present invention, the image feature F with a size of C×H×W after passing through the encoder is input into the fuzzy information perception module for feature enhancement. The fuzzy information perception module is composed of a convolutional layer and a strip convolution. The calculation process is as follows: First, global average pooling is performed on the image feature F, and then strip convolution is used to obtain fuzzy information, and the corresponding feature F 1 and F 2 :

[0068] F 1 =Conv 3×1 (Avg(F))+Conv 1×3 (Avg(F))

[0069] F 2 =Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F))

[0070] Among them, Conv represents convolution and Avg represents average pooling;

[0071] At the same time, global information feature F 3 is extracted from the image feature F:

[0072] F 3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0073] Among them, ReLU represents the activation function, and BN represents normalization;

[0074] Fuse the feature F 1 , F 2 and F 3 through weighted mapping by the Sigmoid function to obtain the enhanced feature F_w:

[0075] F_w = F * Sigmoid(concate((F 1 , F 2 , F 3 ), dim = 0))

[0076] Among them, concate represents concatenation, dim represents dimension, and concatenate and fuse the features F 1 , F 2 and F 3 on the first dimension of the tensor.

[0077] According to the method of the first aspect of the present invention, the image feature F with a size of C×H×W after the encoder is input into the frequency information enhancement module for feature enhancement. The frequency information enhancement module is composed of wavelet transform, convolutional layer, activation function, normalization layer, and inverse wavelet transform. The calculation process is as follows: First, perform wavelet transform on the image feature F to obtain four sub-generation features of high-high frequency, high-low frequency, low-high frequency, and low-low frequency, as follows:

[0078] F HH , F LH , F HL , F LL = DWT(F)

[0079] Among them, DWT represents wavelet transform;

[0080] After performing convolutional learning on the high-frequency information, fuse the high-frequency information with the low-frequency information, obtain the frequency-enhanced feature map through inverse wavelet transform, and map the feature map into the weight feature map M through the Sigmoid function:

[0081] M = Sigmoid(IDWT(concate(Conv(F HH , F LH , F HL ), F LL )))

[0082] Among them, IDWT represents inverse wavelet transform;

[0083] The image feature F contains the original image content and frequency feature information. The image feature is further extracted through a convolution operation to obtain the feature map F 4 , which is multiplied by the weight feature map M to obtain the enhanced feature F_f of the high-frequency information:

[0084] F 4 = Conv(Norm(ReLU(Conv(F))))

[0085]

[0086] where Norm represents normalization.

[0087] According to the method of the first aspect of the present invention, the enhanced features F_w and F_f together with the blurred image feature are input into the decoder to obtain a de-blurred image.

[0088] The second aspect of the present invention discloses an image de-blurring system for improving the recognition rate of blurred images. The system includes a processing unit configured to execute:

[0089] Obtain an image pair, where the image pair includes a blurred image and a clear image corresponding to the blurred image. The image pair is used to train a network, and the network includes a de-blurring network and an object detection network;

[0090] Input the blurred image into the de-blurring network, and the de-blurring network performs de-blurring processing on the blurred image to obtain a de-blurred image. Calculate the frequency reconstruction loss and the content-aware loss based on the de-blurred image and the clear image;

[0091] Input the de-blurred image and the clear image into the object detection network, and the object detection network performs object detection on the de-blurred image to output an object detection image, and calculate the perceptual Yolo loss between the object detection image and the clear image;

[0092] Use the frequency reconstruction loss, the content-aware loss, and the Yolo perceptual loss as the overall loss of the network, and feedback it to the de-blurring network to optimize the parameters of the de-blurring network, thereby completing the training of the network, and using the trained network for image de-blurring processing.

[0093] First Embodiment

[0094] This embodiment proposes an image de-blurring method for improving the recognition rate of blurred images, as Figure 1 shown, mainly consisting of two parts: an image de-blurring network and an object detection network. The main training data of the network is the clear-blurred data pair, which mainly comes from the existing data sets and is composed of screening images containing targets such as pedestrians and vehicles.

[0095] The input of the network is a blurred image, and the deblurred image is obtained through a deblurring network trained with blurred and clear images. The frequency reconstruction loss and content-aware loss are calculated based on the deblurred image and the clear image. At the same time, the deblurred image and the clear image are input into the object detection network. In this method, the Yolo-v8 object detection network is used to calculate the Yolo-aware loss. The frequency reconstruction loss, content-aware loss, and Yolo-aware loss are added together as the overall loss of the network and fed back to the deblurring network for parameter training and optimization of the network.

[0096] Second Embodiment

[0097] This embodiment proposes an image deblurring method to improve the recognition rate of blurred images, which can improve the image deblurring effect and at the same time improve the object detection accuracy of blurred images. It mainly improves the design of the fuzzy information perception and frequency information enhancement algorithms on the basis of traditional algorithms. As Figure 2 shown, this network can adaptively learn the mapping relationship from image blur to clarity, and while better removing blur, improve the accuracy of object detection.

[0098] Compared with general image deblurring algorithms, the method proposed in the present invention mainly acts on the fuzzy information perception and frequency information enhancement modules. In the fuzzy information perception module, strip convolution is used for the acquisition and perception of fuzzy information, and a residual structure is used to obtain the overall fuzzy information. In the frequency information enhancement module, wavelet transform is used to transform the image features from the spatial domain to the frequency domain, and the high-frequency information therein is convolved and enhanced to improve the restoration of texture details. Finally, the two sets of enhanced features are fused, and the deblurred image is obtained through the encoder.

[0099] Third Embodiment

[0100] This embodiment is used to illustrate the module composition of the image deblurring method for improving the recognition rate of blurred images

[0101] 1. Input Module

[0102] An existing blurred image dataset is used as training data, and the blurred-clear image pairs therein are used as the input of the network.

[0103] 2. Encoder Module

[0104] The encoder is mainly composed of convolutional layers. The blurred image is input into the encoder for feature extraction to obtain the feature map of the blurred image.

[0105] 3. Fuzzy Information Perception Module

[0106] The features of the blurred image extracted by the encoder are input into the fuzzy perception module, as Figure 3As shown in the figure. The blur perception module perceives the blur in different regions and different directions in the image through strip convolution and activation functions, etc.

[0107] 4. Frequency Enhancement Module

[0108] Input the features of the blurred image extracted by the encoder into the frequency enhancement module, as Figure 4 shown in the figure. The frequency enhancement module divides the image features into high-frequency and low-frequency information through wavelet transform, and enhances the information through convolution functions and activation functions. By combining spatial and frequency domain information, the image information is better restored.

[0109] 5. Decoder Module

[0110] Fuse the information features of the above two modules and input them into the decoder module. According to the training parameters of the blurred image pair, the de-blurred image is output.

[0111] 6. Object Detection Module

[0112] Input the de-blurred image into the object detection network, predict the object detection result, and calculate and feedback the corresponding perception loss into the de-blurring network for parameter update and optimization. In the present invention, the Yolo-v8 object detection algorithm is used to perform object detection on the de-blurred image.

[0113] Fourth Embodiment

[0114] This embodiment is the process design of an image de-blurring method for improving the recognition rate of blurred images. The purpose is to improve the de-blurring effect of blurred images and at the same time improve the object detection accuracy of blurred images. Therefore, the input is a blurred image, and the output is a de-blurred image and an object detection result map. The algorithm process is as follows:

[0115] Step S1: Obtain image data. After preprocessing, the input size is 3×H×W, where H×W is the size of the input image and 3 is the number of channels of the image. After passing through the encoder, the number of feature channels of the image is expanded to C for subsequent feature processing and enhancement.

[0116] Step S2: Input the image feature F of C×H×W into the blur perception module for feature enhancement. This network is composed of a convolutional layer and a strip convolution, as Figure 3 shown in the figure. The overall calculation process is as follows: First, perform global average pooling on the input features, and then use strip convolution to obtain blur information to obtain the corresponding feature maps F 1 and F 2 :

[0117] F 1 =Conv 3×1 (Avg(F))+Conv1×3 (Avg(F))

[0118] F 2 = Conv 3×3 (Avg(F)) + Conv 3×3 (Avg(F))

[0119] Extract the global information feature F from the input features 3 :

[0120] F 3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0121] Then fuse the fuzzy feature F 1 , F 2 and the global information feature, and perform weighted mapping through the Sigmoid function to obtain the fuzzy feature enhanced feature map F_w:

[0122] F_w = F * Sigmoid(concate((F 1 , F 2 , F 3 ), dim = 0))

[0123] Step S3: Input the C×H×W image feature F into the frequency enhancement module for feature enhancement. This network consists of wavelet transform, convolutional layer, activation function, normalization layer, and inverse wavelet transform. As Figure 4 shown. The overall calculation process is: First, perform wavelet transform on the fuzzy feature map to obtain four sub-generation features: high-high frequency, high-low frequency, low-high frequency, and low-low frequency:

[0124] F HH , F LH , F HL , F LL = DWT(F)

[0125] To enhance the high-frequency information, only perform convolutional learning on the sub-bands containing high-frequency information, enabling the network to adaptively learn the parameters of high-frequency information. Then, fuse the high-frequency and low-frequency information, and obtain the frequency-enhanced feature map through inverse wavelet transform. Map the feature map into a weight feature map M through the Sigmoid function:

[0126] M = Sigmoid(IDWT(concate(Conv(F HH , F LH , F HL ), F LL )))

[0127] The blurred image feature map F contains the original image content and frequency feature information. Through operations such as convolution, the image features are further extracted to obtain the feature map F 1 , which is multiplied by the weight feature map M to obtain the feature map F_f with enhanced high-frequency information for image deblurring. The calculation process is as follows:

[0128] F 1 = Conv(Norm(ReLU(Conv(F))))

[0129]

[0130] Step S4: Input the above enhanced image features F_w and F_f together with the blurred image features into the decoder to obtain the deblurred image.

[0131] Step S5: Input the deblurred image into the target detection network to obtain the target detection result of the blurred image.

[0132] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements an image deblurring method for improving the recognition rate of blurred images according to the first aspect of the present disclosure.

[0133] A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements an image deblurring method for improving the recognition rate of blurred images according to the first aspect of the present disclosure.

[0134] In summary, the present invention proposes an image deblurring scheme for improving the recognition rate of blurred images. The network proposed in this scheme cascades an improved image deblurring network and a target detection network. It can obtain the target detection result of the blurred image while deblurring the blurred image. According to the actual non-uniform blur situation, the present invention proposes a blur-aware network. The blur-aware network adaptively learns the blur in different directions and sizes in the image, and assigns weights to the blur features, which is more helpful for image deblurring. The present invention proposes a frequency enhancement network, which consists of a wavelet transform and a convolutional layer, enhances the high-frequency information in the image features, helps to restore the image texture details, and improves the effect of target detection.

[0135] The present invention has low computational complexity and strong generalization ability. By cascading image deblurring and object detection, it can take into account the requirements of machine perception of images, thereby enhancing the effect of object detection in blurred images. The loss function of the entire network is jointly composed of the frequency reconstruction loss and content perception loss of the deblurring network for generating deblurred images, and the perception Yolo loss of the object detection network. While enhancing the image deblurring effect, it improves the object detection accuracy.

[0136] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification. The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image deblurring method for improving the recognition rate of blurred images, characterized in that: The method comprises: Step S1, obtaining an image pair, wherein the image pair includes a blurred image and a clear image corresponding to the blurred image, and the image pair is used to train a network, wherein the network includes a deblurring network and a target detection network; Step S2, inputting the blurred image into a deblurring network, and the deblurring network deblurs the blurred image to obtain a deblurred image, and calculating a frequency reconstruction loss and a content perception loss based on the deblurred image and the clear image; Step S3, inputting the deblurred image and the clear image into the target detection network, and the target detection network performs target detection on the deblurred image to output a target detection image, and calculates the perceptual Yolo loss between the target detection image and the clear image; Step S4: frequency reconstruction loss, content perception loss and Yolo perception loss are used as the overall loss of the network and fed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network and using the trained network to perform image deblurring processing.

2. The image deblurring method for improving the recognition rate of blurred images according to claim 1, characterized in that: The deblurring network includes an encoder, a fuzzy information perception module, a frequency information enhancement module and a decoder; wherein: The encoder consists of convolutional layers. The blurred image is input into the encoder to extract features and obtain the feature map of the blurred image. The fuzzy information perception module uses strip convolution to collect and perceive fuzzy information, and uses a residual structure to obtain the overall fuzzy information. The feature map of the fuzzy image extracted by the encoder is input into the fuzzy information perception module, and the fuzziness in different areas and directions of the image is perceived through strip convolution and activation functions. The frequency information enhancement module uses wavelet transform to transform the feature map of the blurred image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information. The high-frequency information is enhanced by convolution function and activation function, and the restoration of texture details is improved by combining spatial domain and frequency domain information. The two sets of enhanced features output by the blur information perception module and the frequency information enhancement module are fused, and the deblurred image is obtained through the decoder.

3. The image deblurring method for improving the recognition rate of blurred images according to claim 2, characterized in that: The target detection network is a Yolo-v8 network. The deblurred image is input into the target detection network, the target detection image is obtained through target detection, and the perceptual Yolo loss is calculated.

4. The image deblurring method for improving the recognition rate of blurred images according to claim 3, characterized in that: Get an image pair. The size of the image after preprocessing is 3×H×W, where H×W is the size of the input image and 3 is the number of channels of the image. After the encoder, the number of feature channels of the image is expanded to C.

5. The image deblurring method for improving the recognition rate of blurred images according to claim 4, characterized in that: The image feature F with a size of C×H×W after the encoder is input into the fuzzy information perception module for feature enhancement. The fuzzy information perception module consists of a convolution layer and a strip convolution. The calculation process is as follows: first, the image feature F is globally pooled, and then the strip convolution is used to obtain the fuzzy information to obtain the corresponding features F1 and F2: F1=Conv 3×1 (Avg(F))+Conv 1×3 (Avg(F)) F2=Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F)) Among them, Conv means convolution, Avg means average pooling; At the same time, the image feature F is extracted to obtain the global information feature F3: F3=Conv 1×1 (BN(ReLU(Conv 1×1 (F)))) Among them, ReLU represents the activation function, and BN represents normalization; The features F1, F2 and F3 are fused and weighted mapped through the Sigmoid function to obtain the enhanced feature F_w: F_w=F*Sigmoid(concate((F1,F2,F3),dim=0)) Among them, concate means concatenation, dim means dimension, and the features F1, F2 and F3 are concatenated and fused on the first dimension of the tensor.

6. The image deblurring method for improving the recognition rate of blurred images according to claim 5, characterized in that: The image feature F with a size of C×H×W after the encoder is input into the frequency information enhancement module for feature enhancement. The frequency information enhancement module consists of wavelet transform, convolution layer, activation function, normalization layer and inverse wavelet transform. The calculation process is as follows: first, the image feature F is subjected to wavelet transform to obtain four sub-generation features of high-frequency, high-low frequency, low-high frequency and low-low frequency, which are as follows: F HH ,F LH ,F HL ,F LL =DWT(F) Among them, DWT means wavelet transform; After convolution learning of high-frequency information, the high-frequency information is fused with the low-frequency information, and the frequency-enhanced feature map is obtained through inverse wavelet transform. The feature map is mapped into a weighted feature map M through the Sigmoid function: M=Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL ))) Among them, IDWT means inverse wavelet transform; The image feature F contains the original image content and frequency feature information. The image features are further extracted through convolution operation to obtain the feature map F4, which is multiplied with the weight feature map M to obtain the enhanced feature F_f of the high-frequency information: F4=Conv(Norm(ReLU(Conv(F)))) Among them, Norm means normalization.

7. The image deblurring method for improving the recognition rate of blurred images according to claim 6, characterized in that: The enhanced features F_w and F_f together with the blurred image features are input into the decoder to obtain the deblurred image.

8. An image deblurring system for improving the recognition rate of blurred images, characterized in that: The system comprises a processing unit configured to perform: Acquire an image pair, the image pair comprising a blurred image and a clear image corresponding to the blurred image, the image pair being used to train a network, the network comprising a deblurring network and a target detection network; The blurred image is input into the deblurring network, and the deblurring network deblurs the blurred image to obtain a deblurred image. The frequency reconstruction loss and the content perception loss are calculated based on the deblurred image and the clear image. The deblurred image and the clear image are input into the target detection network, which detects the target on the deblurred image to output the target detection image and calculates the perceptual Yolo loss between the target detection image and the clear image. Frequency reconstruction loss, content-aware loss and Yolo-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network and using the trained network to perform image deblurring.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, an image deblurring method for improving the recognition rate of blurred images as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the image deblurring method for improving the recognition rate of blurred images as described in any one of claims 1 to 7 is implemented.

Images