Image deblurring method for improving recognition rate of blurred image

Abstract

The application discloses an image deblurring method for improving the recognition rate of a blurred image, and belongs to the technical field of image processing. The method has low calculation complexity and strong generalization ability, and can consider the requirements of machine image perception by cascading image deblurring and target detection, thereby enhancing the effect of target detection on a blurred image. The loss function of the whole network is jointly composed of the frequency reconstruction loss and the content perception loss of the deblurring network for generating a deblurred image and the perception Yolo loss of the target detection network. While enhancing the image deblurring effect, the target detection accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to an image deblurring method to improve the recognition rate of blurred images. Background Technology

[0002] Image deblurring is an important image preprocessing technique. Its main purpose is to recover a sharp image from a blurred image for subsequent computer vision tasks. Traditional image deblurring methods mostly utilize prior knowledge, such as gradient priors and patch priors, to obtain a blur kernel that blurs the image. The calculated blur kernel is then used to perform operations such as deconvolution on the blurred image to obtain a sharp image. In recent years, due to the rapid development of deep learning technology, neural network technology has been widely applied to image processing. Many effective end-to-end deep learning deblurring methods have also been proposed for image deblurring. Due to factors such as object motion, camera shake, and defocusing, the blur in blurred images is mostly non-uniform. Different regions of the image contain blur of different directions and sizes, which brings great difficulties and challenges to image deblurring. Although many image deblurring algorithms have been proposed, some algorithms suffer from loss of image details and poor edge texture recovery while removing blur.

[0003] Object detection algorithms are an important task in computer vision, and significant progress has been made with the development of deep learning. However, when faced with blurred images, simply using object detection algorithms is insufficient for effective detection. Using these algorithms on blurred images not only reduces accuracy but also leads to missed detections and false positives. The blurred textures and smooth edges in blurred images are detrimental to object detection. Furthermore, image blurring is unavoidable during image acquisition due to object movement and camera shake. Research on object detection in blurred images not only has significant theoretical importance but also shows broad prospects in practical applications. In security monitoring, it can quickly identify suspicious targets from blurred videos; in autonomous driving, it can analyze blurred road images in real time to ensure driving safety; and in medical imaging, it can extract key information from blurred medical images to aid diagnosis. Therefore, it is necessary to propose a method for object detection in blurred images that eliminates the impact of blur on detection performance, thereby ensuring the accuracy of object detection in blurred images.

[0004] To address the processing of blurred images, researchers have proposed numerous solutions to improve image deblurring results. Generally, deep learning-based image deblurring frameworks can be categorized into methods based on CNNs, GANs, RNNs, Transformers, Diffusion, etc. However, in processing blurred images, the importance of frequency information, especially high-frequency information, is often neglected, leading to the loss of edge details in the recovered image. Furthermore, the non-uniformity of blur in images poses a significant challenge to image deblurring. Non-uniform blur limits the deblurring performance of many methods.

[0005] For object detection in blurred images, many solutions have been proposed by researchers. These methods aim to improve image sharpness, thereby increasing the accuracy of object detection. Generally, image deblurring is used as a data preprocessing technique, with object detection as a downstream task. These methods focus on improving the PSNR of deblurred images, rather than on recovering the feature maps required for object detection. Image deblurring can benefit human visual perception and image perception in video surveillance equipment. However, for machine learning-based object detection tasks, better human perception does not guarantee better object detection results. Currently, some research has proposed algorithms for object detection in blurred images, but few algorithms combine image deblurring with object detection algorithms. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes an image deblurring scheme that improves the recognition rate of blurred images.

[0007] The first aspect of this invention discloses an image deblurring method for improving the recognition rate of blurred images, the method comprising:

[0008] Step S1: Obtain image pairs, the image pairs including a blurred image and a clear image corresponding to the blurred image, the image pairs are used to train the network, the network including a deblurring network and an object detection network;

[0009] Step S2: Input the blurred image into the deblurring network, which deblurs the image to obtain a deblurred image. Calculate the frequency reconstruction loss and content perception loss based on the deblurred image and the clear image.

[0010] Step S3: Input the deblurred image and the sharp image into the object detection network. The object detection network performs object detection on the deblurred image to output the object detection image and calculates the perceptual YOLO loss between the object detection image and the sharp image.

[0011] Step S4: The frequency reconstruction loss, content-aware loss, and YOLO-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network and using the trained network to perform image deblurring.

[0012] According to a first aspect of the method of the present invention, the deblurring network includes an encoder, a fuzzy information sensing module, a frequency information enhancement module, and a decoder; wherein:

[0013] The encoder consists of convolutional layers. A blurred image is input into the encoder, features are extracted, and a feature map of the blurred image is obtained.

[0014] The fuzzy information perception module uses strip convolution to collect and perceive fuzzy information, and uses a residual structure to obtain the overall fuzzy information. The feature map of the fuzzy image extracted by the encoder is input into the fuzzy information perception module, which uses strip convolution and activation function to perceive the fuzziness in different regions and in different directions of the image.

[0015] The frequency information enhancement module uses wavelet transform to transform the feature map of the blurred image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information. It then enhances the high-frequency information through convolution and activation functions, and combines spatial and frequency domain information to improve the recovery of texture details.

[0016] The two sets of enhanced features output by the fuzzy information perception module and the frequency information enhancement module are fused together, and the deblurred image is obtained through the decoder.

[0017] According to the method of the first aspect of the present invention, the target detection network is a YOLO-v8 network. The deblurred image is input into the target detection network, the target detection image is obtained through target detection, and the perceptual YOLO loss is calculated.

[0018] According to the method of the first aspect of the present invention, an image pair is acquired, and the preprocessed image has a size of 3×H×W, where H×W is the size of the input image, 3 is the number of channels of the image, and after being encoded, the number of feature channels of the image is expanded to C.

[0019] According to the method of the first aspect of the present invention, the image features F of size C×H×W after encoding are input into the fuzzy information perception module for feature enhancement. The fuzzy information perception module consists of convolutional layers and strip convolutions. The calculation process is as follows: first, global flat pooling is performed on the image features F, and then strip convolution is used to obtain fuzzy information, resulting in the corresponding features F1 and F2.

[0020] F1 = Conv 3×1 (Avg(F))Conv 1×3 (Avg(F))

[0021] F2 = Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F))

[0022] Where Conv represents convolution and Avg represents average pooling;

[0023] Simultaneously, feature extraction is performed on image feature F to obtain global information feature F3:

[0024] F3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0025] Where ReLU represents the activation function and BN represents normalization;

[0026] Features F1, F2, and F3 are fused and weighted using the Sigmoid function to obtain the enhanced feature F_w.

[0027] F_w=F*Sigmoid(concate((F1,F2,F3),dim=0))

[0028] Here, concate means to concatenate, and dim means dimension. It concatenates and fuses features F1, F2, and F3 on the first dimension of the tensor.

[0029] According to the method of the first aspect of the present invention, the image features F of size C×H×W after encoding are input into the frequency information enhancement module for feature enhancement. The frequency information enhancement module consists of wavelet transform, convolutional layer, activation function, normalization layer and inverse wavelet transform. The computational flow is as follows: First, wavelet transform is performed on the image features F to obtain four sub-features: high-frequency, high-low-frequency, low-frequency, and low-low-frequency, as follows:

[0030] F HH ,F LH ,F HL ,F LL =DWT(F)

[0031] Where DWT represents wavelet transform;

[0032] After performing convolutional learning on high-frequency information, the high-frequency and low-frequency information are fused, and a frequency-enhanced feature map is obtained through inverse wavelet transform. The feature map is then mapped into a weighted feature map M using the sigmoid function.

[0033] M=Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL),F LL )))

[0034] Where IDWT represents the inverse wavelet transform;

[0035] Image feature F contains the original image content and frequency feature information. Further image features are extracted through convolution operations to obtain feature map F4. Multiplying F4 by the weighted feature map M yields the enhanced feature F_f containing high-frequency information.

[0036] F4=Conv(Norm(ReLU(Conv(F))))

[0037]

[0038] Norm represents normalization.

[0039] According to the method of the first aspect of the present invention, the enhanced features F_w and F_f, together with the blurred image features, are input into the decoder to obtain the deblurred image.

[0040] A second aspect of the present invention discloses an image deblurring system for improving the recognition rate of blurred images, the system comprising a processing unit configured to perform:

[0041] Image pairs are acquired, the image pairs including a blurred image and a clear image corresponding to the blurred image, the image pairs are used to train a network, the network including a deblurring network and an object detection network;

[0042] The blurred image is input into the deblurring network, which deblurs the image to obtain a deblurred image. The frequency reconstruction loss and content perception loss are calculated based on the deblurred image and the clear image.

[0043] The deblurred image and the sharp image are input into the object detection network, which performs object detection on the deblurred image to output the object detection image, and calculates the perceptual YOLO loss between the object detection image and the sharp image.

[0044] Frequency reconstruction loss, content-aware loss, and YOLO-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize its parameters, thereby completing the training of the network. The trained network is then used for image deblurring.

[0045] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the image deblurring method for improving the recognition rate of blurred images described in the first aspect of this disclosure.

[0046] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the image deblurring method for improving the recognition rate of blurred images described in the first aspect of this disclosure.

[0047] In summary, this invention features low computational complexity and strong generalization ability. By cascading image deblurring and object detection, it takes into account the machine's image perception requirements, thereby enhancing the performance of object detection in blurred images. The loss function of the entire network is composed of the frequency reconstruction loss and content-aware loss of the deblurred image generated by the deblurring network, and the perceptual YOLO loss of the object detection network. This improves both image deblurring effectiveness and object detection accuracy. Attached Figure Description

[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an image deblurring method to improve the recognition rate of blurred images;

[0050] Figure 2 A schematic diagram of the architecture of a deblurred object detection network;

[0051] Figure 3 This is a schematic diagram of the fuzzy information perception module.

[0052] Figure 4 This is a schematic diagram of the frequency information enhancement module. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Image blurring is one of the most common image information loss phenomena in daily life. As an important information carrier, blurring not only affects people's visual perception but also adversely impacts downstream computer vision tasks. This patent proposes an image deblurring scheme to improve the recognition rate of blurry images.

[0055] The first aspect of this invention discloses an image deblurring method for improving the recognition rate of blurred images, the method comprising:

[0056] Step S1: Obtain image pairs, the image pairs including a blurred image and a clear image corresponding to the blurred image, the image pairs are used to train the network, the network including a deblurring network and an object detection network;

[0057] Step S2: Input the blurred image into the deblurring network, which deblurs the image to obtain a deblurred image. Calculate the frequency reconstruction loss and content perception loss based on the deblurred image and the clear image.

[0058] Step S3: Input the deblurred image and the sharp image into the object detection network. The object detection network performs object detection on the deblurred image to output the object detection image and calculates the perceptual YOLO loss between the object detection image and the sharp image.

[0059] Step S4: The frequency reconstruction loss, content-aware loss, and YOLO-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network and using the trained network to perform image deblurring.

[0060] According to a first aspect of the method of the present invention, the deblurring network includes an encoder, a fuzzy information sensing module, a frequency information enhancement module, and a decoder; wherein:

[0061] The encoder consists of convolutional layers. A blurred image is input into the encoder, features are extracted, and a feature map of the blurred image is obtained.

[0062] The fuzzy information perception module uses strip convolution to collect and perceive fuzzy information, and uses a residual structure to obtain the overall fuzzy information. The feature map of the fuzzy image extracted by the encoder is input into the fuzzy information perception module, which uses strip convolution and activation function to perceive the fuzziness in different regions and in different directions of the image.

[0063] The frequency information enhancement module uses wavelet transform to transform the feature map of the blurred image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information. It then enhances the high-frequency information through convolution and activation functions, and combines spatial and frequency domain information to improve the recovery of texture details.

[0064] The two sets of enhanced features output by the fuzzy information perception module and the frequency information enhancement module are fused together, and the deblurred image is obtained through the decoder.

[0065] According to the method of the first aspect of the present invention, the target detection network is a YOLO-v8 network. The deblurred image is input into the target detection network, the target detection image is obtained through target detection, and the perceptual YOLO loss is calculated.

[0066] According to the method of the first aspect of the present invention, an image pair is acquired, and the preprocessed image has a size of 3×H×W, where H×W is the size of the input image, 3 is the number of channels of the image, and after being encoded, the number of feature channels of the image is expanded to C.

[0067] According to the method of the first aspect of the present invention, the image features F of size C×H×W after encoding are input into the fuzzy information perception module for feature enhancement. The fuzzy information perception module consists of convolutional layers and strip convolutions. The calculation process is as follows: first, global flat pooling is performed on the image features F, and then strip convolution is used to obtain fuzzy information, resulting in the corresponding features F1 and F2.

[0068] F1 = Conv 3×1 (Avg(F))+Conv 1×3 (Avg(F))

[0069] F2 = Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F))

[0070] Where Conv represents convolution and Avg represents average pooling;

[0071] Simultaneously, feature extraction is performed on image feature F to obtain global information feature F3:

[0072] F3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0073] Where ReLU represents the activation function and BN represents normalization;

[0074] Features F1, F2, and F3 are fused and weighted using the Sigmoid function to obtain the enhanced feature F_w.

[0075] F_w=F*Sigmoid(concate((F1,F2,F3),dim=0))

[0076] Here, concate means to concatenate, and dim means dimension. It concatenates and fuses features F1, F2, and F3 on the first dimension of the tensor.

[0077] According to the method of the first aspect of the present invention, the image features F of size C×H×W after encoding are input into the frequency information enhancement module for feature enhancement. The frequency information enhancement module consists of wavelet transform, convolutional layer, activation function, normalization layer and inverse wavelet transform. The computational flow is as follows: First, wavelet transform is performed on the image features F to obtain four sub-features: high-frequency, high-low-frequency, low-frequency, and low-low-frequency, as follows:

[0078] F HH ,F LH ,F HL ,F LL =DWT(F)

[0079] Where DWT represents wavelet transform;

[0080] After performing convolutional learning on high-frequency information, the high-frequency and low-frequency information are fused, and a frequency-enhanced feature map is obtained through inverse wavelet transform. The feature map is then mapped into a weighted feature map M using the sigmoid function.

[0081] M=Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL )))

[0082] Where IDWT represents the inverse wavelet transform;

[0083] Image feature F contains the original image content and frequency feature information. Further image features are extracted through convolution operations to obtain feature map F4. Multiplying F4 by the weighted feature map M yields the enhanced feature F_f containing high-frequency information.

[0084] F4=Conv(Norm(ReLU(Conv(F))))

[0085]

[0086] Norm represents normalization.

[0087] According to the method of the first aspect of the present invention, the enhanced features F_w and F_f, together with the blurred image features, are input into the decoder to obtain the deblurred image.

[0088] A second aspect of the present invention discloses an image deblurring system for improving the recognition rate of blurred images, the system comprising a processing unit configured to perform:

[0089] Image pairs are acquired, the image pairs including a blurred image and a clear image corresponding to the blurred image, the image pairs are used to train a network, the network including a deblurring network and an object detection network;

[0090] The blurred image is input into the deblurring network, which deblurs the image to obtain a deblurred image. The frequency reconstruction loss and content perception loss are calculated based on the deblurred image and the clear image.

[0091] The deblurred image and the sharp image are input into the object detection network, which performs object detection on the deblurred image to output the object detection image, and calculates the perceptual YOLO loss between the object detection image and the sharp image.

[0092] Frequency reconstruction loss, content-aware loss, and YOLO-aware loss are used as the overall loss of the network and fed back to the deblurring network to optimize its parameters, thereby completing the training of the network. The trained network is then used for image deblurring.

[0093] First Embodiment

[0094] This embodiment proposes an image deblurring method to improve the recognition rate of blurred images, such as... Figure 1 As shown, the network mainly consists of two parts: an image deblurring network and an object detection network. The main training data for the network consists of sharp-blurred data pairs, primarily sourced from existing datasets, and composed of images containing targets such as pedestrians and vehicles after filtering.

[0095] The network input is a blurred image. After training the deblurring network using both blurred and sharp images, a deblurred image is obtained. Frequency reconstruction loss and content-aware loss are calculated based on the deblurred and sharp images. Simultaneously, the deblurred and sharp images are input into an object detection network (this method uses a YOLOv8 object detection network) to calculate the YOLO aware loss. The frequency reconstruction loss, content-aware loss, and YOLO aware loss are summed as the overall network loss and fed back to the deblurring network for parameter training and optimization.

[0096] Second Embodiment

[0097] This embodiment proposes an image deblurring method to improve the recognition rate of blurred images, enhancing both the deblurring effect and the target detection accuracy of blurred images. It primarily improves upon traditional algorithms by refining the design of algorithms for perceiving blurred information and enhancing frequency information. For example... Figure 2 As shown, this network can adaptively learn the mapping relationship between blurred and sharp images, thereby improving the accuracy of object detection while better removing blur.

[0098] Compared to conventional image deblurring algorithms, the method proposed in this invention primarily focuses on the blur information perception and frequency information enhancement modules. In the blur information perception module, strip convolution is used to acquire and perceive blur information, employing a residual structure to obtain the overall blur information. In the frequency information enhancement module, wavelet transform is used to convert image features from the spatial domain to the frequency domain, and high-frequency information is enhanced through convolution to improve the recovery of texture details. Finally, the two sets of enhanced features are fused, and the resulting image is obtained through an encoder.

[0099] Third Embodiment

[0100] This embodiment illustrates the module composition of an image deblurring method to improve the recognition rate of blurred images.

[0101] 1. Input module

[0102] The existing blurred image dataset is used as training data, where blurred-sharp image pairs are used as input to the network.

[0103] 2. Encoder Module

[0104] The encoder mainly consists of convolutional layers. The blurred image is input into the encoder for feature extraction to obtain the feature map of the blurred image.

[0105] 3. Fuzzy Information Perception Module

[0106] The features of the blurred image extracted by the encoder are input into the blur perception module, such as... Figure 3 As shown, the blur perception module uses strip convolution and activation functions to perceive blur in different regions of an image, as well as blur in different directions.

[0107] 4. Frequency Enhancement Module

[0108] The features of the blurred image extracted by the encoder are input into the frequency enhancement module, such as... Figure 4 As shown, the frequency enhancement module divides image features into high-frequency and low-frequency information through wavelet transform, and enhances this information through convolution and activation functions. By combining spatial and frequency domain information, image information is better recovered.

[0109] 5. Decoder Module

[0110] The information features of the two modules are fused and input into the decoder module. Based on the training parameters of the blurred image pair, the deblurred image is output.

[0111] 6. Target Detection Module

[0112] The deblurred image is input into a target detection network to predict the target detection result, and the corresponding perceptual loss is fed back into the deblurred network for parameter update and optimization. In this invention, the YOLOv8 target detection algorithm is used to perform target detection on the deblurred image.

[0113] Fourth embodiment

[0114] This embodiment presents the flowchart design of an image deblurring method to improve the recognition rate of blurred images. The aim is to enhance the deblurring effect and target detection accuracy of blurred images. Therefore, the input is a blurred image, and the output is a deblurred image and the target detection result image. The algorithm flow is as follows:

[0115] Step S1: Acquire image data. The preprocessed input size is 3×H×W, where H×W is the size of the input image and 3 is the number of channels. After the encoder, the number of feature channels of the image is expanded to C to facilitate subsequent feature processing and enhancement.

[0116] Step S2: Input the C×H×W image features F into the blur perception module for feature enhancement. This network consists of convolutional layers and strip convolutions, such as... Figure 3 As shown. The overall calculation process is as follows: First, global average pooling is performed on the input features, and then strip convolution is used to obtain fuzzy information, resulting in the corresponding feature maps F1 and F2.

[0117] F1 = Conv 3×1 (Avg(F))+Conv 1×3 (Avg(F))

[0118] F2 = Conv 3×3 (Avg(F))+Conv 3×3 (Avg(F))

[0119] Feature extraction is performed on the input features to obtain global information features F3:

[0120] F3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F))))

[0121] Then, the fuzzy features F1, F2, and global information features are fused together and weighted using the Sigmoid function to obtain the fuzzy feature enhancement map F_w.

[0122] F_w=F*Sigmoid(concate((F1,F2,F3),dim=0))

[0123] Step S3: Input the C×H×W image features F into the frequency enhancement module for feature enhancement. This network consists of wavelet transform, convolutional layers, activation functions, normalization layers, and inverse wavelet transform. Figure 4 As shown. The overall calculation process is as follows: First, perform wavelet transform on the fuzzy feature map to obtain four sub-features: high-frequency, high-low-frequency, low-frequency, and low-low-frequency.

[0124] F HH ,F LH ,F HL ,F LL =DWT(F)

[0125] To enhance high-frequency information, convolutional learning is performed only on the subbands containing high-frequency information, allowing the network to adaptively learn the parameters of high-frequency information. Then, high-frequency and low-frequency information are fused, and a frequency-enhanced feature map is obtained through inverse wavelet transform. Finally, the feature map is mapped to a weighted feature map M using the sigmoid function.

[0126] M=Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL )))

[0127] The blurred image feature map F contains the original image content and frequency feature information. Through operations such as convolution, image features are further extracted to obtain feature map F1. This feature map F1 is then multiplied by the weighted feature map M to obtain the high-frequency information-enhanced feature map F_f, which is used for image deblurring. The calculation process is as follows:

[0128] F1=Conv(Norm(ReLU(Conv(F))))

[0129]

[0130] Step S4: Input the enhanced image features F_w and F_f, along with the blurred image features, into the decoder to obtain the deblurred image.

[0131] Step S5: Input the deblurred image into the target detection network to obtain the target detection result of the blurred image.

[0132] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the image deblurring method for improving the recognition rate of blurred images described in the first aspect of this disclosure.

[0133] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the image deblurring method for improving the recognition rate of blurred images described in the first aspect of this disclosure.

[0134] In summary, this invention proposes an image deblurring scheme to improve the recognition rate of blurred images. The proposed network uses a cascaded improved image deblurring network and an object detection network. This enables simultaneous deblurring and object detection of blurred images. Based on the actual situation of non-uniform blurring, this invention proposes a blur perception network. The blur perception network adaptively learns the blurring of different directions and magnitudes in the image, weighting the blur features, which is more conducive to image deblurring. This invention also proposes a frequency enhancement network, composed of wavelet transform and convolutional layers, to enhance high-frequency information in image features, which helps to restore image texture details and improve object detection performance.

[0135] This invention features low computational complexity and strong generalization ability. By cascading image deblurring and object detection, it takes into account the machine's image perception requirements, thereby enhancing the performance of object detection in blurred images. The loss function of the entire network is composed of the frequency reconstruction loss and content-aware loss of the deblurred image generated by the deblurring network, and the perceptual YOLO loss of the object detection network. This improves both image deblurring effectiveness and object detection accuracy.

[0136] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image deblurring method for improving the recognition rate of a blurred image, characterized by, The method comprises: Step S1, acquiring an image pair, the image pair comprising a blurred image and a clear image corresponding to the blurred image, the image pair being used for training a network, the network comprising a deblurring network and a target detection network; Step S2, inputting the blurred image into the deblurring network, performing deblurring processing on the blurred image by the deblurring network to obtain a deblurred image, and calculating a frequency reconstruction loss and a content perception loss according to the deblurred image and the clear image; Step S3, inputting the deblurred image and the clear image into the target detection network, performing target detection on the deblurred image by the target detection network to output a target detection image, and calculating a perception Yolo loss between the target detection image and the clear image; Step S4, taking the frequency reconstruction loss, the content perception loss and the Yolo perception loss as an overall loss of the network, and feeding back to the deblurring network to optimize parameters of the deblurring network, thereby completing training of the network and performing image deblurring processing by using the trained network; The deblurring network comprises an encoder, a blur information perception module, a frequency information enhancement module and a decoder; wherein: The encoder is composed of a convolution layer, the blurred image is input into the encoder, feature extraction is performed, and a feature map of the blurred image is obtained; The blur information perception module uses strip convolution to collect and perceive blur information, and uses a residual structure to obtain overall blur information; wherein the feature map of the blurred image extracted by the encoder is input into the blur information perception module, and the blur in different regions and the blur in different directions of the image are perceived through strip convolution and an activation function; The frequency information enhancement module uses wavelet transform to convert the feature map of the blurred image from a spatial domain to a frequency domain, divides the image features into high-frequency information and low-frequency information, convolves and enhances the high-frequency information through a convolution function and an activation function, and combines the spatial domain and frequency domain information to improve the recovery of texture details; Two groups of enhanced features output by the blur information perception module and the frequency information enhancement module are fused, and a deblurred image is obtained through the decoder; Wherein, the image pair is preprocessed, and the size of the image is 3xHxW, wherein HxW is the size of the input image, and 3 is the number of channels of the image; after the encoder, the number of feature channels of the image is expanded to C; Wherein, the image feature F with the size of CxHxW after the encoder is input into the frequency information enhancement module for feature enhancement, the frequency information enhancement module is composed of wavelet transform, a convolution layer, an activation function, a normalization layer and wavelet inverse transform, and the calculation flow process is as follows: first, wavelet transform is performed on the image feature F to obtain four sub-feature characteristics of high-high frequency, high-low frequency, low-high frequency and low-low frequency, which are as follows: F HH ,F LH ,F HL ,F LL = DWT(F) DWT represents wavelet transform; After convolution learning of the high-frequency information, the high-frequency information is fused with the low-frequency information, the frequency-enhanced feature map is obtained through wavelet inverse transform, and the feature map is mapped into a weight feature map M through a Sigmoid function: M = Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL ))) IDWT represents wavelet inverse transform; The image feature F contains original image content and frequency feature information, and the image feature is further extracted through convolution operation to obtain a feature map F4, which is multiplied by the weight feature map M to obtain an enhanced feature F_f of high frequency information: F4 = Conv(Norm(ReLU(Conv(F)))) Norm represents normalization.

2. The image deblurring method of claim 1, wherein, The target detection network is a Yolo-v8 network, the deblurred image is input into the target detection network, the target detection image is obtained through target detection, and the perception Yolo loss is calculated.

3. The image deblurring method of claim 2, wherein, The image feature F with a size of CxHxW after the encoder is input into the blur information perception module for feature enhancement. The blur information perception module is composed of convolution layers and strip convolution. The calculation process is as follows: first, the image feature F is globally flattened, and then the strip convolution is used to obtain the blur information, and the corresponding features F1 and F2 are obtained: F1 = Conv 3×1 (Avg(F)) + Conv 1×3 (Avg(F)) F2 = Conv 3×3 (Avg(F)) + Conv 3×3 (Avg(F)) Wherein, Conv represents convolution, and Avg represents average pooling. Meanwhile, the image feature F is extracted to obtain the global information feature F3: F3 = Conv 1×1 (BN(ReLU(Conv 1×1 (F)))) Wherein, ReLU represents an activation function, and BN represents normalization. The features F1, F2 and F3 are fused, and the enhanced feature F_w is obtained through weighted mapping of the Sigmoid function: F_w = F*Sigmoid(concate((F1,F2,F3),dim=0)) Wherein, concate represents splicing, and dim represents dimension. The features F1, F2 and F3 are spliced and fused in the first dimension of the tensor.

4. The image deblurring method of claim 3, wherein, The enhanced features F_w and F_f are input into the decoder together with the blur image feature to obtain the deblurred image.

5. An image deblurring system for improving the recognition rate of a blurred image, characterized by, The system comprises a processing unit configured to perform: obtain an image pair, the image pair comprising a blurred image and a clear image corresponding to the blurred image, the image pair being used to train a network, the network comprising a deblurring network and a target detection network; input the blurred image into the deblurring network, perform deblurring processing on the blurred image by the deblurring network to obtain a deblurred image, and calculate a frequency reconstruction loss and a content perception loss according to the deblurred image and the clear image; input the deblurred image and the clear image into the target detection network, perform target detection on the deblurred image by the target detection network to output a target detection image, and calculate a perception Yolo loss between the target detection image and the clear image; take the frequency reconstruction loss, the content perception loss and the Yolo perception loss as the overall loss of the network, and feed back to the deblurring network to optimize the parameters of the deblurring network, thereby completing the training of the network, and using the trained network to perform image deblurring processing; the deblurring network comprises an encoder, a blur information perception module, a frequency information enhancement module and a decoder; wherein: the encoder is composed of convolution layers, the blurred image is input into the encoder, the feature of the blurred image is extracted to obtain a feature map of the blurred image; The fuzzy information perception module adopts strip convolution to collect and perceive the fuzzy information, and uses a residual structure to obtain the overall fuzzy information; wherein, the feature map of the fuzzy image extracted by the encoder is input into the fuzzy information perception module, and the fuzzy in different regions and the fuzzy in different directions in the image are perceived through strip convolution and an activation function; The frequency information enhancement module adopts wavelet transform to convert the feature map of the fuzzy image from the spatial domain to the frequency domain, and divides the image features into high-frequency information and low-frequency information, and convolves and enhances the high-frequency information through a convolution function and an activation function, and combines the spatial domain and frequency domain information to improve the recovery of texture details; The two groups of enhanced features output by the fuzzy information perception module and the frequency information enhancement module are fused, and a deblurring image is obtained through a decoder; Wherein, the image pair is obtained, and the image size after preprocessing is 3xHxW, wherein HxW is the size of the input image, and 3 is the number of channels of the image, and after the encoder, the number of feature channels of the image is expanded to C; Wherein, the image feature F with a size of CxHxW after the encoder is input into the frequency information enhancement module for feature enhancement, and the frequency information enhancement module is composed of wavelet transform, convolution layer, activation function, normalization layer and wavelet inverse transform, and the calculation flow process is as follows: first, wavelet transform is performed on the image feature F to obtain four sub-feature characteristics of high-high frequency, high-low frequency, low-high frequency and low-low frequency, which are as follows: F HH ,F LH ,F HL ,F LL =DWT(F) DWT represents wavelet transform; After convolution learning of the high-frequency information, the high-frequency information and the low-frequency information are fused, the frequency enhanced feature map is obtained through wavelet inverse transform, and the feature map is mapped into a weight feature map M through a Sigmoid function: M = Sigmoid(IDWT(concate(Conv(F HH ,F LH ,F HL ),F LL ))) IDWT represents wavelet inverse transform; The image feature F contains original image content and frequency feature information, the image feature is further extracted through convolution operation to obtain a feature map F4, and the feature map F4 is multiplied by the weight feature map M to obtain the enhanced feature F_f of the high-frequency information: F4=Conv(Norm(ReLU(Conv(F))) Norm represents normalization.

6. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the image deblurring method for improving the recognition rate of a blurred image according to any one of claims 1-4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the image deblurring method for improving the recognition rate of a blurred image according to any one of claims 1-4.

Images