Image deblurring method, system and device for enhancing barcode and character recognition

Through multi-scale edge feature extraction and frequency feature selection processing, combined with wavelet reconstruction and frequency domain information supervision, the image defuzzing algorithm is optimized, which solves the problems of poor defuzzing effect and insufficient adaptability in the existing technology, and achieves high efficiency and adaptability The highly sensitive image debuffering effect improves the performance of barcode and character recognition systems.

CN118822903BActive Publication Date: 2024-11-26SHANDONG UNIV OF SCI & TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411280556.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-26
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

The existing defuzzing algorithms are difficult to meet high real-time and adaptability in industrial scenarios, resulting in poor defuzzing effects of barcode and character recognition systems, introducing artifacts, poor real-time performance, insufficient adaptability, and affecting recognition performance and efficiency.

Method used

Using multi-scale edge feature extraction and frequency feature selection processing, through edge fusion, spatial feature selection and frequency filtering technology, combined with wavelet reconstruction and frequency domain information loss value supervision, optimize the image defuzzing process, reduce artifacts, and improve image texture recovery. Spend.

Benefits of technology

It has achieved improvement in the defuzzing effect, strong adaptability, and can handle various types and degrees of blurred images, significantly improving the performance and efficiency of the automatic barcode recognition system of characters.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present invention relates to the field of image enhancement technology, and specifically to an image deblurring method, system and device for enhancing barcode and character recognition. In order to solve the technical problem of poor image deblurring effect in the prior art, the present invention first performs feature fusion processing on edge feature images of different scales of a blurred image and a blurred feature map to obtain a plurality of edge fusion feature maps of different scales. Then, each edge fusion feature map is subjected to spatial feature selection processing and frequency feature selection processing to obtain a plurality of frequency filtering feature maps of different scales. Finally, the structural integrity, spatial content integrity and frequency domain content integrity of an output image are supervised by using a wavelet reconstruction loss value, a spatial information loss value and a frequency domain information loss value, and the output image is subjected to fusion processing to obtain a deblurred map. The method has strong adaptability and good deblurring effect, and can be applied to a barcode and character automatic recognition system to improve overall performance and efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to an image deblurring method, system and device for enhancing barcode and character recognition. Background Art

[0002] In practical applications, barcode and character images often become blurred due to factors such as lighting conditions, camera quality, motion blur, and environmental noise. These blur and noise issues seriously affect the recognition rate of barcodes and characters, leading to misreading or missed reading, which in turn reduces the overall performance and efficiency of the image recognition system. Therefore, solving these blurred image quality issues is crucial to improving the reliability and efficiency of barcode and character recognition.

[0003] Among the existing technologies, the image blurriness is reduced as much as possible by improving the quality of hardware equipment; although high-quality hardware equipment, such as high-resolution cameras, high-quality lenses, autofocus systems, and high-performance image sensors, can significantly improve image quality and reduce blur and noise, the cost of these hardware is usually high; the purchase and maintenance costs of high-performance hardware increase the cost of image processing, limiting its popularity in large-scale applications. In order to reduce costs and improve adaptability, deblurring algorithms are currently used to improve image quality, but existing deblurring algorithms are mainly designed for natural images, with complex structures and large computational workloads, making it difficult to meet the needs of high-real-time industrial scene images. When used in industrial scene image processing, problems such as poor deblurring effects, introduction of artifacts, poor real-time performance, and insufficient adaptability will occur. These problems are fatal to the readability of barcodes and characters, and seriously affect the overall performance and efficiency of barcode and character automatic recognition systems. Summary of the invention

[0004] The object of the present invention is to provide an image deblurring method, system and device for enhancing barcode and character recognition with good deblurring effect and strong adaptability.

[0005] The technical solution of the present invention is as follows:

[0006] An image deblurring method for enhancing barcode and character recognition includes the following operations:

[0007] S1. The blurred image is processed by edge extraction to obtain an edge image; edge feature maps of different scales of the edge image are obtained to form a multi-scale edge feature atlas; different scale fuzzy maps of the blurred image are obtained, and channel feature extraction is performed respectively to obtain a number of fuzzy feature maps of different scales to form a multi-scale fuzzy atlas; each edge feature map in the multi-scale edge feature atlas is subjected to feature fusion processing with the fuzzy feature map of the corresponding scale in the multi-scale fuzzy atlas to obtain a number of edge fusion feature maps of different scales;

[0008] S2, several edge fusion feature maps of different scales are processed by spatial feature selection to obtain several spatial feature selection maps of different scales; several spatial feature selection maps of different scales are processed by frequency feature selection to obtain several frequency filtering feature maps of different scales;

[0009] S3, obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up; obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing up; perform spatial domain to frequency domain conversion on the frequency filtering feature map of each scale and the standard map of the corresponding scale, and obtain the frequency domain feature map and the frequency domain standard map of each scale; obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of the corresponding scale, and obtain the frequency domain information loss value after summing up; when the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales;

[0010] S4. Fusing several optimized frequency filtering feature maps of different scales to obtain a deblurred map.

[0011] The operation of obtaining edge feature maps of different scales of the edge image in S1 is as follows: the edge image is subjected to edge region feature extraction to obtain a first-scale edge feature map; the first-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a second-scale edge feature map; the second-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a third-scale edge feature map; and so on to obtain an N-th scale edge feature map; N is a positive integer; the scale edge feature maps of all scales form a multi-scale edge feature atlas; the operation of edge region feature extraction is as follows: the input is subjected to convolution with a kernel of 3×3 Ordinary convolution, nonlinear processing, dilated convolution, nonlinear processing, depthwise separable convolution and nonlinear processing are performed to obtain an edge fine-grained map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3 and nonlinear processing to obtain a first edge fine-grained feature map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3, nonlinear processing, ordinary convolution with a convolution kernel of 1×1 and nonlinear processing to obtain a second edge fine-grained feature map; the second edge fine-grained feature map is multiplied element-by-element with the first edge fine-grained feature map, and then added element-by-element with the edge fine-grained map, and the output obtained is used as the edge feature map.

[0012] The operation of feature fusion processing in S1 is specifically as follows: the current edge feature map is subjected to ordinary convolution with a convolution kernel of 3×3 and nonlinear processing to obtain the current edge convolution feature map; the current edge convolution feature map is subjected to ordinary convolution with a convolution kernel of 1×1 and nonlinear processing, and then multiplied element-by-element with the fuzzy feature map of the corresponding scale to obtain the current initial fusion feature map; the initial fusion feature map and the current edge convolution feature map are added element-by-element to obtain the current edge fusion feature map.

[0013] The operations of spatial feature selection processing in S2 are specifically as follows: the current edge fusion feature map is processed by coordinate attention to obtain the current coordinate attention fusion map; the current coordinate attention fusion map is subjected to the first depth-wise separable convolution and the second depth-wise separable convolution respectively to obtain the current first edge channel separation feature map and the current second edge channel separation feature map; the current coordinate attention fusion map is subjected to average pooling processing and maximum pooling processing respectively, and then superimposed and convolved to obtain the current spatial degraded focus map; the current spatial degraded focus map is subjected to channel dimension feature replication processing, and then element-wise multiplication processing is performed with the current first edge channel separation feature map to obtain the current initial spatial selection map; the current initial spatial selection map and the current second edge channel separation feature map are subjected to element-wise addition processing to obtain the current spatial feature selection map.

[0014] The specific operation of frequency feature selection processing in S2 is: the current spatial feature selection map is processed by mean filtering to obtain the current low-frequency feature map; the difference map between the current spatial feature selection map and the current low-frequency feature map is obtained to obtain the current complementary high-frequency feature map; the current complementary high-frequency feature map is multiplied element by element with the current spatial feature selection map, and then added element by element with the current spatial feature selection map to obtain the current frequency filtering feature map.

[0015] The operations of converting the fusion feature selection map and the standard map from the spatial domain to the frequency domain in S3 are as follows: the frequency filtering feature map or the standard map is gray-scale processed to obtain a gray-scale map; the Fourier transform map of the gray-scale map is obtained, and the pixel information in the Fourier transform map whose pixel frequency is less than the frequency threshold is moved to the center of the spectrum to obtain the spectrum amplitude map; the spectrum amplitude map is low-pass filtered, and then inverse Fourier transformed and denoised to obtain an inverse Fourier denoised map; the inverse Fourier denoised map is high-pass filtered and Fourier transformed to obtain a frequency domain feature map or a frequency domain standard map.

[0016] The fusion processing operation in S4 is specifically as follows: several optimized frequency filtering feature maps of different scales are arranged in order from small to large scale to obtain several sorted optimized frequency filtering feature maps; the previous sorted optimized frequency filtering feature map and the current sorted optimized frequency filtering feature map are subjected to superposition processing and residual block processing to obtain the current clear feature map, which is used to perform superposition processing and residual block processing operations with the next sorted optimized frequency filtering feature map; when the last sorted optimized frequency filtering feature map completes the superposition processing and residual block processing operations, a deblurred map is obtained.

[0017] An image deblurring system for enhancing barcode and character recognition, used to implement the above-mentioned image deblurring method for enhancing barcode and character recognition, comprising:

[0018] The edge fusion feature map generation module is used to obtain an edge image by edge extraction processing of a blurred image; obtain edge feature maps of different scales of the edge image to form a multi-scale edge feature map set; obtain fuzzy maps of different scales of the blurred image, respectively, and obtain a number of fuzzy feature maps of different scales by channel feature extraction processing to form a multi-scale fuzzy map set; perform feature fusion processing on each edge feature map in the multi-scale edge feature map set and the fuzzy feature map of the corresponding scale in the multi-scale fuzzy map set to obtain a number of edge fusion feature maps of different scales;

[0019] The frequency filtering feature map generation module is used to process several edge fusion feature maps of different scales through spatial feature selection to obtain several spatial feature selection maps of different scales; and to process several spatial feature selection maps of different scales through frequency feature selection to obtain several frequency filtering feature maps of different scales;

[0020] The optimized frequency filtering feature map generation module is used to obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up; obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing up; the frequency filtering feature map of each scale and the standard map of the corresponding scale are subjected to spatial domain frequency domain conversion processing respectively to obtain the frequency domain feature map and the frequency domain standard map of each scale; obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of the corresponding scale, and obtain the frequency domain information loss value after summing up; when the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales;

[0021] The deblurred image generation module fuses several optimized frequency filtering feature images of different scales to obtain a deblurred image.

[0022] An image deblurring device for enhancing barcode and character recognition comprises a processor and a memory, wherein when the processor executes a computer program stored in the memory, the above-mentioned image deblurring method for enhancing barcode and character recognition is implemented.

[0023] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the image deblurring method for enhancing barcode and character recognition is implemented.

[0024] The beneficial effects of the present invention are:

[0025] The present invention provides an image deblurring method for enhancing barcode and character recognition. First, edge feature images containing key information at different scales of blurred images and blurred feature maps containing complete information at different scales are subjected to feature fusion processing of corresponding scales to enhance edge information of images at different scales, make the contours and shapes of images at different scales more obvious, reduce noise influence, effectively reduce blur influence, and obtain several edge fusion feature maps at different scales; then, spatial feature selection processing and frequency feature selection processing are performed on the several edge fusion feature maps at different scales to enhance the key information expression capability and significantly improve the image expression quality, and obtain several different The frequency filtering feature map of different scales is obtained; then, the structural integrity, spatial content integrity and frequency domain integrity of the image are output through the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value, so as to improve the quality of deblurring and obtain several optimized frequency filtering feature maps of different scales; finally, several optimized frequency filtering feature maps of different scales are fused to make the image texture restoration more natural and reduce the artifacts that may be generated in the deblurring process to obtain a deblurred map; this method can process blurred images of various types and degrees, has strong adaptability and good deblurring effect, and can be applied to barcode and character automatic recognition systems to improve the overall performance and efficiency. DETAILED DESCRIPTION

[0026] This embodiment provides an image deblurring method for enhancing barcode and character recognition, including the following operations:

[0027] S1. The blurred image is processed by edge extraction to obtain an edge image; edge feature maps of different scales of the edge image are obtained to form a multi-scale edge feature atlas; different scale fuzzy maps of the blurred image are obtained, and channel feature extraction is performed respectively to obtain a number of fuzzy feature maps of different scales to form a multi-scale fuzzy atlas; each edge feature map in the multi-scale edge feature atlas is subjected to feature fusion processing with the fuzzy feature map of the corresponding scale in the multi-scale fuzzy atlas to obtain a number of edge fusion feature maps of different scales;

[0028] S2, several edge fusion feature maps of different scales are processed by spatial feature selection to obtain several spatial feature selection maps of different scales; several spatial feature selection maps of different scales are processed by frequency feature selection to obtain several frequency filtering feature maps of different scales;

[0029] S3, obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up; obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing up; perform spatial domain to frequency domain conversion on the frequency filtering feature map of each scale and the standard map of the corresponding scale, and obtain the frequency domain feature map and the frequency domain standard map of each scale; obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of the corresponding scale, and obtain the frequency domain information loss value after summing up; when the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales;

[0030] S4. Fusing several optimized frequency filtering feature maps of different scales to obtain a deblurred map.

[0031] S1. The blurred image is processed by edge extraction to obtain an edge image; edge feature maps of the edge image at different scales are obtained to form a multi-scale edge feature atlas; fuzzy maps of different scales of the blurred image are obtained, and channel feature extraction is performed on each edge feature map in the multi-scale edge feature atlas to obtain a plurality of fuzzy feature maps at different scales to form a multi-scale fuzzy atlas; feature fusion is performed on each edge feature map in the multi-scale edge feature atlas with the fuzzy feature map of the corresponding scale in the multi-scale fuzzy atlas to obtain a plurality of edge fusion feature maps at different scales.

[0032] The edge feature images containing key information at different scales of the blurred image are fused with the blurred feature maps containing complete information at different scales to enhance the edge information of images of different scales, make the contours and shapes of images of different scales more obvious, reduce the influence of noise, effectively reduce the influence of blur, and obtain several edge fusion feature maps of different scales.

[0033] First, the blurred image is subjected to edge extraction processing to obtain an edge image, which is used to make up for the deficiency of the prior art that focuses on removing the overall blur of the image and easily loses important information, so that important information in the enhanced image is retained and the image quality is improved. The above-mentioned operation of obtaining the edge image includes but is not limited to being implemented by the Sobel operator.

[0034] Then, edge feature maps of different scales of the edge image are obtained to form a multi-scale edge feature atlas. Image edge features are extracted at different scales so that detail information can be retained and restored, further enhancing the deblurring effect and image quality.

[0035] The operation of obtaining edge feature maps of different scales of the edge image is specifically as follows: the edge image is subjected to edge region feature extraction to obtain a first scale edge feature map; the first scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a second scale edge feature map; the second scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a third scale edge feature map; and so on to obtain an Nth scale edge feature map; N is a positive integer; all scale edge feature maps form a multi-scale edge feature map set. The above upsampling operation includes but is not limited to being realized by a dilated convolution with a convolution kernel of 3×3.

[0036] To facilitate understanding by those skilled in the art, an example is given below: an edge image is subjected to edge region feature extraction to obtain a first-scale edge feature map; the first-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a second-scale edge feature map; the second-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a third-scale edge feature map; the first-scale edge feature map, the second-scale edge feature map and the third-scale edge feature map form a multi-scale edge feature atlas.

[0037] The above-mentioned edge area feature extraction operation is as follows: the input is subjected to ordinary convolution with a convolution kernel of 3×3, nonlinear processing (which can be achieved through Relu activation function), hollow convolution, nonlinear processing (which can be achieved through Relu activation function), depth-separable convolution and nonlinear processing (which can be achieved through Relu activation function), to collect finer-grained feature information in the input image and obtain an edge fine-grained map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3 and nonlinear processing (which can be achieved through Relu activation function) to obtain a first edge fine-grained feature map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3, nonlinear processing (which can be achieved through Relu activation function), ordinary convolution with a convolution kernel of 1×1 and nonlinear processing (which can be achieved through Relu activation function and Sigmoid function in sequence) to obtain a second edge fine-grained feature map; the second edge fine-grained feature map is multiplied element-by-element with the first edge fine-grained feature map, and then added element-by-element with the edge fine-grained map, and the output obtained is used as the edge feature map. The whole process focuses on the important edge area information in the input image and suppresses unimportant information, thereby further refining the edge features and obtaining an edge feature map with prominent details, which is used to subsequently improve the deblurring capability and image quality.

[0038] The number of times the above-mentioned ordinary convolution, dilated convolution, and depth-wise separable convolution are performed in each step includes but is not limited to once.

[0039] At the same time, different scale fuzzy images of the blurred image are obtained, and they are processed by channel feature extraction to obtain several different scale fuzzy feature maps, forming a multi-scale fuzzy atlas. Different scale fuzzy images that can reflect both the complete information of the blurred image and the different global and local information of the blurred image are obtained (different scales have different expression capabilities of global and local information of the image), and they are processed by channel feature extraction to enrich the channel features so that the channel dimension of the output image corresponds to the edge feature image. Several different scale fuzzy feature maps are obtained to form a multi-scale fuzzy atlas, which is used to further enhance the deblurring effect and enhance the image quality. The above channel feature extraction and processing operations can be implemented through the ResBlock block.

[0040] Finally, each edge feature map in the multi-scale edge feature map set is fused with the fuzzy feature map of the corresponding scale in the multi-scale fuzzy map set, and the edge information is further enhanced, making the contour and shape of the image more obvious, reducing the influence of noise, and effectively reducing the influence of blur, so as to obtain several edge fusion feature maps of different scales.

[0041] The above-mentioned feature fusion processing operation is specifically as follows: the current edge feature map is subjected to ordinary convolution with a convolution kernel of 3×3 and nonlinear processing (which can be realized by Relu activation function) to refine the edge features and obtain the current edge convolution feature map; the current edge convolution feature map is subjected to ordinary convolution with a convolution kernel of 1×1 and nonlinear processing (which can be realized by Sigmoid function), and then multiplied element by element with the fuzzy feature map of the corresponding scale to obtain the current initial fusion feature map; the initial fusion feature map and the current edge convolution feature map are added element by element to obtain the current edge fusion feature map. After feature fusion processing is performed on all edge feature maps and fuzzy feature maps of corresponding scales, several edge fusion feature maps of different scales are obtained. The above method can focus on the important edge area information in the image, improve the deblurring effect and the accuracy of image recognition.

[0042] The feature fusion processing operation can be achieved through the following formula:

[0043] F 1 =δ(Conv 3×3 (F e )) ,

[0044] F map =σ(Conv 1×1 (F 1 )) ,

[0045] F 2 =F c ×F map +F c ,

[0046] F out =F 1 +F 2 ,

[0047] F 1 is the current edge convolution feature map, δ is the Relu activation function, F e is the current edge feature map, F map is the current edge nonlinear feature map, σ is the Sigmoid activation function, F 1is the current edge convolution feature map, F 2 is the current initial fusion feature map, F out is the current edge fusion feature map, Conv 3×3 , Conv 1×1 They are ordinary convolutions with convolution kernels of 3×3 and 1×1 respectively.

[0048] S2. Several edge fusion feature maps of different scales are processed by spatial feature selection to obtain several spatial feature selection maps of different scales; several spatial feature selection maps of different scales are processed by frequency feature selection to obtain several frequency filtering feature maps of different scales.

[0049] Several edge fusion feature maps of different scales are processed with spatial feature selection and frequency feature selection, focusing on the key features in the spatial domain and frequency domain, enhancing the ability to express key information, significantly improving the image expression quality, and obtaining several frequency filtering feature maps of different scales.

[0050] Firstly, in order to focus on the key area feature extraction in the spatial domain, several edge fusion feature maps of different scales are processed by spatial feature selection to obtain several spatial feature selection maps of different scales with significantly improved feature expression ability.

[0051] Among them, the operation of spatial feature selection processing is specifically as follows: the current edge fusion feature map is processed by coordinate attention, and the importance of each channel and its corresponding position information are comprehensively considered to obtain the current coordinate attention fusion map; the current coordinate attention fusion map is respectively subjected to the first depthwise separable convolution (which can be achieved by cascading convolution kernels with depthwise separable convolution kernels of 5×5 and 7×7 respectively) and the second depthwise separable convolution (which can be achieved by cascading convolution kernels with depthwise separable convolution kernels of 3×3 respectively), and the channel separation transformation is further performed to generate the channel representation, and the current first edge channel separation feature map and the current second Edge channel separation feature map; the current coordinate attention fusion map is respectively processed by average pooling and maximum pooling to realize the compression of the current coordinate attention fusion map along the channel dimension, and then superimposed and convolved to obtain the current spatial degraded focus map; the current spatial degraded focus map is processed by channel dimension feature replication, that is, the current spatial degraded focus map is replicated several times along the channel dimension, and then multiplied element by element with the current first edge channel separation feature map to obtain the current initial space selection map; the current initial space selection map and the current second edge channel separation feature map are added element by element to obtain the current space feature selection map.

[0052] Finally, since the current spatial feature selection map and the clear standard map have similar low-frequency components, but also have different high-frequency components, which affects the image deblurring quality, several spatial feature selection maps of different scales are processed by frequency feature selection, and the lowest frequency information in the spatial feature selection map is removed to further restore the intensity feature, and several frequency filtering feature maps of different scales are obtained.

[0053] The above-mentioned frequency feature selection processing operations are specifically as follows: the current spatial feature selection map is processed by mean filtering to obtain the current low-frequency feature map; the difference map between the current spatial feature selection map and the current low-frequency feature map is obtained, and the low-frequency signal of the current low-frequency feature map is subtracted from the current spatial feature selection map to obtain complementary high-frequency features to obtain the current complementary high-frequency feature map; the current complementary high-frequency feature map is multiplied element by element with the current spatial feature selection map, and then added element by element with the current spatial feature selection map (which can be achieved through residual linking) to obtain the current frequency filtering feature map.

[0054] S3. Obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of corresponding scales on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up. Obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of corresponding scales, and obtain the spatial information loss value after summing up. Perform spatial domain-frequency conversion on the frequency filtering feature map of each scale and the standard map of corresponding scale, and obtain the frequency domain feature map and frequency domain standard map of each scale. Obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of corresponding scales, and obtain the frequency domain information loss value after summing up. When the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales.

[0055] The wavelet reconstruction loss value, spatial information loss value and frequency domain information loss value are used to supervise the structural integrity, spatial content integrity and frequency domain content integrity of the output image, thereby improving the quality of deblurring.

[0056] Firstly, several frequency filtering feature maps of different scales are obtained and compared with the standard maps of corresponding scales. The average pixel difference on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image are summed up to obtain the wavelet reconstruction loss value. The purpose is to supervise the structure and clarity of the output image, ensure the structural integrity of the output image, and improve the quality of deblurring.

[0057] The acquisition of the above-mentioned image low-frequency components, image horizontal high-frequency components, image vertical high-frequency components and image diagonal high-frequency components can be achieved by decomposing the image into an image low-frequency component map, an image horizontal high-frequency component map, an image vertical high-frequency component map and an image diagonal high-frequency component map with different frequency information through discrete wavelet transform.

[0058] The wavelet reconstruction loss function can be obtained by the following formula:

[0059] ,

[0060] L wav is the wavelet reconstruction loss value, K is the total number of scales, k For the k scale, S d,k For the k The pixel information of the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image, or the diagonal high-frequency component of the image is filtered by the frequency filtering feature map of the scale. s d,k For the k The pixel information of the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image, or the diagonal high-frequency component of the image of the standard image of the scale, LL , LH , HL and HH They are low-frequency component, horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component respectively, and ‖ ‖1 is the paradigm processing.

[0061] At the same time, the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale is obtained, and the spatial information loss value is obtained after summation. The purpose is to supervise the loss of spatial domain content information of the output image, which can more effectively restore the content information of the image, ensure the spatial content integrity of the output image, and improve the quality of deblurring.

[0062] The spatial information loss function can be obtained by the following formula:

[0063] ,

[0064] L cont is the spatial information loss value, K is the total number of scales, k For the k scale, S k For the k The pixel information of the frequency-filtered feature map at each scale, s k For thek Pixel information of a standard image of a certain scale.

[0065] In addition, the frequency filtering feature map of each scale and the standard map of the corresponding scale are respectively processed by spatial domain frequency domain conversion to obtain frequency domain feature maps and frequency domain standard maps of different scales; the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard maps of the corresponding scale is obtained, and the frequency domain information loss value is obtained after summation. The purpose is to supervise the detail loss of the output image at the frequency domain level, which can more effectively restore the high-frequency information of the image and ensure the integrity of the frequency domain content of the output image, thereby improving the deblurring effect.

[0066] The specific operations of the spatial domain frequency domain conversion processing are as follows: the frequency filtering feature map or standard map is gray-scale processed to obtain a gray-scale map; the Fourier transform map of the gray-scale map is obtained, and the low-frequency components (pixel information with pixel frequency less than the frequency threshold) in the Fourier transform map are moved to the center of the spectrum (which can be set according to actual needs) to obtain the spectrum amplitude map; the spectrum amplitude map is low-pass filtered, and then inverse Fourier transformed and denoised to obtain an inverse Fourier denoised map; the inverse Fourier denoised map is high-pass filtered and Fourier transformed to obtain a frequency domain feature map or a frequency domain standard map.

[0067] The frequency domain information loss function can be obtained by the following formula:

[0068] ,

[0069] L fre is the frequency domain information loss value, F( S k ) is the k The pixel information of the frequency domain feature map of each scale, F( s k ) is the k The pixel information of the frequency domain standard image at each scale.

[0070] Finally, when the sum of the wavelet reconstruction loss value, the spatial information loss value, and the frequency domain information loss value is less than the loss threshold, several frequency filtering feature maps of different scales are output as several optimized frequency filtering feature maps of different scales. The above process can be realized by the training neural network model corresponding to S1 and S2.

[0071] S4. Fusing several optimized frequency filtering feature maps of different scales to make the image texture restoration more natural and reduce the artifacts that may be generated in the deblurring process, thereby obtaining a deblurred map.

[0072] The fusion processing operation is specifically as follows: several optimized frequency filtering feature maps of different scales are arranged in order from small to large scale to obtain several sorted optimized frequency filtering feature maps; the previous sorted optimized frequency filtering feature map and the current sorted optimized frequency filtering feature map are processed by superposition and residual block to obtain the current clear feature map, which is used to perform superposition and residual block processing operations with the next sorted optimized frequency filtering feature map; when the last sorted optimized frequency filtering feature map completes the superposition and residual block processing operations, a deblurred map is obtained. Among them, after the first sorted optimized frequency filtering feature map is processed by the residual block, it is superimposed with the second sorted optimized frequency filtering feature map.

[0073] To facilitate understanding by those skilled in the art, an example is given below: three optimized frequency filtering feature maps of different scales are arranged in order from small to large scale to obtain a first optimized frequency filtering feature map, a second optimized frequency filtering feature map and a third optimized frequency filtering feature map; after the first optimized frequency filtering feature map is processed by a residual block, it is superimposed and processed by a residual block with the second sorted optimized frequency filtering feature map to obtain an initial deblurred map; the initial deblurred map and the third sorted optimized frequency filtering feature map are superimposed and processed by a residual block to obtain a deblurred map.

[0074] This embodiment also provides an image deblurring system for enhancing barcode and character recognition, which is used to implement the above-mentioned image deblurring method for enhancing barcode and character recognition, including:

[0075] The edge fusion feature map generation module is used to obtain an edge image by edge extraction processing of a blurred image; obtain edge feature maps of different scales of the edge image to form a multi-scale edge feature map set; obtain fuzzy maps of different scales of the blurred image, respectively, and obtain a number of fuzzy feature maps of different scales by channel feature extraction processing to form a multi-scale fuzzy map set; perform feature fusion processing on each edge feature map in the multi-scale edge feature map set and the fuzzy feature map of the corresponding scale in the multi-scale fuzzy map set to obtain a number of edge fusion feature maps of different scales;

[0076] The frequency filtering feature map generation module is used to process several edge fusion feature maps of different scales through spatial feature selection to obtain several spatial feature selection maps of different scales; and to process several spatial feature selection maps of different scales through frequency feature selection to obtain several frequency filtering feature maps of different scales;

[0077] The optimized frequency filtering feature map generation module is used to obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up; obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing up; the frequency filtering feature map of each scale and the standard map of the corresponding scale are subjected to spatial domain frequency domain conversion processing respectively to obtain the frequency domain feature map and the frequency domain standard map of each scale; obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of the corresponding scale, and obtain the frequency domain information loss value after summing up; when the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales;

[0078] The deblurred image generation module fuses several optimized frequency filtering feature images of different scales to obtain a deblurred image.

[0079] This embodiment also provides an image deblurring device for enhancing barcode and character recognition, including a processor and a memory, wherein when the processor executes a computer program stored in the memory, the above-mentioned image deblurring method for enhancing barcode and character recognition is implemented.

[0080] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, the above-mentioned image deblurring method for enhancing barcode and character recognition is implemented.

[0081] The present embodiment provides an image deblurring method for enhancing barcode and character recognition. First, edge feature images containing key information at different scales of a blurred image and blurred feature maps containing complete information at different scales are subjected to feature fusion processing of corresponding scales to enhance edge information of images at different scales, make contours and shapes of images at different scales more obvious, reduce noise influence, effectively reduce blur influence, and obtain edge fusion feature maps at different scales. Then, spatial feature selection processing and frequency feature selection processing are performed on the edge fusion feature maps at different scales to enhance the key information expression capability and significantly improve the image expression quality, and obtain several edge fusion feature maps at different scales. The frequency filtering feature map of different scales is obtained; then, the structural integrity, spatial content integrity and frequency domain integrity of the image are output through the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value, so as to improve the quality of deblurring and obtain several optimized frequency filtering feature maps of different scales; finally, several optimized frequency filtering feature maps of different scales are fused to make the image texture restoration more natural and reduce the artifacts that may be generated in the deblurring process to obtain a deblurred map; this method can process blurred images of various types and degrees, has strong adaptability and good deblurring effect, and can be applied to barcode and character automatic recognition systems to improve the overall performance and efficiency.

Claims

1. An image deblurring method for enhancing barcode and character recognition, characterized in that: The following operations are included: S1. The blurred image is subjected to edge extraction processing to obtain an edge image; edge feature maps of different scales of the edge image are obtained to form a multi-scale edge feature map set; Obtain different scale fuzzy images of the fuzzy image, respectively extract and process the channel features, obtain several different scale fuzzy feature maps, and form a multi-scale fuzzy atlas; Each edge feature map in the multi-scale edge feature map set is subjected to feature fusion processing with a fuzzy feature map of a corresponding scale in the multi-scale fuzzy map set to obtain edge fusion feature maps of several different scales; S2, several edge fusion feature maps of different scales are processed by spatial feature selection to obtain several spatial feature selection maps of different scales; several spatial feature selection maps of different scales are processed by frequency feature selection to obtain several frequency filtering feature maps of different scales; S3, obtaining a plurality of frequency filtering feature maps of different scales and comparing them with the standard map of the corresponding scale, and summing the average pixel differences on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image, and the diagonal high-frequency component of the image to obtain the wavelet reconstruction loss value; Obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing them up; The frequency filtering feature map of each scale and the standard map of the corresponding scale are respectively processed by spatial domain frequency domain conversion to obtain the frequency domain feature map and frequency domain standard map of each scale; the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard maps of the corresponding scales is obtained, and the frequency domain information loss value is obtained by summing them up; When the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, a number of frequency filtering feature maps of different scales are output as the optimized frequency filtering feature maps of different scales; S4. Fusing several optimized frequency filtering feature maps of different scales to obtain a deblurred map.

2. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: In S1, the operation of obtaining edge feature maps of different scales of the edge image is specifically as follows: The edge image is subjected to edge region feature extraction to obtain a first-scale edge feature map; the first-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a second-scale edge feature map; the third-scale edge feature map is subjected to downsampling processing and edge region feature extraction to obtain a third-scale edge feature map; The Nth scale edge feature map is obtained by analogy; N is a positive integer; Scale edge feature maps of all scales form the multi-scale edge feature atlas; The operation of edge area feature extraction is: the input is subjected to ordinary convolution with a convolution kernel of 3×3, nonlinear processing, void convolution, nonlinear processing, depth-separable convolution and nonlinear processing to obtain an edge fine-grained map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3 and nonlinear processing to obtain a first edge fine-grained feature map; the edge fine-grained map is subjected to ordinary convolution with a convolution kernel of 3×3, nonlinear processing, ordinary convolution with a convolution kernel of 1×1 and nonlinear processing to obtain a second edge fine-grained feature map; the second edge fine-grained feature map is multiplied element-by-element with the first edge fine-grained feature map, and then added element-by-element with the edge fine-grained map to obtain the output as the edge feature map.

3. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: The operations of the feature fusion processing in S1 are specifically as follows: The current edge feature map is processed by ordinary convolution with a convolution kernel of 3×3 and nonlinear processing to obtain the current edge convolution feature map; The current edge convolution feature map is subjected to ordinary convolution with a convolution kernel of 1×1 and nonlinear processing, and then multiplied element by element with the fuzzy feature map of the corresponding scale to obtain the current initial fusion feature map; The initial fusion feature map and the current edge convolution feature map are added element by element to obtain the current edge fusion feature map.

4. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: The operations of the spatial feature selection process in S2 are specifically as follows: The current edge fusion feature map is processed by coordinate attention to obtain a current coordinate attention fusion map; The current coordinate attention fusion map is respectively subjected to a first depth-separable convolution and a second depth-separable convolution to obtain a current first edge channel separation feature map and a current second edge channel separation feature map; The current coordinate attention fusion map is processed by average pooling and maximum pooling respectively, and then superimposed and convolved to obtain a current spatial degradation focus map; The current spatial degenerate focus map is processed by channel dimension feature replication and then element-by-element multiplication with the current first edge channel separation feature map to obtain the current initial spatial selection map; the current initial spatial selection map and the current second edge channel separation feature map are processed by element-by-element addition to obtain the current spatial feature selection map.

5. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: The operation of frequency feature selection processing in S2 is specifically as follows: The current spatial feature selection map is processed by mean filtering to obtain a current low-frequency feature map; a difference map between the current spatial feature selection map and the current low-frequency feature map is obtained to obtain a current complementary high-frequency feature map; the current complementary high-frequency feature map is multiplied element by element with the current spatial feature selection map, and then added element by element with the current spatial feature selection map to obtain a current frequency filtering feature map.

6. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: In S3, the operation of converting the fused feature selection graph and the standard graph from the spatial domain to the frequency domain is specifically as follows: The frequency filtering feature map or standard map is subjected to grayscale processing to obtain a grayscale map; the Fourier transform map of the grayscale map is obtained, and the pixel information in the Fourier transform map whose pixel frequency is less than the frequency threshold is moved to the center of the spectrum to obtain a spectrum amplitude map; the spectrum amplitude map is subjected to low-pass filtering, inverse Fourier transform processing and denoising processing to obtain an inverse Fourier denoising map; the inverse Fourier denoising map is subjected to high-pass filtering and Fourier transform processing to obtain a frequency domain feature map or a frequency domain standard map.

7. The image deblurring method for enhancing barcode and character recognition according to claim 1, characterized in that: In S4, the fusion processing operation is specifically as follows: Arrange several optimized frequency filtering feature graphs of different scales in order from small to large scale to obtain several sorted optimized frequency filtering feature graphs; The previous sorting optimization frequency filtering feature map and the current sorting optimization frequency filtering feature map are subjected to superposition processing and residual block processing to obtain a current clear feature map, which is used to perform superposition processing and residual block processing operations with the next sorting optimization frequency filtering feature map; When the last sorted optimized frequency filtering feature map completes the superposition processing and the residual block processing operations, the deblurred map is obtained.

8. An image deblurring system for enhancing barcode and character recognition, used to implement the image deblurring method for enhancing barcode and character recognition as claimed in claim 1, characterized in that: include: The edge fusion feature map generation module is used to obtain an edge image by edge extraction processing of a blurred image; obtain edge feature maps of different scales of the edge image to form a multi-scale edge feature map set; obtain fuzzy maps of different scales of the blurred image, respectively perform channel feature extraction processing to obtain a plurality of fuzzy feature maps of different scales to form a multi-scale fuzzy map set; perform feature fusion processing on each edge feature map in the multi-scale edge feature map set and a fuzzy feature map of a corresponding scale in the multi-scale fuzzy map set to obtain a plurality of edge fusion feature maps of different scales; A frequency filtering feature map generation module is used to process several edge fusion feature maps of different scales through spatial feature selection to obtain several spatial feature selection maps of different scales; Several spatial feature selection maps of different scales are processed by frequency feature selection to obtain several frequency filtering feature maps of different scales; The optimized frequency filtering feature map generation module is used to obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale on the low-frequency component of the image, the horizontal high-frequency component of the image, the vertical high-frequency component of the image and the diagonal high-frequency component of the image, and obtain the wavelet reconstruction loss value after summing up; obtain the average pixel difference between several frequency filtering feature maps of different scales and the standard map of the corresponding scale, and obtain the spatial information loss value after summing up; the frequency filtering feature map of each scale and the standard map of the corresponding scale are subjected to spatial domain frequency domain conversion processing respectively to obtain the frequency domain feature map and the frequency domain standard map of each scale; obtain the average pixel difference between the frequency domain feature maps of all scales and the frequency domain standard map of the corresponding scale, and obtain the frequency domain information loss value after summing up; when the sum of the wavelet reconstruction loss value, the spatial information loss value and the frequency domain information loss value is less than the loss threshold, output several frequency filtering feature maps of different scales as several optimized frequency filtering feature maps of different scales; The deblurred image generation module fuses several optimized frequency filtering feature images of different scales to obtain a deblurred image.

9. An image deblurring device for enhancing barcode and character recognition, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the image deblurring method for enhancing barcode and character recognition as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the image deblurring method for enhancing barcode and character recognition according to any one of claims 1 to 7 is implemented.