A deep learning-based blind restoration method and system for damaged QR code images

Abstract

The application relates to the technical field of image restoration, and discloses a damaged QR code image blind restoration method and system based on deep learning, which comprises the following steps: S1, collecting a dataset of real light pollution two-dimensional codes; S2, obtaining a first-stage processing result of image balancing; S3, learning and reasoning semantic labels in the dataset, and according to the semantic labels in the dataset, image regions with similar features in high dimensions are merged to obtain a second-stage processing result of image balancing; and S4, extracting a coding area from the second-stage processing result of image balancing, fusing and supplementing the coding area with the first-stage processing result of image balancing, and completing blind restoration of the damaged QR code image facing light pollution. The application solves the problem of low picture recognition rate of the prior art facing light pollution, and has the characteristics of fast positioning of a pollution area and targeted restoration.

Description

Technical Field

[0001] This invention relates to the field of image restoration technology, and more specifically, to a method and system for blind restoration of damaged QR code images based on deep learning. Background Technology

[0002] QR codes (Quick Response Codes) are a low-cost, highly secure electronic tag technology that can store large amounts of information in a limited space and has been widely used in IoT scenarios such as mobile payments, express delivery, and warehouse management. However, due to the diverse application scenarios of QR codes, they are prone to being obstructed by foreign objects, stained, or damaged by human intervention, making them difficult to recognize. Traditional methods are no longer sufficient to repair these complex and varied damaged QR codes to restore them to a readable state. Therefore, improving current QR code repair methods to increase the recognition rate of repaired QR codes is a pressing technical problem that needs to be solved.

[0003] Existing technology offers a binarization processing method for QR code images with localized uneven illumination. This QR code image restoration method based on image binarization is an effective solution for damage caused by common "spotted" localized uneven illumination in QR code images. This method can handle the "spotted" localized uneven illumination problem in QR code images, preserving the integrity of QR code images with "spotted" localized uneven illumination after binarization and restoring them to a recognizable state, thereby improving the recognition rate and accuracy of QR code images. Furthermore, this method has good adaptability and scalability, capable of handling complex QR code image cases with uneven illumination and restoring images with poor quality. These characteristics make this method universally applicable in practical applications. However, it should be noted that the restoration capability of this method still has certain limitations. For example, in special cases where the QR code image has interference-type dirt or occlusion, the restoration effect of this method may be affected, leading to a decrease in the recognition rate.

[0004] Existing technology also offers a fast adaptive binarization method for QR codes with uneven illumination. This method combines local thresholding with integral images for binarization of each pixel, making it an effective fast adaptive binarization method for QR code images. This method uses position detection graphics to dynamically define the window size for local threshold binarization, effectively eliminating the effects of uneven illumination and avoiding module information distortion. This ensures the integrity of the light and dark modules after binarization of the QR code image. Furthermore, combining integral images accelerates threshold calculation, improving the binarization speed of QR code images with uneven illumination.

[0005] However, existing technologies still suffer from low recognition rates for images damaged by uneven lighting. How to invent a blind repair method for damaged QR code images based on deep learning is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the problem of low recognition rate of images damaged by uneven lighting in existing technologies, this invention provides a blind repair method and system for damaged QR code images based on deep learning, which has the ability to quickly locate the damaged area and restore it in a targeted manner.

[0007] To achieve the above-mentioned objectives of this invention, the technical solution adopted is as follows: A blind restoration method for damaged QR code images based on deep learning includes the following steps: S1. Collect a dataset of real light-damaged QR codes; S2. Perform image mean filtering on the dataset, and then perform dynamic bottom-hat transformation on the mean-filtered dataset; perform histogram equalization on the dynamic bottom-hat transformation result to obtain the first stage processing result of image equalization. S3. Based on the first-stage processing result of image equalization, perform unsupervised image segmentation on the dataset, assign the same semantic label to image regions in the dataset that have the same semantic information; learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage processing result of image equalization. S4. Extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

[0008] Preferably, in step S2, image mean filtering is performed on the dataset, specifically as follows:

[0009] in, Indicates the center point is at At this location, M represents a size of The filter window, and , Represents the original image. This represents the image obtained after mean filtering.

[0010] Furthermore, in step S2, a dynamic bottom-hat transformation is performed on the mean-filtered dataset, specifically as follows: S201. Perform histogram equalization on black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven lighting conditions. S202. The mode of the dataset after histogram equalization and mean filtering is collected to determine the size of the structuring element of the dynamic cap transformation, and then the dynamic cap transformation is performed on the damaged QR code image.

[0011] Furthermore, the dynamic cap transformation formula is as follows:

[0012] in, I The image in the dataset after histogram equalization and mean filtering ， Indicates through structural elements b right I Perform morphological closing operations. b The definition of is:

[0013] Here, win is the kernel size that needs to be set to perform the morphological closing operation.

[0014] Furthermore, in step S201, searching for black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven lighting conditions specifically involves: extracting the edges of the damaged image using the Canny algorithm, and then filtering by the distance ratio between 6 adjacent pixels in the same row or column to obtain black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven lighting conditions.

[0015] Furthermore, in step S202, the mode of the histogram-equalized mean-filtered dataset is collected to determine the size of the structuring element of the dynamic bottom-hat transformation. Specifically, the size of all candidate wins is set; the size of the candidate wins is taken down to a multiple of 7, and then the mode of the histogram-equalized mean-filtered dataset is collected to determine the size of the structuring element of the dynamic bottom-hat transformation.

[0016] Furthermore, in step S4, the formula for extracting the coded region from the second-stage image equalization processing result and performing detail fusion supplementation between the coded region and the first-stage image equalization processing result is as follows:

[0017] in, The first stage processing result of image equalization This is the result of the second stage of image equalization. and These are the weights used in image fusion. That is the offset.

[0018] Furthermore, the dataset also includes simulated light-damaged QR codes; in step S2, the segmentation network is trained based on backpropagation using the simulated light-damaged QR codes and real light-damaged QR codes; the trained segmentation network is used to perform unsupervised image segmentation of the dataset based on machine learning.

[0019] Furthermore, in step S3, a machine learning-based segmentation network and a deep learning-based CNN neural network with an autoencoder structure are introduced. Based on the first-stage image equalization processing results, the segmentation network performs unsupervised image segmentation on the dataset. The CNN neural network learns and infers semantic labels in the dataset. The machine learning-based image segmentation method provides the labels needed for backpropagation loss calculation for the deep learning-based image segmentation method. In the unsupervised image segmentation task, the neural network used by the deep learning-based image segmentation method learns and infers the fine-grained image segmentation results of the machine learning-based image segmentation algorithm. Based on the segmentation results, during iterative training, it merges blocks with the same semantic information in the damaged image to obtain the expected semantic segmentation result.

[0020] A blind restoration system for damaged QR code images based on deep learning includes a data collection module, a first-stage image equalization processing module, a second-stage image equalization processing module, and a blind restoration module. The data collection module is used to collect datasets of real light-damaged QR codes; The first-stage processing module for image equalization is used to perform image mean filtering on the dataset and dynamic bottom-hat transformation on the mean-filtered dataset; histogram equalization is performed on the dynamic bottom-hat transformation result to obtain the first-stage processing result of image equalization. The second-stage image equalization processing module is used to perform unsupervised image segmentation on the dataset based on the first-stage image equalization processing results, assign the same semantic label to image regions in the dataset that have the same semantic information, learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage image equalization processing results. The blind repair module is used to extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

[0021] The beneficial effects of this invention are as follows: This invention discloses a blind restoration method for damaged QR code images based on deep learning. Addressing the problem that QR code images easily become difficult or even unrecognizable after being damaged by uneven lighting or other forms of contamination, a two-stage image equalization algorithm is proposed. In the first-stage image equalization algorithm, a dynamic window-based bottom-hat transformation and histogram equalization method are used to significantly reduce the impact of contamination on the original image information. In the second-stage image equalization algorithm, an unsupervised image segmentation method is used to classify the original image information remaining within the contamination. Then, the encoded region from the second-stage image equalization algorithm is fused with the result of the first-stage image equalization process to supplement details, thereby effectively restoring the valid decoded data of the contaminated QR code image. The method proposed in this invention solves the problem of low recognition rate for images with uneven lighting and contamination in existing technologies, and has the characteristics of quickly locating contaminated areas and providing targeted restoration. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a blind repair method for damaged QR code images based on deep learning, according to the present invention.

[0023] Figure 2 This is a schematic diagram of the deep learning-based image segmentation network structure in Example 2. Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0025] Example 1 like Figure 1 As shown, a blind restoration method for damaged QR code images based on deep learning includes the following steps: S1. Collect a dataset of real light-damaged QR codes; S2. Perform image mean filtering on the dataset, and then perform dynamic bottom-hat transformation on the mean-filtered dataset; perform histogram equalization on the dynamic bottom-hat transformation result to obtain the first stage processing result of image equalization. S3. Based on the first-stage processing result of image equalization, perform unsupervised image segmentation on the dataset, assign the same semantic label to image regions in the dataset that have the same semantic information; learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage processing result of image equalization. S4. Extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

[0026] Example 2 A blind restoration method for damaged QR code images based on deep learning includes the following steps: S1. Collect a dataset of real light-damaged QR codes; S2. Perform image mean filtering on the dataset, and then perform dynamic bottom-hat transformation on the mean-filtered dataset; perform histogram equalization on the dynamic bottom-hat transformation result to obtain the first stage processing result of image equalization. S3. Based on the first-stage processing result of image equalization, perform unsupervised image segmentation on the dataset, assign the same semantic label to image regions in the dataset that have the same semantic information; learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage processing result of image equalization. S4. Extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

[0027] In this embodiment, the dataset is a real dataset of light-damaged QR codes, consisting of 50 images of damaged QR codes under strong light and 30 images of damaged QR codes under weak light. Furthermore, to verify the effectiveness of the second-order image enhancement algorithm of this invention, a dataset of digitally damaged QR codes affected by indoor lighting, Reflective QR, was also created to meet the needs of the experiment. During the creation of the Reflective QR dataset, images were acquired using the camera system of an iPhone 11, and each acquired image was uniformly cropped and resized using the image editing tool Adobe Photoshop.

[0028] In one specific embodiment, step S2, which involves performing image mean filtering on the dataset, specifically includes:

[0029] in, Indicates the center point is at At this location, M represents a size of The filter window, and , Represents the original image. This represents the image obtained after mean filtering.

[0030] In one specific embodiment, step S2 involves performing a dynamic bottom-hat transformation on the mean-filtered dataset, specifically as follows: S201. Perform histogram equalization on black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven lighting conditions. S202. The mode of the dataset after histogram equalization and mean filtering is collected to determine the size of the structuring element of the dynamic cap transformation, and then the dynamic cap transformation is performed on the damaged QR code image.

[0031] In this embodiment, the dynamic bottom-hat transformation operation on the damaged image is to further restore the original contrast, brightness, and color of the damaged image on the basis of image denoising.

[0032] In one specific embodiment, the dynamic bottom cap transformation formula is:

[0033] in, I The image in the dataset after histogram equalization and mean filtering ， Indicates through structural elements b right I Perform morphological closing operations. b The definition of is:

[0034] Here, win is the kernel size that needs to be set to perform the morphological closing operation.

[0035] In this embodiment, in order to achieve dynamic cap transformation and enable QR code images damaged by uneven illumination under different conditions to undergo cap transformation operation through a suitable kernel, it is necessary to dynamically adjust the kernel using the finder image features of the QR code image itself, that is, win is equal to the size of the finder image.

[0036] In one specific embodiment, in step S201, searching for black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven illumination conditions specifically involves: extracting the edges of the damaged image using the Canny algorithm, and then filtering by the distance ratio between 6 adjacent pixels in the same row or column to obtain black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven illumination conditions.

[0037] In one specific embodiment, step S202 involves performing mode sampling on the histogram-equalized mean-filtered dataset to determine the size of the structuring element of the dynamic bottom-hat transformation. Specifically, this involves setting the size of all candidate wins; taking the size of the candidate wins down to a multiple of 7; and then performing mode sampling on the histogram-equalized mean-filtered dataset to determine the size of the structuring element of the dynamic bottom-hat transformation.

[0038] In one specific embodiment, the formula for extracting the coded region from the second-stage image equalization processing result and performing detail fusion supplementation between the coded region and the first-stage image equalization processing result in step S4 is as follows:

[0039] in, The first stage processing result of image equalization This is the result of the second stage of image equalization. and These are the weights used in image fusion. That is the offset.

[0040] In one specific embodiment, the dataset also includes simulated light-damaged QR codes; in step S2, the segmentation network is trained based on backpropagation according to the simulated light-damaged QR codes and real light-damaged QR codes; the trained segmentation network is used to perform unsupervised image segmentation of the dataset based on machine learning.

[0041] In this embodiment, the deep learning-based image segmentation network structure is as follows: Figure 2 As shown.

[0042] In this embodiment, the histogram equalization can be global histogram equalization or a local histogram method that limits contrast.

[0043] In one specific embodiment, step S3 introduces a machine learning-based segmentation network and a deep learning-based CNN neural network with an autoencoder structure. The machine learning-based segmentation network and the deep learning-based CNN with an autoencoder structure perform unsupervised image segmentation on the dataset based on the first-stage image equalization processing results. The CNN neural network learns and infers semantic labels in the dataset. The machine learning-based image segmentation method provides the labels needed for backpropagation loss calculation for the deep learning-based image segmentation method. In the unsupervised image segmentation task, the neural network used by the deep learning-based image segmentation method learns and infers the fine-grained image segmentation results of the machine learning-based image segmentation algorithm. Based on the segmentation results, during iterative training, it merges blocks with the same semantic information in the damaged image to obtain the expected semantic segmentation result. Example 3 In this embodiment, as shown in Table 1, the blind restoration method for damaged QR code images based on deep learning disclosed in this invention can output higher PSNR and SSIM results more stably compared with existing image restoration techniques.

[0044] Table 1 Comparison of Repair and Recognition Rates of Various Thresholding Algorithms

[0045] in: The images of type (1) demonstrate the image processing effects of various algorithms on light-damaged QR code images with localized low illumination. The images of type (2) demonstrate the image processing effects of each algorithm on light-damaged QR code images with weak local illumination and color difference interference.

[0046] The images of type (3) demonstrate the image processing effects of each algorithm on light-damaged QR code images with strong local illumination.

[0047] The images of type (4) demonstrate the image effects of each algorithm on light-damaged QR code images where the local lighting is strong and the rest of the image has shallow shadows.

[0048] The images of type (5) demonstrate the image processing effects of each algorithm on light-damaged QR code images where the local lighting is strong and the rest of the image has deep shadows.

[0049] The blind restoration method for damaged QR code images based on deep learning introduced in this invention can effectively restore the contrast, brightness, and color difference of light-damaged QR code images while preserving image details. In contrast, Yao's algorithm, Di's algorithm, and Chen's algorithm suffer from image detail loss after image thresholding. To further quantify the processing effects of each comparison algorithm on the above five types of light-damaged QR code images and to conduct a more in-depth comparison with Li's algorithm, it is necessary to perform PSNR and SSIM detection on the above images, and to create corresponding PSNR and SSIM tables and plot corresponding PSNR and SSIM charts from the detection results.

[0050] In this embodiment, as shown in Table 2, when processing QR code images damaged by uneven illumination, the second-order image enhancement method proposed in this invention has better contrast, brightness, and color restoration capabilities than other binarization methods. This is the key to improving the recognition rate of QR code images damaged by uneven illumination after restoration. The first-order image equalization method can accurately restore the entire damaged image even when there are significant differences in illumination across different regions, while the second-order image equalization method can further adjust and improve the continuity of contrast, brightness, and color across different regions of the image based on the first-order method.

[0051] Table 2 Comparison of Repair and Recognition Rates of Various Thresholding Algorithms and the Algorithm in This Chapter

[0052] Example 3 A blind restoration system for damaged QR code images based on deep learning includes a data collection module, a first-stage image equalization processing module, a second-stage image equalization processing module, and a blind restoration module. The data collection module is used to collect datasets of real light-damaged QR codes; The first-stage processing module for image equalization is used to perform image mean filtering on the dataset and dynamic bottom-hat transformation on the mean-filtered dataset; histogram equalization is performed on the dynamic bottom-hat transformation result to obtain the first-stage processing result of image equalization. The second-stage image equalization processing module is used to perform unsupervised image segmentation on the dataset based on the first-stage image equalization processing results, assign the same semantic label to image regions in the dataset that have the same semantic information, learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage image equalization processing results. The blind repair module is used to extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

[0053] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A blind restoration method for damaged QR code images based on deep learning, characterized in that: Includes the following steps: S1. Collect a dataset of real light-damaged QR codes; S2. Perform image mean filtering on the dataset, and then perform dynamic bottom-hat transformation on the mean-filtered dataset. The first stage of image equalization is obtained by performing histogram equalization on the dynamic cap transformation result. S3. Based on the first-stage processing result of image equalization, perform unsupervised image segmentation on the dataset, assign the same semantic label to image regions in the dataset that have the same semantic information; learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage processing result of image equalization. S4. Extract the coding region from the second stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt. In step S2, a dynamic bottom-hat transformation is performed on the mean-filtered dataset, specifically as follows: S201. Perform histogram equalization on black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven lighting conditions. S202. The mode of the dataset after histogram equalization and mean filtering is collected to determine the size of the structuring element of the dynamic cap transformation, and then the dynamic cap transformation is performed on the damaged QR code image. The formula for dynamic cap transformation is: in, I The image in the dataset after histogram equalization and mean filtering ， Indicates through structural elements b right I Perform morphological closing operations. b The definition of is: Here, win is the kernel size that needs to be set to perform the morphological closing operation.

2. The method for blind restoration of damaged QR code images based on deep learning according to claim 1, characterized in that: In step S2, image mean filtering is performed on the dataset, specifically as follows: in, Indicates the center point is at At this location, M represents a size of The filter window, and , Represents the original image. This represents the image obtained after mean filtering.

3. The method for blind restoration of damaged QR code images based on deep learning according to claim 1, characterized in that: In step S201, searching for black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven illumination conditions specifically involves: extracting the edges of the damaged image using the Canny algorithm, and then filtering by the distance ratio between 6 adjacent pixels in the same row or column to obtain black and white pixel blocks with a length ratio of 1:1:3:1:1 in the mean-filtered dataset under uneven illumination conditions.

4. The method for blindly repairing damaged QR code images based on deep learning according to claim 3, characterized in that: In step S202, the mode of the histogram-equalized mean-filtered dataset is collected to determine the size of the structuring element of the dynamic bottom-hat transformation. Specifically, the size of all candidate wins is set; the size of the candidate wins is taken down to a multiple of 7, and then the mode of the histogram-equalized mean-filtered dataset is collected to determine the size of the structuring element of the dynamic bottom-hat transformation.

5. The method for blind restoration of damaged QR code images based on deep learning according to claim 1, characterized in that: In step S4, the formula for extracting the coding region from the second-stage image equalization processing result and performing detail fusion supplementation between the coding region and the first-stage image equalization processing result is as follows: in, The first stage processing result of image equalization This is the result of the second stage of image equalization. and These are the weights used in image fusion. That is the offset.

6. The method for blind restoration of damaged QR code images based on deep learning according to claim 1, characterized in that: The dataset also includes simulated light-damaged QR codes; in step S2, the segmentation network is trained based on backpropagation according to the simulated light-damaged QR codes and real light-damaged QR codes; the trained segmentation network is used to perform unsupervised image segmentation of the dataset based on machine learning.

7. The method for blind restoration of damaged QR code images based on deep learning according to claim 6, characterized in that: In step S3, a machine learning-based segmentation network and a deep learning-based CNN with an autoencoder structure are introduced. Based on the first-stage image equalization processing results, the segmentation network performs unsupervised image segmentation on the dataset. The CNN neural network learns and infers semantic labels in the dataset. The machine learning-based image segmentation method provides the labels needed for backpropagation loss calculation for the deep learning-based image segmentation method. In the unsupervised image segmentation task, the neural network used by the deep learning-based image segmentation method learns and infers the fine-grained image segmentation results of the machine learning-based image segmentation algorithm. Based on the segmentation results, during iterative training, it merges blocks with the same semantic information in the damaged image to obtain the expected semantic segmentation result.

8. A deep learning-based blind restoration system for damaged QR code images, used to perform the deep learning-based blind restoration system for damaged QR code images as described in any one of claims 1 to 7, characterized in that: It includes a data collection module, a first-stage image equalization processing module, a second-stage image equalization processing module, and a blind repair module; The data collection module is used to collect datasets of real light-damaged QR codes; The first-stage processing module for image equalization is used to perform image mean filtering on the dataset and dynamic bottom-hat transformation on the mean-filtered dataset. The first stage of image equalization is obtained by performing histogram equalization on the dynamic cap transformation result. The second-stage image equalization processing module is used to perform unsupervised image segmentation on the dataset based on the first-stage image equalization processing results, assign the same semantic label to image regions in the dataset that have the same semantic information, learn and infer the semantic labels in the dataset, and merge image regions with similar features in high dimension based on the semantic labels in the dataset to obtain the second-stage image equalization processing results. The blind repair module is used to extract the coding region from the second-stage processing result of image equalization, and perform detail fusion and supplementation between the coding region and the first-stage processing result of image equalization to complete the blind repair of damaged QR code images with uneven illumination and dirt.

Images