QR two-dimensional code correction and repair method and system

Through Swin transformer model and feature fusion technology, the distortion, defect and low quality of QR code images are solved, efficient correction and repair of QR code images are achieved, and imaging quality and decoding success rate are improved.

CN118940782APending Publication Date: 2024-11-12NARI INFORMATION & COMM TECH

Patent Information

Application Number
CN202410964544.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing QR code correction and repair methods cannot effectively solve the distortion, defect and low quality problems of QR code images. Especially when the QR code imaging quality is insufficient or the printing is poor, the perspective estimation method fails and the displacement estimation method. The and threshold estimation optimization method cannot cope with low-quality problems caused by motion fuzzy.

Method used

Using the QR QR code correction and repair method based on the Swin transformer model, the distortion, defect and QR code image are solved end-to-end by position coding, feature extraction, predicting deformation field and grayscale field, spatial transformation network and feature fusion. Low quality problems, improve imaging quality.

Benefits of technology

It realizes effective correction and repair of distortion, defect and low-quality QR code images, improves the decoding success rate and imaging quality of QR codes, and can handle distortion and defect problems in various scenarios, including cylinder distortion and torsional distortion and low quality problems caused by motion blur.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940782A_ABST
    Figure CN118940782A_ABST
Patent Text Reader

Abstract

The invention discloses a QR two-dimensional code correction and restoration method and system, and the method comprises the steps: carrying out the feature extraction of an image after position coding, and obtaining a low-resolution image feature and a high-resolution image feature; predicting a deformation field and a gray scale field according to the low-resolution image features, and performing embedding operation on the deformation field and the gray scale field to obtain global distortion and gray scale features; performing elastic transformation on the low-resolution image features according to the deformation field to obtain elastic transformation low-resolution image features; carrying out feature fusion on the low-resolution image features, the global distortion and gray features, the elastic transformation low-resolution image features and the high-resolution image features; and carrying out deconvolution on the fused features to obtain a correction deformation field and a correction gray field, and repairing the two-dimensional code image. According to the method, a learning process from the whole to the local has a better effect on correcting distortion with relatively large displacement, and the gray field and the deformation field are taken as two outputs of the model, so that the imaging quality of the two-dimensional code image is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a QR two-dimensional code correction and repair method and system. Background Art

[0002] QR code is an important information carrier and is widely used on commodities, tickets and other items. It provides these items with the ability to provide credit credentials and transmit key information, becoming an indispensable part of the items. However, in the process of printing or shooting, there are often three major problems: one is distortion, including bending, twisting, miscutting and other morphological defects; the second is defects, including broken needles, frame skipping and other defects; the third is low quality, including camera shaking, uneven inkjet and other imaging quality defects. These problems will reduce the quality of the QR code image, making the QR code information unreadable, thus losing the ability to provide credit credentials and transmit key information.

[0003] There are three mainstream methods for QR code correction and restoration at present: the first is the perspective estimation method, which is used to correct the distortion problem of QR code. Generally, the key points in the QR code (such as the positioning module) are first obtained, and then the homography transformation matrix is ​​estimated through the position relationship between the key points. The homography transformation matrix is ​​applied to the QR code image to correct the distortion of the image; the second is the displacement estimation method, which is also used to correct the distortion problem of QR code images. The displacement estimation method generally estimates the position offset of the distorted QR code image pixel by pixel, and corrects the QR code distortion through the position offset; the third is the threshold estimation optimization method, which is used to improve the quality of the QR code image. Due to the uneven illumination and uneven printing ink density encountered by the QR code during the imaging process, the binarization threshold of the QR code is difficult to determine during the binarization process, which will reduce the decoding success rate of the QR code. The threshold estimation optimization method aims to propose a local adaptive threshold estimation method, so that low-quality QR codes can obtain better binarization effects during the binarization operation, thereby improving the decoding success rate of the QR code.

[0004] The existing methods for correcting and repairing QR code images have the following disadvantages: The idea of ​​the perspective estimation method is to estimate the perspective transformation through key points, which can correct the distortion caused by various perspective imaging. However, the perspective estimation method relies on key points, and when the imaging quality of the QR code image is insufficient or the QR code itself is poorly printed, the key points are easily destroyed, making the perspective estimation method invalid. The perspective estimation method cannot cope with the distortion caused by elastic transformations such as cylindrical distortion and torsion distortion, and cannot solve the distortion caused by wrinkles and misalignment. The scope of use is limited. At the same time, the perspective estimation method can only correct the perspective distortion of the QR code image, but cannot solve the problem of defect and low quality of the QR code image. The displacement estimation method can cope with the distortion problem of the QR code image in various scenes by estimating the displacement of each pixel of the QR code image. However, the displacement estimation method also cannot solve the problem of defect and low quality of the QR code image. The threshold estimation optimization method can only solve the problem of inaccurate threshold estimation in the binarization process. It is only applicable to low-quality images caused by uneven lighting and slight blur. It cannot solve the problem of QR code image distortion, defects, and low-quality QR code images caused by motion blur. Summary of the invention

[0005] Purpose of the invention: The present invention aims to provide a feasible and efficient QR two-dimensional code correction and repair method, which can correct the distorted, defective, and low-quality QR two-dimensional code image to be recognized to a recognizable and easy-to-recognize state; another purpose of the present invention is to provide a QR two-dimensional code correction and repair system.

[0006] Technical solution: The QR code correction and repair method of the present invention comprises the following steps:

[0007] (1) Position encoding of the QR code image;

[0008] (2) extracting features from the position-encoded image to obtain low-resolution image features and high-resolution image features;

[0009] (3) According to the low-resolution image features, the deformation field and grayscale field are predicted, and the deformation field and grayscale field are embedded to convert the deformation field and grayscale field into global distortion and grayscale features;

[0010] (4) Using a spatial transformation network, elastically transform the low-resolution image features according to the deformation field to obtain elastically transformed low-resolution image features;

[0011] (5) Upsampling the low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features, and performing feature fusion with the high-resolution image features;

[0012] (6) Deconvolve the fused features to obtain rectified features with the same resolution as the two-dimensional code image in step (1);

[0013] (7) According to the correction features, the correction deformation field and the correction grayscale field are extracted, and the two-dimensional code image in step (1) is corrected and repaired.

[0014] Furthermore, the relative position encoding in the Swin transformer model is used in step (1), and the position encoding information is used to calculate the self-attention similarity matrix of the feature extraction network.

[0015] Furthermore, in step (2), image slice feature extraction is performed based on the Swin transformer model, and the features of the image slices after position encoding are extracted using window multi-head self-attention and sliding window multi-head self-attention to obtain low-resolution image features and high-resolution image features.

[0016] Furthermore, window multi-head self-attention and sliding window multi-head self-attention are used to extract the features of image slices, and the features of image scales 1 / 32, 1 / 16, 1 / 8, and 1 / 4 are obtained, which are denoted as C5, C4, C3, and C2.

[0017] Furthermore, C5 is upsampled and superimposed on C4, and the superimposed features are used as low-resolution image features. The low-resolution image features pass through the output layer to output the deformation field and grayscale field.

[0018] Furthermore, the output layer is a multi-layer perceptron network, which is implemented in the same way as the multi-layer perceptron layer in the transformer model; at this time, the width and height of the output deformation field and grayscale field are both 1 / 16 of the width and height of the QR code image in step (1).

[0019] Furthermore, in step (5), the low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features are stacked, convolved, and upsampled, and then stacked with C3 to obtain C3_merged features; similarly, the C3_merged features are stacked, convolved, and upsampled, and then stacked with C2 to obtain C2_merged features; that is, feature fusion is completed.

[0020] Furthermore, the C2_merged feature is deconvolved twice to obtain a rectified feature with the same resolution as the QR code image resolution in step (1). The rectified feature is operated by a multi-layer perceptron to output a deformation field and a grayscale field respectively.

[0021] Furthermore, the target loss function Loss of the Swin transformer model is

[0022]

[0023] in, represents the distortion loss, represents grayscale loss, represents the predicted normalized distortion correction offset matrix, W represents the actual normalized distortion correction offset matrix, ρ is the weight adjustment matrix, represents the predicted grayscale correction offset matrix, and G represents the actual grayscale correction offset matrix;

[0024] When calculating the overall distortion and grayscale loss, ρ is 0.1; when calculating the local distortion and grayscale loss, ρ is set to 1.

[0025] The QR code correction and repair system of the present invention comprises:

[0026] A position encoding module, used for performing position encoding on the two-dimensional code image;

[0027] Feature extraction module, used to extract image features of different resolutions;

[0028] A prediction module, used to predict the deformation field and grayscale field according to the low-resolution image features;

[0029] A spatial transformation network module is used to elastically transform the low-resolution image features according to the deformation field to obtain elastically transformed low-resolution image features;

[0030] A feature fusion module is used to upsample low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features, and perform feature fusion with high-resolution image features;

[0031] The correction module is used to deconvolve the fused features to obtain the correction deformation field and the correction grayscale field, and perform correction and repair on the QR code image.

[0032] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: 1. The present invention predicts a deformation field and a grayscale field based on low-resolution image features. The deformation field is used to correct the distortion of the two-dimensional code image, while the grayscale field is used to repair the defects of the two-dimensional code image and improve the quality of the two-dimensional code image; 2. The present invention adopts a learning process from the whole to the part, which has a better effect on correcting the distortion with large displacement. At the same time, the method learns the grayscale field and the deformation field together as two outputs of the learning model, and solves the distortion, defect and low-quality problems of the two-dimensional code image end-to-end, thereby improving the imaging quality of the two-dimensional code image; 3. The present invention uses both the low-resolution features and the high-resolution features of the image to estimate the deformation field and the grayscale field of the two-dimensional code image, wherein the low-resolution features are used to estimate the overall distortion and grayscale offset of the two-dimensional code image, while the high-resolution features are used to estimate the local distortion and grayscale offset of the two-dimensional code image. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings.

[0035] The QR code correction and repair method of the present invention comprises the following steps:

[0036] (1) Position-encode the QR code image.

[0037] The position encoding method is related to the feature extraction method of the feature extraction network. Position encoding is used to introduce position features into the model to improve the model's perception and understanding of position information and position relationships. The present invention adopts a solution of first slicing and then extracting features. The position encoding refers to the relative position encoding of the Swin transformer model. The position encoding information acts on the calculation of the self-attention similarity matrix of the feature extraction network. The calculation method of self-attention is:

[0038]

[0039] Among them, B represents the relative position deviation, that is, the position encoding information calculated by the position encoding module, Q, K, and V represent query, key, and value, respectively, and they are all image features, d is the normalization factor, which is a hyperparameter constant, and SoftMax is the softmax function.

[0040] (2) Perform feature extraction on the position-encoded image to obtain low-resolution image features and high-resolution image features.

[0041] The feature extraction is implemented using the Swin transformer model, which uses image slice feature extraction technology. For the slices of the position-encoded QR code image, the window multi-head self-attention and sliding window multi-head self-attention are used to extract the features of the image slices, and the feature connections between the image slices are established, which can achieve high computational efficiency while ensuring high feature extraction capabilities. The feature extraction network outputs the features of the input QR code image scales of 1 / 32, 1 / 16, 1 / 8, and 1 / 4, which are denoted as C5, C4, C3, and C2.

[0042] (3) According to the low-resolution image features, the deformation field and grayscale field are predicted, and the deformation field and grayscale field are embedded to convert the deformation field and grayscale field into global distortion and grayscale features.

[0043] The C5 and C4 features are selected as the features for predicting global distortion and grayscale. Since the resolution of the C5 feature is different from that of the C4 feature, it is necessary to increase the resolution of the C5 feature through upsampling operation, and then superimpose it on the C4 feature. The superimposed feature is recorded as the C4_merged feature. The C4_merged feature passes through the output layer and outputs the deformation field and grayscale field. The output layer is a multi-layer perceptron network, which is implemented in the same way as the multi-layer perceptron layer in the transformer model, namely the MLP layer. At this time, the scale of the deformation field and grayscale field output by the model is much smaller than that of the input QR code image, and the width and height are both 1 / 16 of the width and height of the input QR code image.

[0044] (4) Using a spatial transformation network, the low-resolution image features are elastically transformed according to the deformation field to obtain elastically transformed low-resolution image features.

[0045] The spatial transformer network is a differentiable elastic transformation operation used to transform the C4_merged features according to the deformation field to predict local distortion and grayscale difference.

[0046] (5) The low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features are upsampled and fused with the high-resolution image features.

[0047] The fused features include low-resolution image features, global distortion and grayscale features, elastic transformation low-resolution image features, and high-resolution image features. Among them, the embedded features of the deformation field and grayscale field predicted by the low-resolution image features, namely the global distortion and grayscale features, are realized through the embedding layer Embedding, that is, by embedding the deformation field and grayscale field predicted by the low-resolution image features, respectively. The channel dimension of the embedded features obtained after the embedding operation is the same as the channel dimension of the C4 feature. The embedded features and the C4_merged features before and after the spatial transformation are subjected to the Unsample(Conv(Concat())) operation, namely stacking + convolution + upsampling operation, and then stacked with the C3 features to obtain the C3_merged features; similarly, the C3_merged features are subjected to the Unsample(Conv(Concat())) operation, and then stacked with the C2 features to obtain the C2_merged features. In this way, the fusion of low-resolution image features and high-resolution image features is completed.

[0048] (6) Deconvolve the fused features to obtain rectified features with the same resolution as the two-dimensional code image in step (1).

[0049] Local distortion and grayscale are predicted by high-resolution image features, because only high-resolution image features can retain enough details. After two deconvolutions, the C2_merged feature can obtain a correction feature with the same resolution as the input QR code image. The C2_merged feature integrates low-resolution image features, overall distortion and grayscale features, and high-resolution image features.

[0050] (7) According to the correction feature, the correction deformation field and the correction gray field are extracted, and the two-dimensional code image in step (1) is corrected and repaired. This correction feature is then operated by the multi-layer perceptron, i.e., the MLP layer, and the correction deformation field and the correction gray field are output respectively. At this time, the deformation field and the gray field can be directly applied to the input image to correct the distortion, defect and low quality of the image.

[0051] The output of the model includes deformation field and grayscale field, and the target loss function of the model is:

[0052]

[0053] in, represents the distortion loss, represents grayscale loss, represents the predicted normalized distortion correction offset matrix, W represents the real (or labeled) normalized distortion correction offset matrix, ρ is the weight adjustment matrix, represents the predicted grayscale correction offset matrix, G represents the actual grayscale correction offset matrix, and ||*|| represents the L1 loss.

[0054] When calculating the overall distortion and grayscale loss, ρ is set to 0.1; when calculating the local distortion and grayscale loss, ρ is set to 1. The total loss is the sum of the overall distortion, grayscale loss and the local distortion, grayscale loss.

[0055] Model training. The samples for training the model are artificially created sample pairs. A pair of samples includes the same standard QR code image, namely the reference QR code image, and a defective QR code image. Both images are collected from the same QR code. The resolution of all samples is 256*256. The center of the reference QR code image coincides with the center of the QR code, and the blank area outside the reference QR code is the same. Before training, the distortion and grayscale offset of all sample pairs are known. During the training process, the overall distortion and grayscale reduce the resolution according to the feature downsampling method as the loss of the overall deformation field and grayscale field.

[0056] Inference prediction: During inference, a 256*256 QR code image is input, and the local deformation field and grayscale field are output. The distortion is corrected by the deformation field, and then the grayscale field is superimposed to repair the image quality. This completes the correction and repair of the input QR code image.

Claims

1. A QR code correction and repair method, characterized in that: The following steps are involved: (1) Position encoding of the QR code image; (2) extracting features from the position-encoded image to obtain low-resolution image features and high-resolution image features; (3) According to the low-resolution image features, the deformation field and grayscale field are predicted, and the deformation field and grayscale field are embedded to convert the deformation field and grayscale field into global distortion and grayscale features; (4) Using a spatial transformation network, elastically transform the low-resolution image features according to the deformation field to obtain elastically transformed low-resolution image features; (5) Upsampling the low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features, and performing feature fusion with the high-resolution image features; (6) Deconvolve the fused features to obtain rectified features with the same resolution as the two-dimensional code image in step (1); (7) According to the correction features, the correction deformation field and the correction grayscale field are extracted, and the two-dimensional code image in step (1) is corrected and repaired.

2. The QR code correction and repair method according to claim 1, characterized in that: In step (1), the relative position encoding in the Swintransformer model is used, and the position encoding information is used to calculate the self-attention similarity matrix of the feature extraction network.

3. The QR code correction and repair method according to claim 2, characterized in that: In step (2), image segmentation feature extraction is adopted based on the Swintransformer model, and the features of the image segmentation after position encoding are extracted by using window multi-head self-attention and sliding window multi-head self-attention to obtain low-resolution image features and high-resolution image features.

4. The QR code correction and repair method according to claim 3, characterized in that: Window multi-head self-attention and sliding window multi-head self-attention are used to extract the features of image slices, and the features of image scales 1 / 32, 1 / 16, 1 / 8, and 1 / 4 are obtained, which are denoted as C5, C4, C3, and C2.

5. The QR code correction and repair method according to claim 4, characterized in that: After upsampling, C5 is superimposed on C4. The superimposed features are used as low-resolution image features. The low-resolution image features pass through the output layer to output the deformation field and grayscale field.

6. The QR code correction and repair method according to claim 5, characterized in that: The output layer is a multi-layer perceptron network, which is implemented in the same way as the multi-layer perceptron layer in the transformer model. At this time, the width and height of the output deformation field and grayscale field are both 1 / 16 of the width and height of the QR code image in step (1).

7. The QR code correction and repair method according to claim 6, characterized in that: In step (5), the low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features are stacked, convolved, and upsampled, and then stacked with C3 to obtain C3_merged features; Similarly, the C3_merged feature is stacked, convolved, and upsampled, and then stacked with C2 to obtain the C2_merged feature, thus completing feature fusion.

8. According to the QR code correction and repair method according to claim 7, it is characterized in that: After two deconvolutions, the C2_merged feature can obtain a rectified feature with the same resolution as the QR code image resolution in step (1). The rectified feature is operated by a multi-layer perceptron to output a deformation field and a grayscale field respectively.

9. According to the QR code correction and repair method according to claim 8, it is characterized in that: The objective loss function Loss of the Swin transformer model is in, represents the distortion loss, represents grayscale loss, represents the predicted normalized distortion correction offset matrix, W represents the actual normalized distortion correction offset matrix, ρ is the weight adjustment matrix, represents the predicted grayscale correction offset matrix, and G represents the actual grayscale correction offset matrix; When calculating the overall distortion and grayscale loss, ρ is 0.1; when calculating the local distortion and grayscale loss, ρ is set to 1.

10. A QR code correction and repair system, characterized in that: include: A position encoding module, used for performing position encoding on the two-dimensional code image; Feature extraction module, used to extract image features of different resolutions; A prediction module, used to predict the deformation field and grayscale field according to the low-resolution image features; A spatial transformation network module is used to elastically transform the low-resolution image features according to the deformation field to obtain elastically transformed low-resolution image features; A feature fusion module is used to upsample low-resolution image features, global distortion and grayscale features, and elastic transformation low-resolution image features, and perform feature fusion with high-resolution image features; The correction module is used to deconvolve the fused features to obtain the correction deformation field and the correction grayscale field, and perform correction and repair on the QR code image.

Citation Information

Patent Citations

  • Multi-source fused aerial view sensing target detection method, device and equipment and medium

    CN117315424A

  • Generative system and method for enhancing readability of barcodes using frequency guided computer vision

    US11875259B1

  • System, method, device, and computer program product for providing image correction

    US20080144964A1

  • License plate recognition method and apparatus, storage medium and terminal

    WO2022111355A1

Cited By

  • Image correction method, homework correction method, related device, equipment and medium

    CN120976077A