Two-dimensional code image restoration method based on deep learning

Abstract

The invention relates to the technical field of image processing, and discloses a two-dimensional code image restoration method based on deep learning, and the method comprises the steps: obtaining an original bar code image, carrying out the multi-angle transformation and cutting operation of the original bar code image, and generating an image sample containing part of bar code information loss; arranging the enhanced image samples into a training data set, marking each sample in the training data set, and mining image features of the samples in the training data set by adopting an image feature extraction algorithm; the two-dimensional code image restoration model is trained, the AIGC is integrated into the two-dimensional code image restoration model, and the two-dimensional code image restoration model automatically generates a restoration image according to the input damaged image through the generative capability of the AIGC; obtaining a restored image output by the two-dimensional code image restoration model, performing post-processing on the restored image, and performing positioning, code word extraction and decoding on the post-processed and optimized image to obtain a restored image; according to the invention, the recognizable rate of the repaired two-dimensional code is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a two-dimensional code image repair method based on deep learning. Background Art

[0002] In today's era of rapid digital information flow, QR codes, as a convenient medium for information storage and transmission, are widely used in various fields, such as product traceability, mobile payments, and identity verification. However, in actual use, QR code images are often affected by various factors, resulting in problems such as staining, fading, and incompleteness. These problems make it difficult for scanners to accurately recognize and decode the code, seriously affecting the efficiency of information transmission and interaction. Traditional image processing methods have limited effectiveness in repairing QR codes in complex and damaged conditions, and are unable to meet practical needs. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems and design a two-dimensional code image repair method based on deep learning.

[0004] The present invention provides a deep learning-based two-dimensional code image restoration method, which obtains original barcode images of various sizes and contents, and uses image enhancement technology to perform multi-angle transformation and cropping operations on the original barcode images to generate image samples containing some missing barcode information;

[0005] The enhanced image samples are organized into a training data set, and each sample in the training data set is labeled. An image feature extraction algorithm is used to mine the image features of the samples in the training data set, where the image features include at least texture features, geometric features, and frequency domain features.

[0006] The QR code image restoration model is trained using a training dataset that has undergone image feature extraction. AIGC is integrated into the model. The model automatically generates a restored image based on the input damaged image through the generative capabilities of AIGC. The QR code image restoration model uses a multi-layer neural network, including a convolutional neural network for feature extraction, a recurrent neural network for capturing barcode image sequence information, and a generative adversarial network for generating high-quality restored images.

[0007] The repaired image output by the QR code image repair model is obtained, the repaired image is post-processed, and positioning, codeword extraction and decoding are performed on the image after post-processing optimization to obtain the repaired image.

[0008] Optionally, in the first implementation of the present invention, obtaining original barcode images of various sizes and contents, performing multi-angle transformation and cropping operations on the original barcode images using image enhancement technology to generate image samples containing partially missing barcode information includes:

[0009] Obtain original barcode images of various sizes and contents, and rotate each original barcode image according to a preset rotation angle of 5°-15°;

[0010] Flip the rotated original barcode image horizontally or vertically to make the image symmetrical or upside down;

[0011] The original barcode image that has undergone multi-angle transformation is cropped from its four edges, with 10% to 20% of the pixels being cropped each time, thereby generating image samples containing varying degrees of missing barcode information.

[0012] Optionally, in a second implementation of the present invention, organizing the enhanced image samples into a training data set, annotating each sample in the training data set, and using an image feature extraction algorithm to mine image features of the samples in the training data set includes:

[0013] The enhanced image samples are organized into a training data set, and key information is annotated for each sample in the training data set, where the key information annotation includes at least the barcode type annotation, the original complete information annotation, and the damage description annotation;

[0014] The color images in the training data set are converted into grayscale images, and the texture features of the grayscale images are extracted using the gray-level co-occurrence matrix;

[0015] Obtain a first edge image corresponding to the grayscale image using the Canny edge detection algorithm, determine the barcode's bounding box from the first edge image, and obtain the barcode's relative position in the entire grayscale image and its offset ratio relative to the image boundary;

[0016] Fast Fourier transform is used to convert samples in the training data set from the spatial domain to the frequency domain to obtain frequency domain features.

[0017] Optionally, in a third implementation of the present invention, obtaining a first edge image corresponding to the grayscale image using a Canny edge detection algorithm, determining a barcode bounding box from the first edge image, and obtaining a relative position of the barcode in the entire grayscale image and an offset ratio relative to the image boundary include:

[0018] The noise level of each pixel in the grayscale image is estimated using a local window, an adaptive Gaussian smoothing kernel is generated based on the local noise estimation value of each pixel, and the adaptive Gaussian kernel is applied to the grayscale image pixel by pixel;

[0019] The grayscale image is processed by multiple Gaussian smoothing kernels of different scales to obtain a series of smoothed images, where each scale corresponds to a different smoothing standard deviation;

[0020] Calculate the gradient magnitude and direction at each scale respectively, apply non-maximum suppression to obtain the corresponding edge image, combine the edge images at each scale to obtain the multi-scale edge detection result, and thus construct the first edge image corresponding to the grayscale image;

[0021] After obtaining the first edge image, find the coordinate set of the edge points, and by traversing the edge points, find the minimum and maximum values of the horizontal coordinates and the minimum and maximum values of the vertical coordinates to determine the bounding box of the barcode;

[0022] The center coordinates of the barcode in the grayscale image are calculated to obtain the relative position of the barcode in the entire grayscale image. The offset ratio relative to the image boundary is obtained by calculating the ratio of the coordinates of the bounding box relative to the width and height of the grayscale image.

[0023] Optionally, in a fourth implementation of the present invention, converting samples in the training data set from the spatial domain to the frequency domain using a fast Fourier transform to obtain frequency domain features includes:

[0024] The samples in the training data set are subjected to row-wise FFT transformation using the radix-2 time decimation FFT algorithm to obtain the intermediate frequency domain matrix.

[0025] Apply the radix-2 time extraction FFT algorithm to each column of the intermediate frequency domain matrix to perform FFT transformation in the column direction to obtain the frequency domain image F(u,v);

[0026] After calculating the frequency domain image, the spectrum amplitude |F(u,v)| is obtained:

[0027]

[0028] Where Re(F(u,v)) is the real part of F(u,v), and Im(F(u,v)) is the imaginary part of F(u,v);

[0029] The frequency domain image is divided into low-frequency and high-frequency areas based on the spectrum amplitude. The low-frequency energy, high-frequency energy and the ratio of low-frequency to high-frequency energy are calculated respectively, and the frequency domain features are obtained by integration.

[0030] Optionally, in a fifth implementation of the present invention, the training of the QR code image restoration model using a training dataset from which image features have been extracted includes:

[0031] The weight matrices and bias terms of the convolutional neural network, recurrent neural network, and generative adversarial network are randomly initialized. In each round of training, samples are randomly extracted from the training dataset and input into the convolutional neural network for feature extraction to obtain feature vectors. The feature vectors are then input into the recurrent neural network for sequence information capture to obtain a fused feature vector.

[0032] The fused feature vector is input into the generator to generate the repaired image. At the same time, the corresponding real undamaged image is selected from the training data set. The generated repaired image and the real undamaged image are input into the discriminator to calculate the discrimination probability;

[0033] According to the discrimination probability and the generated repair image, the loss function values of the generator and discriminator are calculated respectively, and the model parameters are updated using the gradient descent algorithm. The iterative training process is repeated until the performance index of the model on the validation set reaches the preset threshold, and the trained QR code image repair model is obtained, where the preset threshold is the similarity index structure similarity index greater than or the decoding accuracy is higher than 90%.

[0034] Optionally, in a sixth implementation of the present invention, the weight matrices and bias terms of the convolutional neural network, the recurrent neural network, and the generative adversarial network are randomly initialized. In each round of training, samples are randomly extracted from the training data set, the samples are input into the convolutional neural network for feature extraction to obtain feature vectors, and then the feature vectors are input into the recurrent neural network for sequence information capture to obtain a fused feature vector, including:

[0035] Convolutional neural networks are composed of multiple convolutional layers, pooling layers, and fully connected layers. For the samples in the input training data set, they first pass through the convolution layer for convolution operation. The convolution layer contains multiple convolution kernels, followed by the pooling layer using maximum pooling. After alternating multiple convolution and pooling layers, the feature vector is obtained through the fully connected layer.

[0036] Input the feature vector into the recurrent neural network, divide the feature vector into sequences by rows or columns, and update the hidden state based on the current input and the hidden state at the previous moment:

[0037] h t =tanh(W ih I t +W hh h t-1 +b n );

[0038] Among them, h t is the hidden state at the current moment, I t is the input at the current moment, h t-1 is the hidden state of the previous moment, W ih、W hh is the weight matrix, W ih Used to convert the current moment input into part of the hidden state, W hh Used to convert the hidden state of the previous moment into part of the hidden state of the current moment, b n is the bias term, and tanh is the activation function used to introduce nonlinear transformation;

[0039] The hidden state of the entire sequence after processing is obtained, and it is fused with the feature vector extracted by the convolutional neural network through a splicing operation to obtain a fused feature vector.

[0040] Optionally, in a seventh implementation of the present invention, obtaining the repaired image output by the QR code image repair model and post-processing the repaired image includes:

[0041] Obtain the inpainted image output by the QR code image inpainting model, select a square structuring element for dilation, traverse all coordinates in the structuring element, and assign the maximum pixel value in the neighborhood of each pixel in the image to the point, so that the image boundary is expanded to obtain the dilated image;

[0042] The square structure element is used again to erode the expanded image. All coordinates in the structure element are traversed and the minimum value of the pixel value in the neighborhood of each pixel in the image is assigned to the point. This shrinks the boundaries of the image objects, removes the redundant parts in the expansion process, and cleans up the point-like vacancies to obtain the eroded image.

[0043] After multiple alternating dilation and erosion operations, the point-like vacancies in the repaired image output by the QR code image repair model are removed to obtain an optimized image.

[0044] Optionally, in an eighth implementation of the present invention, the performing positioning, codeword extraction, and decoding on the post-processed optimized image to obtain a repaired image includes:

[0045] Use a two-dimensional Gaussian filter to smooth the post-processed optimized image, and obtain a Gaussian filtered image through convolution operation;

[0046] Use the Sobel operator to calculate the gradient magnitude and direction of the Gaussian filtered image in the x and y directions, and perform non-maximum suppression on the gradient magnitude.

[0047] Set two thresholds T low and T high , the gradient amplitude is greater than T high The pixel points with gradient amplitude between T low and T higt The pixels between are marked as weak edge points, and the pixels with gradient amplitude less than Tlow The pixels of are set to 0, and the second edge image is obtained by connecting the weak edge points to the strong edge points;

[0048] Matching the second edge image with a preset barcode template, calculating the similarity between the barcode template and each region in the second edge image using a normalized cross-correlation method, and locating the position of the barcode pattern based on the similarity, wherein the positions of the barcode pattern are the upper left corner, the upper right corner, and the lower left corner;

[0049] The coordinates of the four vertices of the QR code are determined according to the center coordinates of the three positioning patterns found, and the coordinates of the four vertices of the QR code are obtained through geometric calculation.

[0050] Optionally, in a ninth implementation of the present invention, after determining the coordinates of the four vertices of the QR code based on the center coordinates of the three found positioning patterns and obtaining the coordinates of the four vertices of the QR code through geometric calculation, the method further includes:

[0051] Perform perspective transformation on the post-processed optimized image based on the four vertex coordinates of the QR code to obtain a corrected QR code image;

[0052] The corrected QR code image I corrected (x, y) is divided into small codewords. Assume that the size of each codeword is s×s. For the codeword region R in the pth row and qth column, pq (x,y), calculate the average gray value g of the pixels in the area pq :

[0053]

[0054] Among them, I corrected (x,y) is the QR code image, and (x,y) is the coordinate in the image after post-processing optimization;

[0055] According to the average gray value g pq With threshold T code Compare and determine the value of the code word. If g pq <T code , then the code word is 0, otherwise it is 1;

[0056] The extracted codeword sequence is corrected using a Reed-Solomon error correction code to obtain a corrected codeword sequence, which is then decoded according to the encoding mode of the barcode to obtain a repaired image.

[0057] In the technical solution provided by the present invention, original barcode images of various sizes and contents are obtained, and the original barcode images are subjected to multi-angle transformation and cropping operations using image enhancement technology to generate image samples containing missing barcode information; the enhanced image samples are organized into a training data set, and each sample in the training data set is labeled, and an image feature extraction algorithm is used to mine the image features of the samples in the training data set; the training data set after image feature extraction is used to train a two-dimensional code image repair model, and AIGC is integrated into the two-dimensional code image repair model. The two-dimensional code image repair model automatically generates a repaired image based on the input damaged image through the generative ability of AIGC; the repaired image output by the two-dimensional code image repair model is obtained, the repaired image is post-processed, and the image after post-processing optimization is positioned, codewords are extracted and decoded to obtain a repaired image; the present invention is diversified The barcode image generation and enhancement technology has built a comprehensive data set, which enables the repair model to learn the barcode features in various complex scenarios, greatly improving the generalization ability of the model, and can cope with various types of damaged QR code repair needs generated in different industries and environments; applying AIGC technology to QR code repair can intelligently learn barcodes, barcode features and barcode textures in various scenarios through feature repair, fading, damage, and incomplete repair, and restore the damaged barcode according to the original barcode to ensure the decoding success to the greatest extent, effectively improve the recognition rate of the repaired QR code, and reduce the decoding failure rate; introducing expansion and corrosion operations in the post-processing stage to specifically optimize the quality of the repaired image, remove noise interference, enhance the stability and clarity of the barcode image, provide reliable protection for the decoding link, ensure the accuracy and efficiency of the entire repair and decoding process, and effectively promote the widespread application of QR code technology in complex real-life scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0059] Figure 1 A schematic diagram of a first embodiment of a deep learning-based QR code image restoration method provided by an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a second embodiment of a deep learning-based QR code image restoration method provided by an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a third embodiment of the deep learning-based QR code image repair method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, apparatus, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0063] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A schematic diagram of a first embodiment of a deep learning-based QR code image restoration method provided by an embodiment of the present invention, wherein the method specifically comprises the following steps:

[0064] Step 101: Obtain original barcode images of various sizes and contents, and use image enhancement technology to perform multi-angle transformation and cropping operations on the original barcode images to generate image samples containing missing barcode information.

[0065] In this embodiment, original barcode images of various sizes and contents are obtained, and each original barcode image is rotated according to a preset rotation angle, where the rotation angle is 5°-15°; the rotated original barcode image is flipped horizontally or vertically to make the image symmetrical or upside down; and the four edges of the original barcode image that has undergone multi-angle transformation are cropped, with 10% to 20% of the pixels being cropped each time, thereby generating image samples containing varying degrees of missing barcode information.

[0066] Step 102: organize the enhanced image samples into a training data set, label each sample in the training data set, and use an image feature extraction algorithm to mine image features of the samples in the training data set;

[0067] In this embodiment, the image features include at least texture features, geometric features, and frequency domain features;

[0068] Step 103: Using the training dataset that has been subjected to image feature extraction, the QR code image restoration model is trained. AIGC is integrated into the QR code image restoration model. The QR code image restoration model automatically generates a restoration image based on the input damaged image through the generative capability of AIGC.

[0069] In this embodiment, the QR code image repair model adopts a multi-layer neural network, including a convolutional neural network for feature extraction, a recurrent neural network for capturing barcode image sequence information, and a generative adversarial network for generating high-quality repaired images.

[0070] In this embodiment, the generative capability of AIGC is utilized to enable the model to automatically generate a reasonable repair plan based on the input damaged image. By training the generator, it can learn the repair patterns under different damage conditions, thereby generating high-quality repaired images. During the repair process, the model can make intelligent decisions on how to repair the damaged part based on the extracted features and learned knowledge. For example, for faded areas, the model can restore their original colors through the learned color distribution. For incomplete parts, the model can reasonably fill in the missing information based on the surrounding features and the encoding rules of the barcode. With the emergence of new QR code images and damage conditions, the model can continue to evolve through continuous learning. Incremental learning, transfer learning and other technologies can be used to allow the model to quickly adapt to new situations without retraining.

[0071] Step 104: Obtain the repaired image output by the QR code image repair model, perform post-processing on the repaired image, perform positioning, codeword extraction and decoding on the image after post-processing optimization, and obtain a repaired image.

[0072] In this embodiment, a repaired image output by a two-dimensional code image repair model is obtained, a square structure element is selected to perform an expansion operation, all coordinates in the structure element are traversed, and the maximum value of the pixel value in the neighborhood of each pixel point in the image is assigned to the point, so that the image boundary is expanded to obtain an expanded image; the square structure element is used again to perform an erosion operation on the expanded image, all coordinates in the structure element are traversed, and the minimum value of the pixel value in the neighborhood of each pixel point in the image is assigned to the point, so that the image object boundary is contracted, the redundant parts in the expansion process are removed, and the point-like vacancies are cleaned up to obtain an eroded image; after multiple alternating expansion and erosion operations, the point-like vacancies in the repaired image output by the two-dimensional code image repair model are removed to obtain an optimized image.

[0073] In this embodiment, a two-dimensional Gaussian filter is used to smooth the image after post-processing optimization, and a Gaussian filtered image is obtained by convolution operation;

[0074] Use the Sobel operator to calculate the gradient magnitude and direction of the Gaussian filtered image in the x and y directions, and perform non-maximum suppression on the gradient magnitude.

[0075] Set two thresholds T low and T high , the gradient amplitude is greater than T highThe pixel points with gradient amplitude between T low and T high The pixels between are marked as weak edge points, and the pixels with gradient amplitude less than T low The pixels of are set to 0, and the second edge image is obtained by connecting the weak edge points to the strong edge points;

[0076] Matching the second edge image with a preset barcode template, calculating the similarity between the barcode template and each region in the second edge image using a normalized cross-correlation method, and locating the position of the barcode pattern based on the similarity, wherein the positions of the barcode pattern are the upper left corner, the upper right corner, and the lower left corner;

[0077] The coordinates of the four vertices of the QR code are determined according to the center coordinates of the three positioning patterns found, and the coordinates of the four vertices of the QR code are obtained through geometric calculation.

[0078] In this embodiment, the perspective transformation of the post-processed optimized image is performed based on the four vertex coordinates of the QR code to obtain the corrected QR code image.

[0079] The corrected QR code image I corrected (x, y) is divided into small codewords. Assume that the size of each codeword is s×s. For the codeword region R in the pth row and qth column, pq (x,y), calculate the average gray value g of the pixels in the area pq :

[0080]

[0081] Among them, I corrected (x,y) is the QR code image, and (x,y) is the coordinate in the image after post-processing optimization;

[0082] According to the average gray value g pq With threshold T code Compare and determine the value of the code word. If g pq <T code , then the code word is 0, otherwise it is 1;

[0083] The extracted codeword sequence is corrected using a Reed-Solomon error correction code to obtain a corrected codeword sequence, which is then decoded according to the encoding mode of the barcode to obtain a repaired image.

[0084] See also Figure 2 , a schematic diagram of a second embodiment of a deep learning-based two-dimensional code image repair method provided by an embodiment of the present invention, the method comprising:

[0085] Step 201: Organize the enhanced image samples into a training data set, and annotate key information for each sample in the training data set, where the key information annotation includes at least a barcode type annotation, an original complete information annotation, and a damage description annotation;

[0086] Step 202: Convert the color images in the training data set into grayscale images, and use the gray-level co-occurrence matrix to extract texture features of the grayscale images;

[0087] Step 203: Obtain a first edge image corresponding to the grayscale image using the Canny edge detection algorithm, determine the barcode's bounding box from the first edge image, and obtain the barcode's relative position in the entire grayscale image and its offset ratio relative to the image boundary;

[0088] In this embodiment, a local window is used to estimate the noise level of each pixel in the grayscale image, an adaptive Gaussian smoothing kernel is generated based on the local noise estimate of each pixel, and the adaptive Gaussian kernel is applied to the grayscale image pixel by pixel; the grayscale image is processed using Gaussian smoothing kernels of multiple scales to obtain a series of smoothed images, where each scale corresponds to a different smoothing standard deviation; the gradient amplitude and direction at each scale are calculated, and non-maximum suppression is applied to obtain the corresponding edge image. The edge images at each scale are combined to obtain a multi-scale edge detection result, thereby constructing a first edge image corresponding to the grayscale image; after obtaining the first edge image, a coordinate set of edge points is searched, and by traversing the edge points, the minimum and maximum values of the horizontal coordinate and the minimum and maximum values of the vertical coordinate are found to determine the bounding box of the barcode; the center coordinates of the barcode in the grayscale image are calculated to obtain the relative position of the barcode in the entire grayscale image, and the offset ratio relative to the image boundary is calculated by calculating the ratio of the coordinates of the bounding box to the width and height of the grayscale image.

[0089] Step 204: Use fast Fourier transform to convert the samples in the training data set from the spatial domain to the frequency domain to obtain frequency domain features.

[0090] In this embodiment, a radix-2 time decimation FFT algorithm is used to perform row-wise FFT transformation on the samples in the training data set to obtain an intermediate frequency domain matrix;

[0091] Apply the radix-2 time extraction FFT algorithm to each column of the intermediate frequency domain matrix to perform FFT transformation in the column direction to obtain the frequency domain image F(u,v);

[0092] After calculating the frequency domain image, the spectrum amplitude |F(u,v)| is obtained:

[0093]

[0094] Where Re(F(u,v)) is the real part of F(u,v), and Im(F(u,v)) is the imaginary part of F(u,v);

[0095] The frequency domain image is divided into low-frequency and high-frequency areas based on the spectrum amplitude. The low-frequency energy, high-frequency energy and the ratio of low-frequency to high-frequency energy are calculated respectively, and the frequency domain features are obtained by integration.

[0096] See also Figure 3 , a schematic diagram of a third embodiment of a deep learning-based two-dimensional code image repair method provided by an embodiment of the present invention, the method comprising:

[0097] Step 301: randomly initialize the weight matrices and bias terms of the convolutional neural network, recurrent neural network, and generative adversarial network. In each round of training, randomly extract samples from the training data set, input the samples into the convolutional neural network for feature extraction, obtain feature vectors, and then input the feature vectors into the recurrent neural network for sequence information capture to obtain a fused feature vector.

[0098] In this embodiment, the convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers. For the samples in the input training data set, the convolution layer first performs a convolution operation. The convolution layer contains multiple convolution kernels, followed by a pooling layer using maximum pooling. After alternating multiple convolution and pooling layers, the feature vector is obtained through the fully connected layer.

[0099] Input the feature vector into the recurrent neural network, divide the feature vector into sequences by rows or columns, and update the hidden state based on the current input and the hidden state at the previous moment:

[0100] h t =tanh(W ih I t +W hh h t-1 +b n );

[0101] Among them, h t is the hidden state at the current moment, I t is the input at the current moment, h t-1 is the hidden state of the previous moment, W ih 、W hh is the weight matrix, W ih Used to convert the current moment input into part of the hidden state, W hh Used to convert the hidden state of the previous moment into part of the hidden state of the current moment, b n is the bias term, and tanh is the activation function used to introduce nonlinear transformation;

[0102] The hidden state of the entire sequence after processing is obtained, and it is fused with the feature vector extracted by the convolutional neural network through a splicing operation to obtain a fused feature vector.

[0103] Step 302: Input the fused feature vector into the generator to generate a repaired image. At the same time, select the corresponding real undamaged image from the training data set, input the generated repaired image and the real undamaged image into the discriminator, and calculate the discrimination probability;

[0104] Step 303: Calculate the loss function values of the generator and the discriminator respectively based on the discrimination probability and the generated repaired image, use the gradient descent algorithm to update the model parameters, and repeat the iterative training process until the performance index of the model on the validation set reaches the preset threshold, thereby obtaining the trained QR code image repair model.

[0105] In this embodiment, the preset threshold is that the similarity index of the similarity indicator structure is greater than or the decoding accuracy is higher than 90%.

[0106] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A QR code image restoration method based on deep learning, characterized in that: The deep learning-based QR code image repair method includes the following steps: Obtaining original barcode images of various sizes and contents, and performing multi-angle transformation and cropping operations on the original barcode images using image enhancement technology to generate image samples containing some missing barcode information; The enhanced image samples are organized into a training data set, and each sample in the training data set is labeled. An image feature extraction algorithm is used to mine the image features of the samples in the training data set, where the image features include at least texture features, geometric features, and frequency domain features. The QR code image restoration model is trained using a training dataset that has undergone image feature extraction. AIGC is integrated into the model. The model automatically generates a restored image based on the input damaged image through the generative capabilities of AIGC. The QR code image restoration model uses a multi-layer neural network, including a convolutional neural network for feature extraction, a recurrent neural network for capturing barcode image sequence information, and a generative adversarial network for generating high-quality restored images. The repaired image output by the QR code image repair model is obtained, the repaired image is post-processed, and positioning, codeword extraction and decoding are performed on the image after post-processing optimization to obtain the repaired image.

2. The deep learning-based two-dimensional code image restoration method according to claim 1, wherein: The method of obtaining original barcode images of various sizes and contents, performing multi-angle transformation and cropping operations on the original barcode images using image enhancement technology to generate image samples containing missing barcode information includes: Obtain original barcode images of various sizes and contents, and rotate each original barcode image according to a preset rotation angle of 5°-15°; Flip the rotated original barcode image horizontally or vertically to make the image symmetrical or upside down; The original barcode image that has undergone multi-angle transformation is cropped from its four edges, with 10% to 20% of the pixels being cropped each time, thereby generating image samples containing varying degrees of missing barcode information.

3. The deep learning-based two-dimensional code image repair method according to claim 1, characterized in that: The enhanced image samples are organized into a training data set, each sample in the training data set is labeled, and an image feature extraction algorithm is used to mine the image features of the samples in the training data set, including: The enhanced image samples are organized into a training data set, and key information is annotated for each sample in the training data set, where the key information annotation includes at least the barcode type annotation, the original complete information annotation, and the damage description annotation; The color images in the training data set are converted into grayscale images, and the texture features of the grayscale images are extracted using the gray-level co-occurrence matrix; Obtain a first edge image corresponding to the grayscale image using the Canny edge detection algorithm, determine the barcode's bounding box from the first edge image, and obtain the barcode's relative position in the entire grayscale image and its offset ratio relative to the image boundary; Fast Fourier transform is used to convert samples in the training data set from the spatial domain to the frequency domain to obtain frequency domain features.

4. The deep learning-based two-dimensional code image restoration method according to claim 3, wherein: The method of obtaining a first edge image corresponding to the grayscale image by using the Canny edge detection algorithm, determining the bounding box of the barcode from the first edge image, and obtaining the relative position of the barcode in the entire grayscale image and the offset ratio relative to the image boundary includes: The noise level of each pixel in the grayscale image is estimated using a local window, an adaptive Gaussian smoothing kernel is generated based on the local noise estimation value of each pixel, and the adaptive Gaussian kernel is applied to the grayscale image pixel by pixel; The grayscale image is processed by multiple Gaussian smoothing kernels of different scales to obtain a series of smoothed images, where each scale corresponds to a different smoothing standard deviation; Calculate the gradient magnitude and direction at each scale respectively, apply non-maximum suppression to obtain the corresponding edge image, combine the edge images at each scale to obtain the multi-scale edge detection result, and thus construct the first edge image corresponding to the grayscale image; After obtaining the first edge image, find the coordinate set of the edge points, and by traversing the edge points, find the minimum and maximum values of the horizontal coordinates and the minimum and maximum values of the vertical coordinates to determine the bounding box of the barcode; The center coordinates of the barcode in the grayscale image are calculated to obtain the relative position of the barcode in the entire grayscale image. The offset ratio relative to the image boundary is obtained by calculating the ratio of the coordinates of the bounding box relative to the width and height of the grayscale image.

5. The deep learning-based two-dimensional code image repair method according to claim 3, characterized in that: The method of converting samples in the training dataset from the spatial domain to the frequency domain using the fast Fourier transform to obtain frequency domain features includes: The samples in the training data set are subjected to row-wise FFT transformation using the radix-2 time decimation FFT algorithm to obtain the intermediate frequency domain matrix. Apply the radix-2 time extraction FFT algorithm to each column of the intermediate frequency domain matrix to perform FFT transformation in the column direction to obtain the frequency domain image F(u,v); After calculating the frequency domain image, the spectrum amplitude |F(u,v)| is obtained: Where Re(F(u,v)) is the real part of F(u,v), and Im(F(u,v)) is the imaginary part of F(u,v); The frequency domain image is divided into low-frequency and high-frequency areas based on the spectrum amplitude. The low-frequency energy, high-frequency energy and the ratio of low-frequency to high-frequency energy are calculated respectively, and the frequency domain features are obtained by integration.

6. The deep learning-based two-dimensional code image restoration method according to claim 1, wherein: The method of training the QR code image restoration model using the training data set after image feature extraction includes: The weight matrices and bias terms of the convolutional neural network, recurrent neural network, and generative adversarial network are randomly initialized. In each round of training, samples are randomly extracted from the training dataset and input into the convolutional neural network for feature extraction to obtain feature vectors. The feature vectors are then input into the recurrent neural network for sequence information capture to obtain a fused feature vector. The fused feature vector is input into the generator to generate the repaired image. At the same time, the corresponding real undamaged image is selected from the training data set. The generated repaired image and the real undamaged image are input into the discriminator to calculate the discrimination probability; According to the discrimination probability and the generated repair image, the loss function values of the generator and discriminator are calculated respectively, and the model parameters are updated using the gradient descent algorithm. The iterative training process is repeated until the performance index of the model on the validation set reaches the preset threshold, and the trained QR code image repair model is obtained, where the preset threshold is the similarity index structure similarity index greater than or the decoding accuracy is higher than 90%.

7. The deep learning-based two-dimensional code image repair method according to claim 6, characterized in that: The weight matrices and bias items of the convolutional neural network, recurrent neural network, and generative adversarial network are randomly initialized. In each round of training, samples are randomly extracted from the training data set, the samples are input into the convolutional neural network for feature extraction to obtain feature vectors, and then the feature vectors are input into the recurrent neural network for sequence information capture to obtain a fused feature vector, including: Convolutional neural networks are composed of multiple convolutional layers, pooling layers, and fully connected layers. For the samples in the input training data set, they first pass through the convolution layer for convolution operation. The convolution layer contains multiple convolution kernels, followed by the pooling layer using maximum pooling. After alternating multiple convolution and pooling layers, the feature vector is obtained through the fully connected layer. Input the feature vector into the recurrent neural network, divide the feature vector into sequences by rows or columns, and update the hidden state based on the current input and the hidden state at the previous moment: h t =tanh(W ih I t +W hh h t-1 +b n ); Among them, h t is the hidden state at the current moment, I t is the input at the current moment, h t-1 is the hidden state of the previous moment, W ih 、W hh is the weight matrix, W ih Used to convert the current moment input into part of the hidden state, W hh Used to convert the hidden state of the previous moment into part of the hidden state of the current moment, b n is the bias term, and tanh is the activation function used to introduce nonlinear transformation; The hidden state of the entire sequence after processing is obtained, and it is fused with the feature vector extracted by the convolutional neural network through a splicing operation to obtain a fused feature vector.

8. The deep learning-based two-dimensional code image restoration method according to claim 1, wherein: The step of obtaining the repaired image output by the QR code image repair model and performing post-processing on the repaired image includes: Obtain the inpainted image output by the QR code image inpainting model, select a square structuring element for dilation, traverse all coordinates in the structuring element, and assign the maximum pixel value in the neighborhood of each pixel in the image to the point, so that the image boundary is expanded to obtain the dilated image; The square structure element is used again to erode the expanded image. All coordinates in the structure element are traversed and the minimum value of the pixel value in the neighborhood of each pixel in the image is assigned to the point. This shrinks the boundaries of the image objects, removes the redundant parts in the expansion process, and cleans up the point-like vacancies to obtain the eroded image. After multiple alternating dilation and erosion operations, the point-like vacancies in the repaired image output by the QR code image repair model are removed to obtain an optimized image.

9. The deep learning-based two-dimensional code image restoration method according to claim 1, wherein: The positioning, codeword extraction and decoding of the post-processed optimized image to obtain a repaired image includes: Use a two-dimensional Gaussian filter to smooth the post-processed optimized image, and obtain a Gaussian filtered image through convolution operation; Use the Sobel operator to calculate the gradient magnitude and direction of the Gaussian filtered image in the x and y directions, and perform non-maximum suppression on the gradient magnitude. Set two thresholds T low and T high , the gradient amplitude is greater than T high The pixel points with gradient amplitude between T low and T high The pixels between are marked as weak edge points, and the pixels with gradient amplitude less than T low The pixels of are set to 0, and the second edge image is obtained by connecting the weak edge points to the strong edge points; Matching the second edge image with a preset barcode template, calculating the similarity between the barcode template and each region in the second edge image using a normalized cross-correlation method, and locating the position of the barcode pattern based on the similarity, wherein the positions of the barcode pattern are the upper left corner, the upper right corner, and the lower left corner; The coordinates of the four vertices of the QR code are determined according to the center coordinates of the three positioning patterns found, and the coordinates of the four vertices of the QR code are obtained through geometric calculation.

10. The deep learning-based two-dimensional code image repair method according to claim 9, characterized in that: After determining the coordinates of the four vertices of the QR code based on the center coordinates of the three found positioning patterns and obtaining the coordinates of the four vertices of the QR code through geometric calculation, the method further includes: Perform perspective transformation on the post-processed optimized image based on the four vertex coordinates of the QR code to obtain a corrected QR code image; The corrected QR code image I corrected (x, y) is divided into small codewords. Assume that the size of each codeword is s×s. For the codeword region R in the pth row and qth column, pq (x,y), calculate the average gray value g of the pixels in the area pq : Among them, I corrected (x,y) is the QR code image, and (x,y) is the coordinate in the image after post-processing optimization; According to the average gray value g pq With threshold T code Compare and determine the value of the code word. If g pq <T code , then the code word is 0, otherwise it is 1; The extracted codeword sequence is corrected using a Reed-Solomon error correction code to obtain a corrected codeword sequence, which is then decoded according to the encoding mode of the barcode to obtain a repaired image.

Images