A fuzzy two-dimensional code recognition method based on a generative adversarial network and an attention mechanism

Abstract

The application discloses a motion blur two-dimensional code recognition method based on a generative adversarial network and an attention mechanism. The method is implemented by adopting a convolutional neural network model and specifically comprises the following steps: firstly, a multi-scale residual block and a multi-scale feature extraction module are adopted to construct a backbone network; secondly, a hollow convolutional layer and an efficient attention mechanism are introduced to better capture the details and texture information in the two-dimensional code image; in addition, a perception loss function and a WGAN-div adversarial loss function are combined to train the deblurring network proposed by the application; finally, a blurred two-dimensional code dataset is made to train and verify the deblurring network. The motion blur two-dimensional code recognition method proposed by the application has achieved significant improvement in processing time and recognition accuracy when processing the task of recognizing a seriously blurred two-dimensional code.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent information processing, and specifically relates to a fuzzy QR code recognition method based on generative adversarial networks and attention mechanisms. Background Technology

[0002] With the rapid development of IoT technology and the information industry, QR codes, as a means of information storage and sharing, have been widely used in many fields such as logistics management, mobile payment, product traceability, medical services, and intelligent transportation. Various QR code recognition methods already exist, such as ZBar, Zxing, and OpenCV, which, based on different algorithms and strategies, have achieved some success. However, these existing methods have some significant drawbacks in specific situations, limiting their effectiveness and applicability in practical applications. For example, in intelligent logistics and transportation systems, when intelligent mobile robots are recognizing and navigating, their QR code systems may be affected by factors such as movement speed, camera shake, or equipment vibration, resulting in blurred QR code images and reduced recognition accuracy. Therefore, developing QR code recognition algorithms with image deblurring capabilities is an urgent problem to be solved at this stage.

[0003] Motion blur removal is a challenging problem in computer vision and image processing. Its core task is to recover an image that closely resembles the original, sharp image from a blurred one. Motion blur is typically caused by various factors, such as camera shake, object movement, and camera calibration errors. Deblurring methods can be categorized into non-blind image deblurring with a known blur kernel and blind image deblurring with an unknown blur kernel. Traditional algorithms usually simulate features by applying various constraints and estimating different prior information. However, these traditional algorithms perform poorly when processing images with non-uniform motion blur and have limited performance in complex scenes. Furthermore, their computational cost is high due to the need for complex physical models, sometimes making them unacceptable in practical applications.

[0004] In recent years, with the rapid development of deep learning technology, many blind motion deblurring algorithms based on deep learning have emerged. These end-to-end methods can produce better results than traditional methods. However, some deep learning-based methods are difficult to train due to their complex structure and too many neural network layers. Furthermore, there are very few learning-based algorithms specifically designed for motion-blurred QR code recognition.

[0005] In the process of QR code image restoration, focusing on detail recovery is crucial. Generative Adversarial Networks (GANs) have been widely applied to image processing tasks due to their superior characteristics, including the ability to preserve rich details in images and generate near-realistic images. Meanwhile, efficient attention mechanisms also play an important role in image processing. This mechanism successfully captures the correlation between channels through a local cross-channel interaction strategy without dimensionality reduction. This mechanism can adaptively adjust features channel-wise to enhance useful feature information and suppress useless feature information. When this efficient attention mechanism is introduced into deblurring networks, the effect of restoring image details and structure can be further improved. Summary of the Invention

[0006] This invention addresses the problems mentioned in the technical background by proposing a blurred QR code recognition method based on generative adversarial networks (GANs) and an attention mechanism. This method employs improved residual blocks and multi-scale feature modules as basic building blocks, aiming to enhance the receptive field of the deblurring network and improve feature richness. Furthermore, an improved attention mechanism module is introduced to more effectively capture image details and texture information. The discriminator network uses the PatchGAN discriminator to better capture local information and provide fine-grained feedback. This network structure can efficiently handle complex motion blur situations, achieving end-to-end QR code image restoration. Specifically, it includes the following steps:

[0007] (1) Construct a deblurring network structure. The network structure is based on a generative adversarial network. The generator network uses an improved residual block and a multi-scale feature extraction module as the basic building blocks. An improved efficient attention mechanism and a multi-scale dilated convolutional layer are introduced in the middle. The discriminator network adopts a fully convolutional network structure and includes instance normalization and LeakyReLU activation function. The negative slope of LeakyReLU is set to α = 0.2.

[0008] (2) Create a QR code dataset. By shooting QR code videos and averaging the consecutive frames, a blurry image can be generated. The middle frame of these consecutive frames is used as the clear image, thus creating a blurry QR code dataset.

[0009] (3) Design the loss function. Our loss function is a weighted sum of the perceptual loss and the adversarial loss, as shown in the following formula:

[0010] L = L adv +100L p (1)

[0011] Where L adv To counteract the loss function, L p For the perceptual loss function;

[0012] (4) Dataset preprocessing: the blurred QR codes and clear QR codes are matched and randomly cropped to form images with a resolution of 256x256;

[0013] (5) Train the network model by inputting the processed images in pairs into the deblurring network for training. Set the batch size to 1 and use the Adam optimization algorithm to optimize the generator and discriminator respectively. The learning rate is initially set to 10. -4 After 150 epochs, the learning rate is linearly decayed to 0, and a total training process of 300 epochs is performed. Then the weight parameters are saved to obtain the final model.

[0014] (6) Deblurring: Input the blurred QR code image into the final deblurring network model to obtain the reconstructed QR code image.

[0015] (7) Decoding and recognition: Input the reconstructed QR code image into the QR code decoding library for decoding, thereby obtaining the QR code recognition content.

[0016] Furthermore, the generator network has the following detailed structure: two 3×3 convolutional blocks with a stride of 1, two multi-scale feature extraction modules, two 3×3 convolutional blocks with a stride of 2, nine improved residual modules, two transposed convolutional blocks, and two improved efficient attention mechanism modules.

[0017] Furthermore, the improved residual block is mainly based on the res2net module, which adds a multi-scale dilated convolutional block after the first 1×1 convolutional layer, and adds an improved efficient attention mechanism module after the second 1×1 convolutional layer of the module.

[0018] Furthermore, the multi-scale feature extraction module captures features at different scales using multiple parallel convolutional branches and global average pooling. Specifically, the first branch includes a hierarchy from 1×1 convolutional layers to multi-scale dilated convolutional blocks to 3×3 convolutional layers; the second branch includes a hierarchy from 1×1 convolutional layers to multi-scale dilated convolutional blocks to 3×3 convolutional layers; the third branch includes a hierarchy from 1×1 convolutional layers to multi-scale dilated convolutional blocks; and the fourth branch includes max pooling layers and 1×1 convolutional layers.

[0019] Furthermore, the improved efficient attention mechanism module divides the input into two channels, performs global average pooling and global max pooling operations on each channel, and then performs 1D convolution on each channel before inputting the results in parallel into the Sigmoid layer.

[0020] Furthermore, the adversarial loss function adopts WGAN-div, which combines distribution divergence and Wasserstein distance, as the adversarial loss function, and its formula is as follows:

[0021]

[0022] Where z represents the blurred image and x represents the sharp image. P is obtained by sampling through a linear combination of real and spurious data points. u It is a Radon probability measure, where k controls the influence of the gradient term on the objective function, and p corresponds to the L-factor of the function f. p In addition, the value was 2 in the experiment and 6 in the space;

[0023] Perceptual loss reduces the loss of high-frequency details in the generated image by comparing higher-level feature representations. Its calculation formula is as follows:

[0024]

[0025] Among them, I B I S and G(I) B ) represent the blurred image, the sharp image, and the deblurred image, respectively. φ i,j W represents the feature map obtained from the j-th convolutional layer (after activation) before the i-th max-pooling layer of VGG-19. i,j and H i,j This represents the size of the feature map.

[0026] Furthermore, the multi-scale dilated convolutional block is composed of three convolutional layers with dilation rates of 1, 2 and 4 in parallel, corresponding to receptive field sizes of 3, 5 and 9, respectively.

[0027] According to the technical solution proposed in this paper, we can train a deblurring network using pairs of blurred QR code images and their corresponding clear QR code images. The main goal of this network is to learn how to restore a blurred QR code image to a clear state. Through this training method, we can build a model specifically for blurred QR code recognition. In complex and dynamic scenarios, this trained blurred QR code recognition model can handle blurred QR code images. By extracting information from the image, the model can reconstruct the corresponding clear QR code image. Furthermore, we can identify the QR code content contained in the clear QR code image. This entire process enables reliable recognition and decoding of blurred QR codes even in challenging environments, thereby improving the practicality and applicability of QR code technology. Attached Figure Description

[0028] Figure 1 This is a flowchart of the deblurring network in this invention;

[0029] Figure 2 This is a flowchart of the QR code recognition process in this invention;

[0030] Figure 3 A schematic diagram of the adversarial network is generated for this invention;

[0031] Figure 4 This is a detailed framework diagram of the deblurring network of the present invention;

[0032] Figure 5 This invention provides an improved residual block diagram.

[0033] Figure 6 This is a diagram of the multi-scale feature extraction module of the present invention;

[0034] Figure 7 This is a diagram of the multi-scale dilated convolution block of the present invention;

[0035] Figure 8 This is a block diagram of the improved efficient attention mechanism of the present invention; Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings of the embodiments thereof:

[0037] The fuzzy QR code recognition method of this invention includes the following parts: constructing a deblurring network structure, creating a QR code dataset, designing a loss function, preprocessing the dataset, training the network model, deblurring the fuzzy QR code, decoding, and recognizing it. The process of this method is as follows: Figure 1 and Figure 2 As shown.

[0038] The main features of the fuzzy QR code recognition method based on Generative Adversarial Networks (GANs) and attention mechanisms proposed in this invention lie in the two key aspects of defuzzification network construction and network training. This method integrates and improves upon the principles of GANs, attention mechanisms, and the characteristics of various network modules. The detailed steps and features of this invention are as follows:

[0039] Step 1: Constructing the deblurring network structure

[0040] Utilizing the principles and training methods of Generative Adversarial Networks (GANs), a GAN model specifically designed for QR code image deblurring was constructed. This GAN consists of a generator network and a discriminator network. The generator network receives a blurred QR code image as input and generates a clear QR code image. The discriminator network's task is to distinguish between the QR code image generated by the generator and the real, clear QR code image. The overall network structure is as follows: Figure 3 As shown.

[0041] The generator network employs improved residual blocks and multi-scale feature modules as basic building blocks. The improved residual blocks feature hierarchical residual connections to represent multi-scale features and incorporate multi-scale dilated convolutional blocks. Furthermore, an improved, efficient attention mechanism module is added to the residual blocks to enhance network performance. Detailed structure is as follows: Figure 5 As shown. The multi-scale feature extraction module introduces convolutional kernels and pooling operations of different sizes to increase the number of branches in the network, extracting and fusing image features from different scales. The detailed structure is shown below. Figure 6 As shown.

[0042] In this invention, an improved, efficient attention mechanism module is integrated into the generative adversarial network. It splits the input into two channels, performing global average pooling and global max pooling operations on each channel respectively, followed by 1D convolution and input to a sigmoid layer to enhance the attention given to the image. Detailed structure is as follows... Figure 8 As shown.

[0043] The final deblurring network structure is as follows: Figure 4 As shown, the generator network comprises multiple 3x3 convolutional blocks with a stride of 1, a multi-scale feature extraction module, 3x3 convolutional blocks with a stride of 2, an improved residual module, and transposed convolutional blocks. Each convolutional unit includes an instance normalization layer and a LeakyReLU activation function layer to stabilize the training process. Global skip connections are also introduced so that the network learns a mapping from the blurred image to the residual correction. The discriminator network employs a fully convolutional network structure, including instance normalization and a LeakyReLU activation function, where the negative slope of LeakyReLU is set to 0.2.

[0044] Step 2: Creating a QR code dataset

[0045] This invention generates a blurred image by capturing 240 frames of QR code video and then averaging consecutive frames from frame 15 to frame 21. Simultaneously, the middle frames of these consecutive frames are used as sharp images to create a blurred QR code dataset. Subsequently, the QR code dataset is divided into training and testing sets at a 2:1 ratio for model training and performance evaluation.

[0046] Step 3: Design the loss function

[0047] Our loss function is a weighted sum of the perceptual loss and the adversarial loss, as shown in the following formula:

[0048] L = L adv +100L p (4)

[0049] Where L adv To counteract the loss function, L pFor the perceptual loss function;

[0050] The adversarial loss function uses WGAN-div, which combines distribution divergence and Wasserstein distance. WGAN-div more comprehensively measures the difference between the generator output distribution and the real sample distribution, and can significantly improve the visual quality of the generated images and the stability of the network. Its formula is as follows:

[0051]

[0052] Where z represents the blurred image and x represents the sharp image. P is obtained by sampling through a linear combination of real and spurious data points. u It is a Radon probability measure, where k controls the influence of the gradient term on the objective function, and p corresponds to the L-factor of the function f. p In addition, the value was 2 in the experiment and 6 in the space;

[0053] Perceptual loss reduces the loss of high-frequency details in the generated image by comparing higher-level feature representations. Its calculation formula is as follows:

[0054]

[0055] Among them, I B I S and G(I) B ) represent the blurred image, the sharp image, and the deblurred image, respectively. φ i,j W represents the feature map obtained from the j-th convolutional layer (after activation) before the i-th max-pooling layer of VGG-19. i,j and H i,j This represents the size of the feature map.

[0056] Step 4: Dataset Preprocessing

[0057] The blurred and clear QR codes were matched and randomly cropped to form an image with a resolution of 256x256.

[0058] Step 5: Train the network model

[0059] The processed images are input in pairs into the deblurring network for training. The batch size is set to 1. The Adam optimization algorithm is used to optimize the generator and discriminator respectively. The learning rate is initially set to 10⁻⁴. After 150 epochs, the learning rate is linearly decayed to 0. The training process is carried out for a total of 300 epochs. Then the weight parameters are saved to obtain the final model.

[0060] Step 6: Deblurring. Input the blurred QR code image into the final deblurring network model to obtain the reconstructed QR code image.

[0061] Step 7: Decoding and recognition. Input the reconstructed QR code image into the QR code decoding library for decoding to obtain the QR code's recognition content.

[0062] The relevant experiments quantitatively evaluated a GoPro test set containing 1111 images using mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM) values. The results are shown in Table 1.

[0063] Table 1. Performance comparison on large GoPro datasets

[0064]

[0065] According to the data in Table 1, the method described in this invention exhibits the best performance among all the compared methods.

[0066] The test evaluation results of the training set of the fuzzy QR code dataset are shown in Table 2. The fuzzy QR codes without any processing, the fuzzy QR codes processed by DeblurGAN, the fuzzy QR codes processed by DeblurGAN-v2, and the fuzzy QR codes processed by the method of this invention were identified respectively.

[0067] Table 2. Comparison of Recognition Rate and Average Running Time

[0068] method Recognition rate (%) Time (s) benchmark 14.56 0.07 DeblurGAN 36.76 1.09 DeblurGAN-v2 56.91 0.67 This invention 71.02 0.55

[0069] As shown in Table 2, the recognition rate of unprocessed blurred QR codes is extremely low, while the recognition rate of the method described in this invention is as high as 71.02%, significantly higher than other algorithms, improving the QR code recognition rate by 387%. Furthermore, the average processing time for each QR code was calculated. The processing and recognition time for each QR code using the method described in this invention is 0.55 seconds, only half the time used by DeblurGAN, and the processing speed is 18% faster than DeblurGAN-v2. Although the processing time is longer than the unprocessed case, the significant improvement in recognition rate is even more significant.

[0070] The above description is merely an embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for recognizing fuzzy QR codes based on generative adversarial networks and attention mechanisms, characterized in that, Includes the following steps: (1) Construct a defuzzification network structure, with generative adversarial network as the main architecture; The generator network uses improved residual blocks and multi-scale feature extraction modules as basic building blocks, and introduces an improved efficient attention mechanism and multi-scale dilated convolutional layers in the middle. The discriminator network adopts a fully convolutional network structure and includes instance normalization and LeakyReLU activation function. (2) Create a QR code dataset by shooting QR code videos and averaging the consecutive frames to generate a blurry image. Use the middle frame of these consecutive frames as the clear image to create a blurry QR code dataset. (3) Design the loss function, which is a weighted sum of the perceptual loss and the adversarial loss, as shown in the following formula: in To counteract the loss function, For the perceptual loss function; (4) Dataset preprocessing: the blurred QR codes and clear QR codes are matched and randomly cropped to form images with a resolution of 256x256; (5) Train the network model by inputting the processed images in pairs into the deblurring network for training. Set the batch size to 1 and use the Adam optimization algorithm to optimize the generator and discriminator respectively. The learning rate is initially set to 10. -4 After 150 epochs, the learning rate is linearly decayed to 0, and a total training process of 300 epochs is performed. Then the weight parameters are saved to obtain the final model. (6) Deblurring: Input the blurred QR code image into the final deblurring network model to obtain the reconstructed QR code image; (7) Decoding and recognition: Input the reconstructed QR code image into the QR code decoding library for decoding, thereby obtaining the QR code recognition content; The generator network in step 1) has the following components: two 3×3 convolutional blocks with a stride of 1, two multi-scale feature extraction modules, two 3×3 convolutional blocks with a stride of 2, nine improved residual modules, two transposed convolutional blocks, and two improved efficient attention mechanism modules. The residual block in step 1) has been improved. Based on the res2net module, a multi-scale dilated convolution block is added after the first 1×1 convolutional layer, and an improved efficient attention mechanism module is added after the second 1×1 convolutional layer of the module. The multi-scale feature extraction module in step 1) captures features at different scales using multiple parallel convolutional branches and global average pooling. The first branch includes layers from 1×1 convolutional layers to multi-scale dilated convolutional blocks, then to 3×3 convolutional layers, and finally to 3×3 convolutional layers. The second branch includes layers from 1×1 convolutional layers to multi-scale dilated convolutional blocks, then to 3×3 convolutional layers. The third branch includes layers from 1×1 convolutional layers to multi-scale dilated convolutional blocks. The fourth branch includes max pooling layers and 1×1 convolutional layers. The improved efficient attention mechanism module in step 1) divides the input into two channels, performs global average pooling and global max pooling operations on them respectively, and then performs 1D convolution on them respectively before inputting the results in parallel into the Sigmoid layer.

2. The fuzzy QR code recognition method based on generative adversarial networks and attention mechanisms according to claim 1, characterized in that, In step 3), the adversarial loss function uses WGAN-div, which combines the distribution divergence and Wasserstein distance, as its formula: Where z represents the blurred image and x represents the sharp image. It is obtained by sampling through a linear combination of real data points and spurious data points. It is a Radon probability measure, where k controls the effect of the gradient term on the objective function, and p corresponds to the gradient of function f. space; Perceptual loss reduces the loss of high-frequency details in the generated image by comparing higher-level feature representations. Its calculation formula is as follows: in, , and These represent blurred image, sharp image, and deblurred image, respectively. The feature map obtained from the j-th convolutional layer before the i-th max-pooling layer of VGG-19 is represented. and This represents the size of the feature map.

3. The fuzzy QR code recognition method based on generative adversarial networks and attention mechanisms according to claim 1, characterized in that, The multi-scale dilated convolutional block consists of three convolutional layers in parallel with dilation rates of 1, 2 and 4, corresponding to receptive field sizes of 3, 5 and 9, respectively.

Images