Two-dimensional code image restoration method and device, two-dimensional code image recognition method and device, chip and electronic equipment

Abstract

The embodiment of the invention provides a two-dimensional code image restoration method and device, an identification method and device, a chip and electronic equipment, and the method comprises the steps: carrying out the marking of a target damaged region of a two-dimensional code image, obtaining a two-dimensional code marking image, inputting the two-dimensional code marking image into a trained image restoration model, and obtaining a two-dimensional code restoration image; the recognition method comprises the steps of obtaining a two-dimensional code image, repairing the two-dimensional code image by using a two-dimensional code image repairing method to obtain a two-dimensional code repairing image, and recognizing the two-dimensional code repairing image to obtain target information. According to the scheme, the image resolution is improved, the damaged area of the two-dimensional code pattern is effectively repaired, the integrity of the two-dimensional code pattern is recovered, the two-dimensional code image can be accurately recognized to obtain the target information, and good user experience is guaranteed.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method and device for repairing a two-dimensional code image, a recognition method, a chip, and an electronic device. Background Art

[0002] With the continuous development and progress of information technology, people's demand for information storage and transmission is increasing day by day. Among them, as an important information carrier, the two-dimensional code has gradually become an important tool for information transmission due to its advantages of large storage capacity, fast reading speed, high error tolerance, etc., and is widely used in modern commerce, payment, logistics and other fields. However, in the actual application process, the two-dimensional code image is often affected by external factors such as scratches, stains, and pollution, resulting in the distortion or occlusion of the two-dimensional code pattern, so that the two-dimensional code cannot be correctly recognized.

[0003] In the related art, image repair methods such as image enhancement and noise removal are used to repair the two-dimensional code. However, the foregoing solutions mainly redistribute the gray values of the image to improve the visual effect of the image, and cannot effectively repair the image details of the damaged or low-clarity two-dimensional code image, and it is easy to cause the failure of two-dimensional code recognition, affecting the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a method and device for repairing a two-dimensional code image, a recognition method, a chip, and an electronic device, which solve the problem that the related art cannot effectively repair the image details of the damaged or low-clarity two-dimensional code image, and it is easy to cause the failure of two-dimensional code recognition, affecting the user experience. The method realizes improving the image resolution, effectively repairing the damaged area of the two-dimensional code pattern, restoring the integrity of the two-dimensional code pattern, so that the two-dimensional code image can be accurately recognized to obtain the target information, and ensuring a good user experience.

[0005] In a first aspect, the embodiments of the present application provide a method for repairing a two-dimensional code image, the method comprising:

[0006] Performing labeling processing on the target damaged area of the two-dimensional code image to obtain a two-dimensional code labeled image;

[0007] Inputting the two-dimensional code labeled image into a trained image repair model to obtain a two-dimensional code repaired image.

[0008] In a second aspect, the embodiments of the present application provide a method for recognizing a two-dimensional code image, the method comprising:

[0009] Obtaining a two-dimensional code image;

[0010] Using the method for repairing a two-dimensional code image according to any one of claims 1-8 to repair the two-dimensional code image to obtain a two-dimensional code repaired image;

[0011] Identify the target information from the repaired QR code image.

[0012] Thirdly, an embodiment of the present application further provides a QR code image repair device, including:

[0013] An image annotation module configured to perform annotation processing on the damaged target area of the QR code image to obtain a QR code annotation image;

[0014] A QR code repair module configured to input the QR code annotation image into a trained image repair model to obtain a QR code repair image.

[0015] Fourthly, an embodiment of the present application further provides a QR code image recognition device, including:

[0016] An acquisition module configured to acquire a QR code image;

[0017] An image repair module configured to repair the QR code image using the QR code image repair method described in any embodiment of the present application to obtain a QR code repair image;

[0018] A QR code recognition module configured to identify the target information from the repaired QR code image.

[0019] Fifthly, an embodiment of the present application further provides a chip, which includes:

[0020] One or more processors;

[0021] A storage device configured to store one or more programs,

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the QR code image repair method or the QR code image recognition method described in the embodiments of the present application.

[0023] Sixthly, an embodiment of the present application further provides an electronic device, which includes the chip described in the embodiment of the present application.

[0024] Seventhly, an embodiment of the present application further provides a non-volatile storage medium storing computer-executable instructions, and the computer-executable instructions are configured to execute the QR code image repair method or the QR code image recognition method described in the embodiments of the present application when executed by a computer processor.

[0025] In the embodiments of the present application, the QR code repair method obtains a QR code annotation image by performing annotation processing on the damaged area of the QR code image, and inputs the QR code annotation image into the trained image repair model to obtain a QR code repair image. In the above solution, by performing annotation processing on the damaged area of the QR code image to obtain a QR code annotation image, the key area to be repaired in the QR code can be accurately calibrated. By inputting the QR code annotation image into the trained image repair model, the damaged area of the QR code pattern can be effectively repaired, and the integrity of the QR code pattern can be restored. The QR code recognition method obtains a QR code image, repairs the QR code image using the QR code image repair method to obtain a QR code repair image, and recognizes the QR code repair image to obtain target information. In the above solution, by using the QR code image repair method to repair the QR code image, the damaged area of the QR code pattern can be effectively repaired, the integrity of the QR code pattern can be restored, and by recognizing the QR code repair image to obtain target information, the target information can be accurately recognized, ensuring a good user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a QR code image repair method provided by an embodiment of the present application;

[0027] Figure 2 is a flowchart of a QR code image repair method provided by an embodiment of the present application, including a process of performing annotation processing on a QR code image;

[0028] Figure 3 is a flowchart of a QR code image repair method provided by an embodiment of the present application, including a process of performing mask annotation processing on a QR code enhanced image;

[0029] Figure 4 is a flowchart of a training process of a target detection model in step S201 or step S301 provided by an embodiment of the present application;

[0030] Figure 5 is a flowchart of a training process of an image enhancement model in step S202 or step S302 provided by an embodiment of the present application;

[0031] Figure 6 is a flowchart of a training process of an image repair model in step S102, step S204 or step S305 provided by an embodiment of the present application;

[0032] Figure 7 is a flowchart of a QR code image repair method provided by an embodiment of the present application, including a process of performing preprocessing on a QR code image;

[0033] Figure 8Flowchart of a QR code image recognition method provided by an embodiment of the present application;

[0034] Figure 9 Flowchart of a QR code image recognition method provided by an embodiment of the present application, including the process of recognizing a repaired QR code image;

[0035] Figure 10 Block diagram of the structure of a QR code image repair device provided by an embodiment of the present application;

[0036] Figure 11 Block diagram of the structure of a QR code image recognition device provided by an embodiment of the present application;

[0037] Figure 12 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0038] The following further describes the embodiments of the present application in detail with reference to the accompanying drawings and examples. It can be understood that the specific embodiments described herein are only used to explain the embodiments of the present application, rather than limiting the embodiments of the present application. In addition, it should be noted that, for the sake of description, only some parts related to the embodiments of the present application are shown in the drawings, rather than all the structures.

[0039] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same type, and do not limit the number of objects. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally represents an "or" relationship between the associated objects before and after.

[0040] The QR code image repair method provided by the embodiments of the present application is used to enhance and repair QR code images. Moreover, the QR code image recognition method provided by the embodiments of the present application is used to enhance and repair QR code images and accurately recognize the processed QR code images to obtain target information. Specific application scenarios may include: online payment, logistics warehousing, coding management, etc. The several application scenarios listed above are only exemplary and explanatory. In actual applications, the QR code image repair method and the QR code recognition method may also be used in QR code image processing in other scenarios, and the embodiments of the present application do not limit this. The present application aims to provide a QR code image repair method to solve the problem in the related art that the damaged or low-clarity QR code images cannot be effectively repaired in terms of image details, resulting in easy failure of QR code recognition and affecting the user experience.

[0041] For the QR code image repair method and recognition method provided by the embodiments of the present application, the execution subject of each step may be a computer device, which refers to any electronic device with data calculation, processing, and storage capabilities, such as terminal devices like mobile phones, PCs (Personal Computers), and tablet computers, or devices such as servers. The embodiments of the present application do not limit this.

[0042] Figure 1 The flowchart of a QR code image repair method provided by the embodiments of the present application can be implemented with a QR code image repair device as the execution subject. As Figure 1 shown, the QR code image repair method specifically includes the following steps:

[0043] Step S101: Perform annotation processing on the target damaged area of the QR code image to obtain a QR code annotation image.

[0044] Among them, the QR code image may be an image containing the QR code pattern to be recognized. In one embodiment, a target detection model can be used to recognize the QR code image, determine the position information corresponding to the target damaged area, and based on this position information, perform mask annotation processing on the QR code image to obtain a QR code annotation image. In one embodiment, an image enhancement model can be used to enhance the QR code image to obtain a QR code enhanced image, then use a target detection model to recognize the QR code enhanced image, determine the position information corresponding to the target damaged area, and finally, based on this position information, perform mask annotation processing on the QR code enhanced image to obtain a QR code annotation image. Through this mask annotation processing, effective positioning information of the target damaged area can be provided for the subsequent image repair model, which is beneficial to the repair of the damaged part of the image.

[0045] Step S102: Input the QR code marked image into the trained image inpainting model to obtain the QR code repaired image.

[0046] Among them, the image inpainting model can adopt deep learning models such as LaMa (LArge MAsk inpainting) and GAN (Generative Adversarial Networks). This application does not make limitations here. Combining with the marking function in step S102, the purpose of this image inpainting model is to infer and reconstruct the content of the missing area based on the context information and surrounding pixels of the image to generate a realistic and consistent image. Specifically, a generative adversarial network model with better performance in quantitative and qualitative evaluations can be adopted, which can reconstruct better textures and synthesize better global structures. This generative adversarial network model can learn the complex distribution and potential features of the data through training, which is beneficial to generating synthetic data very close to the real data, meeting different application requirements, and having strong generalization ability. The generative adversarial network model can include a generator and a discriminator. The generator can be used to generate a reasonable repair result for the target damaged area according to the image information around the marked target damaged area during the actual application process. The discriminator can be used to evaluate the authenticity of the repair effect of the generator during the simulation training process. The generator can include an input layer, multiple convolutional layers, skip connections, deconvolution layers, padding layers, and an output layer. The input layer can be used to receive the QR code marked image. The multiple convolutional layers can be used to perform multi-layer convolutional operations on the QR code marked image to gradually extract the local features of the image to obtain feature maps. Among them, each convolutional layer can adopt standard convolutional operations and the ReLU (Rectified Linear Unit) activation function to accelerate training and enhance the non-linear expression ability. The skip connections can be used to directly transfer the early low-level feature maps to the later stage of the network, which helps the generator maintain the high-frequency information and details of the image during the repair process. The deconvolution layer can be used to gradually restore the resolution of the image until it reaches the same size as the input image. The padding layer can be used to fill the target damaged area according to the surrounding pixel information of the target damaged area to achieve the repair effect. The output layer can be used to output the QR code repaired image. The discriminator can include an input layer, multiple convolutional layers, a fully connected layer, and an output layer. The input layer can be used to receive the QR code repaired image output by the generator and the original lossless QR code image. The multiple convolutional layers can be used to perform multi-layer convolutional operations on the QR code repaired image to extract image features and gradually reduce the spatial resolution to capture the high-level semantic features in the image. The fully connected layer can be used to map the extracted features to a binary classification output, indicating whether the QR code repaired image is real or generated. The output layer can be used to output a probability value indicating the authenticity of the input QR code repaired image.

[0047] As described above, a QR code annotation image is obtained by performing annotation processing on the damaged target area of the QR code image, and the QR code annotation image is input into a trained image restoration model to obtain a QR code restored image. In the above solution, a QR code annotation image is obtained by performing annotation processing on the damaged target area of the QR code image, which can accurately calibrate the key area to be restored in the QR code. By inputting the QR code annotation image into a trained image restoration model, the damaged area of the QR code pattern can be effectively restored, and the integrity of the QR code pattern can be restored.

[0048] Figure 2 The following is a flowchart of a QR code image restoration method provided by an embodiment of the present application, which includes a process of performing annotation processing on a QR code image, as Figure 2 shown. The QR code image restoration method specifically includes the following steps:

[0049] Step S201: Input the QR code image into a trained object detection model to obtain the position information corresponding to the damaged target area in the QR code image.

[0050] Among them, the object detection model can be a YOLOv5 model, a Faster R-CNN model, etc., and the present application does not make any limitations here. The object detection model can locate the damaged target area in the QR code image and output the position information corresponding to the damaged target area. The damaged target area can be one or more abnormal scratch areas or abnormal pollution areas in the QR code image, and the position information can be the bounding box coordinates corresponding to the damaged target area, for example, the upper left corner coordinates and the lower right corner coordinates, which are used to indicate the specific position of the damaged target area in the QR code image.

[0051] Step S202: Input the QR code image into a trained image enhancement model to obtain a QR code enhanced image.

[0052] Among them, the image enhancement model can be a convolutional neural network model constructed based on a super-resolution algorithm. Specifically, developers can select a suitable super-resolution algorithm according to the requirements of the actual application scenario, and the present application does not make any limitations here. Specifically, the image enhancement model can include an input layer, a convolutional layer, a residual block, an upsampling layer, and an output layer connected in sequence. The input layer can be used to receive the QR code image, the convolutional layer can be used to initially extract features to obtain a feature image, the residual block can be used to enhance feature learning through skip connections, the upsampling layer can perform super-resolution upsampling on the feature image to restore image clarity, and the output layer can be used to output the QR code enhanced image. Thus, the high-frequency information and details of the low-resolution QR code image can be effectively restored by using the image enhancement model, achieving the purpose of enhancing the clarity of the QR code image.

[0053] Step S203: Based on the position information corresponding to the target damaged area, perform mask annotation processing on the enhanced QR code image to obtain a QR code annotation image.

[0054] Among them, by combining the position information corresponding to the target damaged area, the pixel area corresponding to the target damaged area can be located in the enhanced QR code image. This mask annotation processing can be to adjust the pixel values of the pixel area corresponding to the target damaged area in the enhanced QR code image. For example, set the pixel values to missing values or a certain gray value, so as to obtain a QR code annotation image marked with the target damaged area, which is beneficial to subsequent instructions for the generative adversarial network model to perform effective repair.

[0055] Step S204: Input the QR code annotation image into the trained image repair model to obtain a QR code repaired image.

[0056] As described above, by inputting the QR code image into the trained target detection model, the damaged area in the QR code image can be effectively located, providing reliable position information for subsequent QR code repair. By inputting the QR code image into the trained image enhancement model, the image resolution can be improved, the high-frequency details in the image can be restored, and the clarity of the image can be improved. By performing mask annotation processing on the enhanced QR code image to obtain a QR code annotation image, the key area that needs to be repaired in the QR code can be accurately calibrated.

[0057] Figure 3 The flowchart of a QR code image repair method provided by an embodiment of the present application, which includes a process of performing mask annotation processing on the enhanced QR code image, as Figure 3 shown, the QR code image repair method specifically includes the following steps:

[0058] Step S301: Input the QR code image into the trained target detection model to obtain the position information corresponding to the target damaged area in the QR code image.

[0059] Step S302: Input the QR code image into the trained image enhancement model to obtain an enhanced QR code image.

[0060] Step S303: Locate at least one pixel area in the enhanced QR code image that matches the position information corresponding to the target damaged area.

[0061] Among them, since the position information corresponding to the target damaged area can indicate the specific pixel area where the target damaged area is located in the enhanced QR code image, therefore, at least one pixel area corresponding to the target damaged area can be located from the enhanced QR code image, which is convenient for subsequent mask assignment.

[0062] Step S304: Set the pixel value corresponding to each pixel in the pixel region of the QR code enhanced image to a preset mask value to obtain a QR code labeled image.

[0063] Among them, in order to effectively mark the pixel region corresponding to the target damaged area in the QR code enhanced image, the pixel value corresponding to each pixel in the relevant pixel region can be set to a preset mask value. Since the QR code pattern is usually composed of black and white patterns, therefore, the preset mask value can be set to a certain gray value between black and white, or a missing value, to distinguish it from the original QR code pattern.

[0064] Step S305: Input the QR code labeled image into the trained image inpainting model to obtain a QR code inpainted image.

[0065] As described above, by locating the pixel region corresponding to the target damaged area in the QR code enhanced image and setting the pixel value corresponding to each pixel in the pixel region to a preset mask value, the damaged part can be effectively marked on the basis of the original QR code pattern, thus providing effective indication information for the subsequent repair function of the generative adversarial network model.

[0066] Figure 4 The following is a flowchart of a training process of the target detection model in step S201 or step S301 provided by an embodiment of the present application. It should be noted that the training process of the target detection model in step S201 or step S301 is at least executed before step S201 or step S301. As Figure 4 shown, the training process of the target detection model specifically includes the following steps:

[0067] Step S401: Obtain a sample QR code image and reference position information corresponding to a damaged area in the sample QR code image, and the sample QR code image includes at least one damaged area.

[0068] Among them, the sample QR code image can be QR code images of different scenarios, different formats or different degrees of damage collected through different channels, which can be captured in actual application scenarios or simulated and generated by developers. The present application does not make any limitations here. And the position information corresponding to the damaged area can be accurately labeled by developers in advance for the sample QR code image. For example, the position information can be the bounding box coordinates of the damaged area.

[0069] Step S402: Train the target detection model set based on the sample QR code image, the reference position information corresponding to the damaged area, and the set regression loss function until the optimal target detection model is obtained. The optimal target detection model is the trained target detection model.

[0070] Among them, during the training process, inputting the sample QR code image into the target detection model can obtain the predicted position information, and based on the set regression loss function, the loss value between the predicted position information and the reference position information can be calculated. The regression loss function can be the mean square error loss function, the mean absolute error loss function, etc. Then, the optimizer is used to iteratively update the parameters of the target detection model according to the loss value to minimize the loss function until the loss converges to obtain the optimal target detection model. The optimizer can be the stochastic gradient descent optimizer, the Adam (Adaptive Moment Estimation) optimizer, etc.

[0071] The optimal target detection model is used as the target detection model trained in step S201 or step S301 to process the QR code image and obtain the position information corresponding to the damaged area of the target in the QR code image.

[0072] As described above, by training the target detection model based on the sample QR code image, the reference position information corresponding to the damaged area, and the set regression loss function, the target detection model can be enabled to have the ability to accurately calibrate the damaged area in the QR code image, which is beneficial to accurately determining the position of the damaged area of the target in the QR code image during the actual application process.

[0073] Figure 5 This is a flowchart of a training process of the image enhancement model in step S202 or step S302 provided by the embodiments of the present application. It should be noted that the training process of the image enhancement model in step S202 or step S302 is at least executed before step S202 or step S302. As Figure 5 shown, the training process of the image enhancement model specifically includes the following steps:

[0074] Step S501: Obtain a sample QR code image and perform downsampling on the sample QR code image to obtain a low-resolution image.

[0075] Among them, the sample QR code image can be high-definition QR code images in different scenarios and different formats collected through different channels, which can be taken in the actual application scenario or simulated and generated by developers. This application does not make any limitations here. By performing downsampling on the high-resolution sample QR code image, a low-resolution image can be generated. Among them, the downsampling factor can be 2 times, 4 times, or 8 times. This application does not make any limitations here. Optionally, noise, blur, distortion, etc. can be added to the low-resolution image to simulate the QR code image in the actual environment. Optionally, data augmentation can also be performed on the QR code image through rotation, scaling, cropping, translation, blur, etc. to increase the robustness of the model.

[0076] Step S502: Train the image enhancement model set based on the sample QR code image, the low-resolution image, and the set mean square error loss function until the optimal image enhancement model is obtained. The optimal image enhancement model is the trained image enhancement model.

[0077] During the training process, inputting the sample QR code image into the image enhancement model can obtain a super-resolution image, and based on the set mean square error loss function, the loss value between the super-resolution image and the original sample QR code image can be calculated. Then, use the Adam optimizer to iteratively update the parameters of the image enhancement model according to this loss value to minimize the loss function until the loss converges to obtain the optimal object detection model. The calculation formula of the mean square error loss function is as follows:

[0078]

[0079] Where, I SR (x i ) is the super-resolution image generated by the super-resolution model, I HR (x i ) is the original sample QR code image, and N is the number of pixels in the image.

[0080] This optimal image enhancement model is used as the trained image enhancement model in Step S202 or Step S302 to process the QR code image and obtain a QR code enhanced image.

[0081] Through the above training of the image enhancement model based on the sample QR code image, the low-resolution image, and the set mean square error loss function, the image enhancement model can be made to have the ability to restore the high-frequency details of the QR code, which is beneficial to enhancing the texture details of the QR code image and improving the clarity of the QR code image during the actual application process.

[0082] Figure 6 This is a flowchart of a training process of the image repair model in Step S102, Step S204, or Step S305 provided by the embodiments of the present application. It should be noted that the training process of the image repair model in Step S102, Step S204, or Step S305 is at least executed before Step S102, Step S204, or Step S305. This image repair model is a generative adversarial network model, specifically including a generator and a discriminator. As Figure 6 shown, the training process of this image repair model specifically includes the following steps:

[0083] Step S601: Obtain a sample QR code image and a lossless QR code image. The sample QR code image contains at least one masked pixel area.

[0084] Among them, the lossless QR code image can be high-quality QR code images in different scenarios and different formats collected through different channels, which can be captured in actual application scenarios or simulated and generated by developers. This application does not make any limitations here. The sample QR code image can be obtained by developers predefining at least one masked pixel region for the lossless QR code image. The masked pixel region can be used to represent the damaged region after the lossless QR code image simulates the influence of adverse factors in the actual environment. Specifically, the pixel value corresponding to each pixel in the masked pixel region can be a missing value or a certain gray value. This application does not make any limitations here.

[0085] Step S602: Alternately train the generator and the discriminator based on the sample QR code image, the lossless QR code image, and the set cross-entropy loss function until the optimal generative adversarial network model is obtained.

[0086] Among them, for training the discriminator, the lossless QR code image and the predicted QR code image generated by the generator based on the sample QR code image can be respectively input into the discriminator to obtain the authenticity probability results of the lossless QR code image and the predicted QR code image output by the discriminator. Then, according to the discriminator loss function, the sum of the losses of the discriminator for the lossless QR code image and the predicted QR code image can be calculated. Finally, through backpropagation, the gradient of the loss function with respect to the discriminator parameters can be calculated, and the optimizer is used to update the discriminator parameters to minimize the discriminator loss until the loss converges to obtain the optimal discriminator. It should be noted that the closer the authenticity judgment result of the discriminator for the lossless QR code image is to 1, the better, and the closer the authenticity judgment result of the discriminator for the predicted QR code image generated by the generator is to 0, the better. Therefore, the discriminator loss function aims to maximize the judgment accuracy for the lossless QR code image while minimizing the judgment accuracy for the predicted QR code image, thereby improving the discrimination ability between the lossless QR code image and the predicted QR code image. For training the generator, the sample QR code image can be input into the generator to obtain the masked pixel region and the repaired predicted QR code image. Then, the predicted QR code image is input into the discriminator to obtain the authenticity probability result of the discriminator for the predicted QR code image. Then, according to the generator loss function, the loss value of this authenticity probability result relative to the maximum probability value 1 can be calculated. Finally, through backpropagation, the gradient of the generator loss function with respect to the generator parameters is calculated, and then the optimizer is used to update the generator parameters to minimize the generator loss until the loss converges to obtain the optimal generator. It should be noted that the generator loss function aims to maximize the judgment error rate of the discriminator for the generated data, so that the generator can gradually improve the quality of the generated data, making it more and more difficult for the discriminator to distinguish between the lossless QR code image and the predicted QR code image. During the training process, the discriminator and the generator can be alternately trained. For example, the discriminator is trained for k steps first, and then the generator is trained for 1 step. The value range of k can be 1 - 5. Thus, it helps the generator and the discriminator to jointly improve their performance in the confrontation with each other.

[0087] The optimal image repair model, as the image repair model trained in step S102, step S204 or step S305, is used to process the QR code labeled image and obtain the QR code repaired image.

[0088] As described above, through the alternating training of the generator and the discriminator based on the sample QR code image, the lossless QR code image and the set cross-entropy loss function, the generative adversarial network model can be made to have the ability to repair the calibrated masked pixel region, which is beneficial to effectively repair the damaged region in the QR code image during the actual application process and restore the integrity of the QR code image.

[0089] Figure 7The flowchart of a QR code image repair method provided by an embodiment of the present application, which includes a process of preprocessing a QR code image, is as follows Figure 7 As shown, the QR code image repair method specifically includes the following steps:

[0090] Step S701: Perform grayscale conversion on the QR code image to obtain a grayscale image, perform Gaussian filtering on the grayscale image to obtain a filtered image, and perform equalization on the filtered image to obtain a target QR code image.

[0091] Among them, since the QR code pattern is usually composed of black and white modules, the color information in the color image does not play a significant role in the repair of the QR code pattern. Therefore, if the QR code image is in color, convert the QR code image to a grayscale image. This grayscale conversion process can specifically generate a single-channel grayscale image by weighted averaging the pixel values of the RGB channels in the QR code image. Since the QR code image is usually easily affected by noise during shooting or printing, Gaussian filtering can be performed on the grayscale image to remove the noise in the image, smooth the image, and retain the edge information. Since the QR code image may be taken in an environment with uneven illumination, histogram equalization based on adaptive histogram can be performed on the filtered image to enhance the image contrast.

[0092] Step S702: Perform annotation processing on the target damaged area of the target QR code image to obtain a QR code annotation image.

[0093] Step S703: Input the QR code annotation image into the trained image repair model to obtain a QR code repair image.

[0094] As described above, by performing grayscale conversion on the QR code image to obtain a grayscale image, the computational complexity can be reduced. By performing Gaussian filtering on the grayscale image to obtain a filtered image, the influence of redundant noise in the image can be effectively removed, ensuring the image quality. By performing equalization on the filtered image to obtain a target QR code image, the image contrast can be enhanced, improving the recognizability of the QR code pattern.

[0095] Figure 8 The flowchart of a QR code image recognition method provided by an embodiment of the present application is as follows Figure 8 As shown, the QR code image recognition method specifically includes the following steps:

[0096] Step S801: Obtain a QR code image.

[0097] Step S802: Use the QR code image repair method provided by any embodiment of the present application to repair the QR code image to obtain a QR code repair image.

[0098] Step S803: Recognize the QR code repair image to obtain target information.

[0099] Among them, the target information can be a website link, text information, contact information, product details, geographical location, etc., and this application does not make any limitations here. In one embodiment, identifying the QR code repair image may be to first input the QR code repair image into a set QR code detection model to obtain the positioning points and data areas of the QR code, and extract the QR code pattern according to the positioning points and data areas. The QR code detection model can be a YOLOv5 model, a Faster R-CNN model, etc., and this application does not make any limitations here. Then, input the QR code pattern into a set QR code decoding model to obtain the target information. The QR code decoding model can be constructed based on machine learning or a convolutional neural network. In one embodiment, identifying the QR code repair image may be to determine the corner positions of the QR code pattern based on a set corner detection algorithm, extract the QR code pattern according to the corner positions, and then decode the QR code pattern according to a preset decoding algorithm to obtain the target information.

[0100] As described above, by obtaining the QR code image, using the QR code image repair method to repair the QR code image to obtain the QR code repair image, and identifying the QR code repair image to obtain the target information. In the above solution, by using the QR code image repair method to repair the QR code image, the damaged area of the QR code pattern can be effectively repaired, the integrity of the QR code pattern can be restored, and by identifying the QR code repair image to obtain the target information, the target information can be accurately identified, ensuring a good user experience.

[0101] Figure 9 The following is a flowchart of a QR code image recognition method provided by an embodiment of this application, which includes a process of identifying a QR code repair image. As Figure 9 shown, the QR code image recognition method specifically includes the following steps:

[0102] Step S901, obtain a QR code image.

[0103] Step S902, use the QR code image repair method provided by any embodiment of this application to repair the QR code image to obtain a QR code repair image.

[0104] Step S903, perform corner detection on the QR code repair image to obtain the target corner positions, and extract the target QR code pattern from the QR code repair image according to the target corner positions.

[0105] Among them, corner detection of the QR code repaired image can be based on corner detection algorithms such as Hough transform, Moravec corner detection algorithm, Harris corner detection algorithm, etc., which are not limited in this application. By locating the positions of the target corners, the rotation angle and position of the QR code pattern can be further determined, so as to accurately extract the target QR code pattern.

[0106] Step S904, decode the target QR code pattern according to the preset coding rule to obtain the target information.

[0107] Among them, the data area can be divided from the target QR code pattern according to the preset coding rule, and the black and white modules in the QR code pattern can be decoded using the set QR code decoding algorithm. This QR code decoding algorithm can be based on QR code decoding libraries such as the Zxing library, and finally the target information stored in the target QR code pattern can be extracted.

[0108] As described above, by extracting the target QR code pattern from the QR code repaired image according to the target corner positions, the integrity and accuracy of the extracted QR code pattern can be ensured, and the irrelevant background information in the image can be removed. By decoding the target QR code pattern according to the preset coding rule to obtain the target information, the target information stored in the target QR code pattern can be effectively extracted.

[0109] Figure 10 The following is a structural block diagram of a QR code image repair device provided by an embodiment of the present application. This device is configured to execute the QR code image repair method provided by the above embodiment, and has corresponding functional modules and beneficial effects for executing the method. As Figure 10 shown, the device specifically includes:

[0110] An image annotation module 101, configured to perform annotation processing on the damaged area of the QR code image to obtain a QR code annotated image;

[0111] A QR code repair module 102, configured to input the QR code annotated image into a trained image repair model to obtain a QR code repaired image.

[0112] As described above, by performing annotation processing on the damaged area of the QR code image to obtain a QR code annotated image, and inputting the QR code annotated image into a trained image repair model to obtain a QR code repaired image. In the above solution, by performing annotation processing on the damaged area of the QR code image to obtain a QR code annotated image, the key area that needs to be repaired in the QR code can be accurately calibrated. By inputting the QR code annotated image into a trained image repair model, the damaged area of the QR code pattern can be effectively repaired, and the integrity of the QR code pattern can be restored.

[0113] In a possible embodiment, the image annotation module 101 is further configured to:

[0114] Input the QR code image into the trained object detection model to obtain the position information corresponding to the damaged target area in the QR code image, and input the QR code image into the trained image enhancement model to obtain the enhanced QR code image;

[0115] Based on the position information corresponding to the damaged target area, perform mask annotation processing on the enhanced QR code image to obtain the annotated QR code image.

[0116] In a possible embodiment, the image annotation module 101 is further configured to:

[0117] Locate at least one pixel region in the enhanced QR code image that matches the position information corresponding to the damaged target area;

[0118] Set the pixel value corresponding to each pixel in the pixel region of the enhanced QR code image to a preset mask value to obtain the annotated QR code image.

[0119] In a possible embodiment, it further includes a first model training module, configured to:

[0120] Obtain a sample QR code image and the reference position information corresponding to the damaged area in the sample QR code image, and the sample QR code image includes at least one damaged area;

[0121] Train the object detection model set based on the sample QR code image, the reference position information corresponding to the damaged area, and the set regression loss function until the optimal object detection model is obtained, and the optimal object detection model is the trained object detection model.

[0122] In a possible embodiment, it further includes a second model training module, configured to:

[0123] Obtain a sample QR code image, and perform downsampling on the sample QR code image to obtain a low-resolution image;

[0124] Train the image enhancement model set based on the sample QR code image, the low-resolution image, and the set mean square error loss function until the optimal image enhancement model is obtained, and the optimal image enhancement model is the trained image enhancement model.

[0125] In a possible embodiment, the image repair model is a generative adversarial network model, the image repair model includes a generator and a discriminator, and it further includes a third model training module, configured to:

[0126] Obtain a sample QR code image and a lossless QR code image, and the sample QR code image includes at least one masked pixel region;

[0127] Alternately train the generator and discriminator based on the sample QR code image, the lossless QR code image, and the set cross-entropy loss function until the optimal generative adversarial network model is obtained.

[0128] In a possible embodiment, it further includes a preprocessing module, configured to:

[0129] Perform grayscale conversion processing on the QR code image to obtain a grayscale image;

[0130] Perform Gaussian filtering processing on the grayscale image to obtain a filtered image;

[0131] Perform equalization processing on the filtered image to obtain the target QR code image;

[0132] Correspondingly, the image annotation module is further configured to:

[0133] Perform annotation processing on the target damaged area of the target QR code image to obtain a QR code annotation image.

[0134] In a possible embodiment, the image enhancement model is a convolutional neural network model constructed based on a super-resolution algorithm.

[0135] Figure 11 This is a structural block diagram of a QR code image recognition device provided by an embodiment of the present application. The device is configured to execute the QR code image recognition method provided by the above embodiment, and has corresponding functional modules and beneficial effects for executing the method. As Figure 11 shown, the device specifically includes:

[0136] An acquisition module 201, configured to acquire a QR code image;

[0137] An image repair module 202, configured to repair the QR code image using the QR code image repair method provided by any embodiment of the present application to obtain a QR code repair image;

[0138] A QR code recognition module 203, configured to recognize the QR code repair image to obtain target information.

[0139] As described above, by acquiring a QR code image, using the QR code image repair method to repair the QR code image to obtain a QR code repair image, and recognizing the QR code repair image to obtain target information. In the above solution, by using the QR code image repair method to repair the QR code image, the damaged area of the QR code pattern can be effectively repaired, the integrity of the QR code pattern can be restored, and by recognizing the QR code repair image to obtain target information, the target information can be accurately recognized, ensuring a good user experience.

[0140] In a possible embodiment, the QR code recognition module 203 is further configured to:

[0141] Perform corner detection on the repaired QR code image to obtain the target corner positions, and extract the target QR code pattern from the repaired QR code image according to the target corner positions;

[0142] Decode the target QR code pattern according to a preset encoding rule to obtain the target information.

[0143] The embodiment of the present application also provides a chip, which includes a processor and a memory; the number of processors in the chip can be one or more. The memory, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the QR code image repair method or the QR code image recognition method in the embodiment of the present application. The processor executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, implements the above-mentioned QR code image repair method or QR code image recognition method.

[0144] Figure 12 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 12 shown, the device includes the chip 301, an input device 302, and an output device 303 provided in the foregoing embodiment; the number of processors 3011 in the chip 301 can be one or more, and a memory 3012 is also provided in the chip 301. Figure 12 Take one processor 3011 as an example; the chip 301, the input device 302, and the output device 303 in the device can be connected through a bus or other means. Figure 12 Take the connection through a bus as an example. The input device 302 can be configured to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device. The output device 303 can include a display device such as a display screen.

[0145] The above-provided electronic device can be used to execute the QR code image repair method or the QR code image recognition method provided in any of the foregoing embodiments, and has corresponding functions and beneficial effects.

[0146] The embodiment of the present application also provides a non-volatile storage medium containing computer-executable instructions. The computer-executable instructions are configured to execute a QR code image repair method or a QR code image recognition method described in any of the foregoing embodiments when executed by a computer processor. Among them, it includes: performing annotation processing on the target damaged area of the QR code image to obtain a QR code annotation image; inputting the QR code annotation image into a trained image repair model to obtain a QR code repair image.

[0147] Storage medium - Any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks, or magnetic tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media, or optical storage; registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network such as the Internet. The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media located in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions executable by one or more processors (e.g., embodied as a computer program).

[0148] Of course, for a storage medium containing computer-executable instructions provided in an embodiment of the present application, the computer-executable instructions are not limited to the above QR code image repair method, and may also perform related operations in the QR code image repair method provided in any embodiment of the present application.

[0149] It should be noted that in the embodiments of the above QR code image repair device or QR code image recognition device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and are not configured to limit the protection scope of the embodiments of the present application.

[0150] It should be noted that the numbering of the steps in this solution is only used to describe the overall design framework of this solution and does not represent an inevitable sequence between the steps. On the basis that the overall implementation process conforms to the overall design framework of this solution, it falls within the protection scope of this solution. The sequential order in the form of text during description is not an exclusive limitation on the specific implementation process of this solution. Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory. The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0151] It should also be noted that the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, commodity or device comprising the element.

[0152] Note that the above is only a preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments here, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments only. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A two-dimensional code image repair method, characterized in that: The method is used to repair a two-dimensional code image, and the method comprises: Annotating the target damaged area of ​​the two-dimensional code image to obtain a two-dimensional code annotated image; The two-dimensional code annotated image is input into the trained image restoration model to obtain a two-dimensional code restoration image.

2. The two-dimensional code image repair method according to claim 1, characterized in that: The step of performing labeling processing on the target damaged area of ​​the two-dimensional code image to obtain a two-dimensional code labeled image includes: Inputting the two-dimensional code image into a trained target detection model to obtain position information corresponding to the damaged target area in the two-dimensional code image, and inputting the two-dimensional code image into a trained image enhancement model to obtain a two-dimensional code enhanced image; Based on the position information corresponding to the target damaged area, the two-dimensional code enhanced image is subjected to mask annotation processing to obtain a two-dimensional code annotated image.

3. The two-dimensional code image repair method according to claim 2, characterized in that: The step of performing mask annotation processing on the two-dimensional code enhanced image based on the position information corresponding to the target damaged area to obtain a two-dimensional code annotated image includes: Locating at least one pixel region in the two-dimensional code enhanced image that matches the position information corresponding to the target damaged region; The pixel value corresponding to each pixel in the pixel area in the two-dimensional code enhanced image is set as a preset mask value to obtain a two-dimensional code annotated image.

4. The two-dimensional code image repair method according to claim 2, characterized in that: Before inputting the two-dimensional code image into the trained target detection model, the method further includes: Acquire a sample two-dimensional code image and reference position information corresponding to a damaged area in the sample two-dimensional code image, wherein the sample two-dimensional code image includes at least one damaged area; The set target detection model is trained based on the sample two-dimensional code image, the reference position information corresponding to the damaged area and the set regression loss function until an optimal target detection model is obtained, and the optimal target detection model is the trained target detection model.

5. The two-dimensional code image repair method according to claim 2, characterized in that: Before inputting the two-dimensional code image into the trained image enhancement model, the method further includes: Acquire a sample two-dimensional code image, and downsample the sample two-dimensional code image to obtain a low-resolution image; The set image enhancement model is trained based on the sample two-dimensional code image, the low-resolution image and the set mean square error loss function until an optimal image enhancement model is obtained, and the optimal image enhancement model is the trained image enhancement model.

6. The two-dimensional code image repair method according to claim 1, characterized in that: The image restoration model is a generative adversarial network model, which includes a generator and a discriminator. Before inputting the two-dimensional code annotated image into the trained image restoration model, it also includes: Acquire a sample two-dimensional code image and a lossless two-dimensional code image, wherein the sample two-dimensional code image includes at least one mask pixel region; The generator and the discriminator are alternately trained based on the sample two-dimensional code image, the lossless two-dimensional code image and the set cross-entropy loss function until an optimal generative adversarial network model is obtained.

7. The two-dimensional code image repair method according to claim 1, characterized in that: Before the target damaged area is annotated on the two-dimensional code image to obtain the two-dimensional code annotated image, the method further includes: Performing grayscale conversion processing on the two-dimensional code image to obtain a grayscale image; Performing Gaussian filtering on the grayscale image to obtain a filtered image; Performing equalization processing on the filtered image to obtain a target two-dimensional code image; Correspondingly, the step of performing labeling processing on the target damaged area of ​​the two-dimensional code image to obtain a two-dimensional code labeled image includes: The target damaged area of ​​the target two-dimensional code image is annotated to obtain a two-dimensional code annotated image.

8. The two-dimensional code image repair method according to claim 2, characterized in that: The image enhancement model is a convolutional neural network model constructed based on a super-resolution algorithm.

9. A two-dimensional code image recognition method, characterized in that: include: Get the QR code image; Repairing the two-dimensional code image using the two-dimensional code image repair method according to any one of claims 1 to 8 to obtain a two-dimensional code repair image; The two-dimensional code repair image is identified to obtain target information.

10. The two-dimensional code image recognition method according to claim 9, characterized in that: The step of identifying the two-dimensional code repair image to obtain target information includes: Performing corner point detection on the two-dimensional code repair image to obtain target corner point positions, and extracting a target two-dimensional code pattern from the two-dimensional code repair image according to the target corner point positions; The target two-dimensional code pattern is decoded according to a preset coding rule to obtain target information.

11. A two-dimensional code image repair device, characterized in that: include: An image annotation module is configured to annotate a target damaged area of ​​the two-dimensional code image to obtain a two-dimensional code annotated image; The two-dimensional code repair module is configured to input the two-dimensional code annotated image into the trained image repair model to obtain a two-dimensional code repair image.

12. A two-dimensional code image recognition device, characterized in that: include: An acquisition module configured to acquire a QR code image; An image repair module, configured to repair the two-dimensional code image using the two-dimensional code image repair method according to any one of claims 1 to 8 to obtain a two-dimensional code repair image; The two-dimensional code recognition module is configured to recognize the two-dimensional code repair image to obtain target information.

13. A chip, comprising: one or more processors; A storage device configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the two-dimensional code image repair method described in any one of claims 1-8, or the two-dimensional code image recognition method described in any one of claims 9-10.

14. An electronic device comprising the chip according to claim 13.

15. A non-volatile storage medium storing computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are configured to execute the two-dimensional code image repair method described in any one of claims 1-8, or the two-dimensional code image recognition method described in any one of claims 9-10.

Images