A two-dimensional code repairing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a two-dimensional code repairing method and device, electronic equipment and storage medium, and relate to the technical field of image processing. The two-dimensional code repairing method comprises: establishing an initial two-dimensional code repairing model according to an encoder-decoder structure, and training the initial two-dimensional code repairing model to obtain a two-dimensional code repairing model; and performing repairing processing on an obtained two-dimensional code image based on the two-dimensional code repairing model to obtain a target two-dimensional code image. The embodiments of the present application can achieve the technical effect of efficiently repairing various defective two-dimensional codes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a two-dimensional code repairing method and device, an electronic device and a storage medium. BACKGROUND

[0002] Two-dimensional code technology is widely used in various aspects of production and life. In actual application scenarios, two-dimensional codes are easily affected by various natural forces and human factors, resulting in defects such as damage, dirt and blur, which makes it impossible to read the defective two-dimensional code.

[0003] At present, three two-dimensional code repairing methods are mainly used to repair defective two-dimensional codes. The first two-dimensional code repairing method adds error correction codes to the two-dimensional code. For defective two-dimensional codes, the error correction code is used as redundant information for decoding. The first two-dimensional code repairing method is only suitable for repairing two-dimensional codes with small defect areas and is not suitable for repairing two-dimensional codes with large defect areas that make it impossible to identify error correction codes. The second two-dimensional code repairing method uses filtering and other digital image processing techniques to process the image of the defective two-dimensional code to obtain a clear two-dimensional code. The repair effect of the second two-dimensional code repairing method depends on the image quality. When the image quality of the defective two-dimensional code is poor, the second two-dimensional code repairing method is likely to fail to repair the defective two-dimensional code. The third two-dimensional code repairing method is based on a generative adversarial neural network to repair defective two-dimensional codes. The third two-dimensional code repairing method requires the use of two neural networks at the same time, which is complex and costly to process, and can only repair defective two-dimensional codes to visually clear two-dimensional codes, and cannot guarantee that the repaired two-dimensional code is consistent with the original two-dimensional code. It can be seen that the existing two-dimensional code repairing methods still cannot efficiently repair various defective two-dimensional codes. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a two-dimensional code repairing method, device, electronic device and storage medium to achieve the technical effect of efficiently repairing various defective two-dimensional codes.

[0005] In a first aspect, the embodiments of the present application provide a two-dimensional code repairing method, comprising:

[0006] An initial two-dimensional code repairing model is established according to an encoder-decoder structure, and the initial two-dimensional code repairing model is trained to obtain a two-dimensional code repairing model;

[0007] Based on the two-dimensional code repairing model, the obtained two-dimensional code image is repaired to obtain a target two-dimensional code image.

[0008] In the above implementation process, the initial two-dimensional code repairing model established by training in the encoder-decoder structure is trained, and the two-dimensional code image is repaired based on the trained two-dimensional code repairing model to obtain a target two-dimensional code image, which can efficiently repair various defective two-dimensional codes.

[0009] Further, the initial two-dimensional code repair model is trained to obtain a two-dimensional code repair model, specifically comprising:

[0010] An analog data set is obtained; wherein the analog data set comprises at least one normal two-dimensional code image and at least one defective two-dimensional code image;

[0011] The normal two-dimensional code image is input into the initial two-dimensional code repair model, and the feature of the normal two-dimensional code image is extracted through an auxiliary encoder network;

[0012] The defective two-dimensional code image is input into the initial two-dimensional code repair model, the feature of the defective two-dimensional code image is extracted through an encoder network, a two-dimensional code is restored according to the feature of the defective two-dimensional code image through a decoder network, a repaired two-dimensional code image is obtained, and the feature of the defective two-dimensional code image and the feature of the repaired two-dimensional code image are fused for learning through an attention network; wherein the initial two-dimensional code repair model comprises the auxiliary encoder network, the encoder network, the decoder network and the attention network;

[0013] The network loss of the initial two-dimensional code repair model is determined in combination with the feature of the normal two-dimensional code image, the feature of the defective two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image;

[0014] The parameters of the encoder network, the decoder network and the attention network are updated according to the network loss, and the two-dimensional code repair model is established in combination with the updated encoder network, the updated decoder network and the updated attention network.

[0015] In the above implementation process, the initial two-dimensional code repair model is established by combining the auxiliary encoder network, the encoder network, the decoder network and the attention network, the encoder network, the decoder network and the attention network are updated by using the analog data set for training, and the two-dimensional code repair model is established only by combining the updated encoder network, the updated decoder network and the updated attention network, which can further efficiently repair various defective two-dimensional codes.

[0016] Further, the analog data set is obtained, specifically comprising:

[0017] At least one normal two-dimensional code image is generated based on an open source algorithm library;

[0018] For each normal two-dimensional code image, a defect is simulated on the normal two-dimensional code image according to a predefined two-dimensional code defect type to obtain at least one defective two-dimensional code image.

[0019] In the implementation process, at least one normal two-dimensional code image is generated based on an open source algorithm library, for each normal two-dimensional code image, a pre-defined any type of defect is simulated on the normal two-dimensional code image to obtain at least one defect two-dimensional code image, an initial two-dimensional code repair model is quickly trained by using a large number of defect two-dimensional code images of different defect types, the effectiveness and generalization of model training are ensured, and various defect two-dimensional codes are further efficiently repaired.

[0020] Further, the network loss of the initial two-dimensional code repair model is determined by combining the features of the normal two-dimensional code image, the features of the defect two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image, and specifically includes:

[0021] According to the features of the normal two-dimensional code image and the features of the defect two-dimensional code image, a feature difference degree between the defect two-dimensional code image and the normal two-dimensional code image is calculated to obtain a first network loss;

[0022] According to the normal two-dimensional code image and the repaired two-dimensional code image, a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the defect area is calculated to obtain a second network loss;

[0023] According to the normal two-dimensional code image and the repaired two-dimensional code image, a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the two-dimensional code functional area is calculated to obtain a third network loss;

[0024] According to the normal two-dimensional code image and the repaired two-dimensional code image, a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the information encoding area is calculated to obtain a fourth network loss.

[0025] In the implementation process, the feature difference degree between the defect two-dimensional code image and the normal two-dimensional code image is taken as the first network loss, and the difference degrees between the repaired two-dimensional code image and the normal two-dimensional code image in the defect area, the two-dimensional code functional area and the information encoding area are taken as the second network loss, the third network loss and the fourth network loss, so that the repair effect of different areas of the two-dimensional code can be focused on during the model training process, not only the defect area of the repaired two-dimensional code is ensured, but also the non-defect area of the two-dimensional code is prevented from being tampered, the repaired two-dimensional code is consistent with the original two-dimensional code, and various defect two-dimensional codes are further efficiently repaired.

[0026] Further, the first network loss is:

[0027] Or,

[0028]

[0029] L1=∑i=1n(xi-yi)21loss the first network loss, gt k the k-th dimensional feature value on the feature map corresponding to the normal two-dimensional code image, pre k the k-th dimensional feature value on the feature map corresponding to the defective two-dimensional code image, k = 1, 2, …, K, K is the number of feature dimensions of the feature map;

[0030] The second network loss is:

[0031] Or,

[0032]

[0033] Wherein, L 2loss the second network loss, gt l the l-th pixel value on the simulated defective area in the normal two-dimensional code image, pre l the l-th pixel value on the defective area in the repaired two-dimensional code image, l = 1, 2, …, L, L is the number of pixels on the image defective area;

[0034] The third network loss is:

[0035] Or,

[0036]

[0037] Wherein, L 3loss the third network loss, gt m the m-th pixel value on the two-dimensional code functional area in the normal two-dimensional code image, pre m the m-th pixel value on the two-dimensional code functional area in the repaired two-dimensional code image, m = 1, 2, …, M, M is the number of pixels on the image two-dimensional code functional area;

[0038] The fourth network loss is:

[0039] Or,

[0040]

[0041] Wherein, L 4loss the fourth network loss, gt n the n-th pixel value on the information encoding area in the normal two-dimensional code image, pre n the n-th pixel value on the information encoding area in the repaired two-dimensional code image, n = 1, 2, …, N, N is the number of pixels on the image information encoding area.

[0042] In the implementation process, the four network losses of the initial two-dimensional code repair model are quickly determined by calculating the first network loss, the second network loss, the third network loss and the fourth network loss according to the calculation method, and various defective two-dimensional codes are further efficiently repaired.

[0043] Further, the updating the parameters of the encoder network, the decoder network and the attention network according to the network losses specifically includes:

[0044] The parameters of the encoder network, the decoder network and the attention network are updated by back propagation after weighted sum of the first network loss, the second network loss, the third network loss and the fourth network loss;

[0045] The training operation is repeatedly performed until the latest first network loss, the latest second network loss, the latest third network loss and the latest fourth network loss all reach the corresponding preset threshold, and the updated encoder network, the updated decoder network and the updated attention network are obtained.

[0046] In the implementation process, the parameters of the encoder network, the decoder network and the attention network are updated by combining the first network loss, the second network loss, the third network loss and the fourth network loss, which can ensure that the two-dimensional code repair model quickly and accurately repairs the defective area and the non-defective area of the two-dimensional code, and further efficiently repairs various defective two-dimensional codes.

[0047] Further, before the two-dimensional code image obtained is repaired based on the two-dimensional code repair model to obtain a target two-dimensional code image, the method further includes:

[0048] An object detection technology is used to locate the defective two-dimensional code according to the initial two-dimensional code image to obtain a positioning position of the defective two-dimensional code.

[0049] The region image in the initial two-dimensional code image is intercepted as the two-dimensional code image according to the positioning position of the defective two-dimensional code.

[0050] In the implementation process, the defective two-dimensional code in the initial two-dimensional code image is located by using the object detection technology, and the image of the region where the defective two-dimensional code is located in the initial two-dimensional code image is intercepted as the two-dimensional code image, so that the two-dimensional code repair model can accurately repair the defective two-dimensional code according to the two-dimensional code image, and further efficiently repair various defective two-dimensional codes.

[0051] In a second aspect, an embodiment of the present application provides a two-dimensional code repair device, which includes:

[0052] The model establishing module is configured to establish an initial two-dimensional code repairing model according to an encoder-decoder structure, and train the initial two-dimensional code repairing model to obtain a two-dimensional code repairing model.

[0053] The two-dimensional code repairing module is configured to perform repairing processing on the obtained two-dimensional code image based on the two-dimensional code repairing model to obtain a target two-dimensional code image.

[0054] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor, and the processor implements the two-dimensional code repairing method as described above when executing the computer program.

[0055] In a fourth aspect, a computer readable storage medium is provided, which includes a stored computer program; when the computer program is executed, the computer readable storage medium controls a device where the computer readable storage medium is located to perform the two-dimensional code repairing method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0057] Figure 1 A flowchart of a two-dimensional code repairing method provided by the first embodiment of the present application;

[0058] Figure 2 A data flow diagram of training an initial two-dimensional code repairing model for an optional embodiment example of the first embodiment of the present application;

[0059] Figure 3 A data flow diagram of applying a two-dimensional code repairing model for an optional embodiment example of the first embodiment of the present application;

[0060] Figure 4 A structure diagram of a two-dimensional code repairing device provided by the second embodiment of the present application;

[0061] Figure 5 A structure diagram of an electronic device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions of the embodiments of the present application will be described below in combination with the drawings in the embodiments of the present application.

[0063] It should be noted that in the description of the present application, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. At the same time, the step number in the text is only for the convenience of explaining the embodiment of the present application, and does not serve as the function of limiting the execution sequence of the steps. The method provided by the embodiment of the present application can be executed by the related terminal device, and the processor is taken as an example for explanation in the following.

[0064] Please refer to Figure 1 , Figure 1 A flowchart of a two-dimensional code repairing method provided by the first embodiment of the present application. The first embodiment of the present application provides a two-dimensional code repairing method, including steps S101-S102:

[0065] S101, an initial two-dimensional code repairing model is established according to the encoder-decoder structure, and the initial two-dimensional code repairing model is trained to obtain a two-dimensional code repairing model;

[0066] S102, based on the two-dimensional code repairing model, the obtained two-dimensional code image is repaired to obtain a target two-dimensional code image.

[0067] As an example, according to the actual application requirement, the number of encoders and decoders is configured, and the encoder-decoder structure is designed. According to the encoder-decoder structure, the initial two-dimensional code repairing model is established, and the initial two-dimensional code repairing model is trained, so that the initial two-dimensional code repairing model automatically and comprehensively learns the characteristics of the defective two-dimensional code, and the two-dimensional code repairing model is obtained.

[0068] Among them, the network structure of the initial two-dimensional code repairing model adopts modular design, the structure is simple and clear, allows to adapt to various application requirements by increasing or decreasing the number of encoders and decoders, without the need to redesign the network structure, and can quickly train the two-dimensional code repairing model to repair various defective two-dimensional codes.

[0069] The two-dimensional code image is obtained, the two-dimensional code image is input into the two-dimensional code repairing model, and the two-dimensional code image is repaired based on the two-dimensional code repairing model to obtain a target two-dimensional code image.

[0070] The embodiment of the present application trains the initial two-dimensional code repairing model established by the encoder-decoder structure, repairs the two-dimensional code image based on the trained two-dimensional code repairing model to obtain a target two-dimensional code image, and can efficiently repair various defective two-dimensional codes.

[0071] In an optional embodiment, the training of the initial two-dimensional code repair model to obtain a two-dimensional code repair model specifically comprises: obtaining a simulation data set; wherein the simulation data set comprises at least one normal two-dimensional code image and at least one defective two-dimensional code image; inputting the normal two-dimensional code image into the initial two-dimensional code repair model, and extracting features of the normal two-dimensional code image through an auxiliary encoder network; inputting the defective two-dimensional code image into the initial two-dimensional code repair model, extracting features of the defective two-dimensional code image through an encoder network, restoring a two-dimensional code according to the features of the defective two-dimensional code image through a decoder network to obtain a repaired two-dimensional code image, and learning by fusing the features of the defective two-dimensional code image and the features of the repaired two-dimensional code image through an attention network; wherein the initial two-dimensional code repair model comprises the auxiliary encoder network, the encoder network, the decoder network and the attention network; combining the features of the normal two-dimensional code image, the features of the defective two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image to determine a network loss of the initial two-dimensional code repair model; updating parameters of the encoder network, the decoder network and the attention network according to the network loss, and combining the updated encoder network, the updated decoder network and the updated attention network to establish the two-dimensional code repair model.

[0072] As an example, the simulation data set is obtained, wherein the simulation data set comprises at least one normal two-dimensional code image and at least one defective two-dimensional code image. The normal two-dimensional code image is an image of a two-dimensional code without defects, and the defective two-dimensional code image is an image of a two-dimensional code with at least one defect.

[0073] The normal two-dimensional code image in the simulation data set is input into the initial two-dimensional code repair model, and features of the normal two-dimensional code image are extracted through an auxiliary encoder network in the initial two-dimensional code repair model.

[0074] And the defective two-dimensional code image in the simulation data set is input into the initial two-dimensional code repair model, features of the defective two-dimensional code image are extracted through an encoder network in the initial two-dimensional code repair model; a two-dimensional code is restored according to the features of the defective two-dimensional code image through a decoder network in the initial two-dimensional code repair model to obtain a repaired two-dimensional code image; and learning is performed by fusing the features of the defective two-dimensional code image and the features of the repaired two-dimensional code image through an attention network in the initial two-dimensional code repair model, so as to achieve a better two-dimensional code repair effect.

[0075] The auxiliary encoder network and the encoder network have the same structure and parameters. The encoder network includes x (x is greater than or equal to 1) levels of encoders, each level of encoder is used for extracting image features of different scales, and each level of encoder includes a convolution layer, a pooling layer, an activation layer, a down-sampling layer and the like. The decoder network includes x levels of decoders, each level of decoder is used for restoring an image according to image features of different scales, and each level of decoder includes a convolution layer, a pooling layer, an up-sampling layer and the like. The attention network includes x levels of attention units, each level of attention unit is used for fusing image features of different scales generated by different encoders and image features of different scales restored by corresponding decoders, and each level of attention unit includes a 1x1 convolution layer, an activation function and the like.

[0076] The network loss of the initial two-dimensional code restoration model is determined in combination with the features of the normal two-dimensional code image, the features of the defective two-dimensional code image, the normal two-dimensional code image and the restored two-dimensional code image.

[0077] The parameters of the auxiliary encoder network, the encoder network, the decoder network and the attention network are updated according to the network loss, to obtain an updated auxiliary encoder network, an updated encoder network, an updated decoder network and an updated attention network.

[0078] The updated auxiliary encoder network is deleted, and a two-dimensional code restoration model is established in combination with the updated encoder network, the updated decoder network and the updated attention network.

[0079] In the model training stage, the auxiliary encoder network is added, the initial two-dimensional code restoration model can be trained and updated by considering the features of the normal two-dimensional code image, and the training effect of the initial two-dimensional code restoration model is optimized. In the model using stage, the auxiliary encoder network is deleted, the network structure of the two-dimensional code restoration model is more lightweight, and the processing speed of the two-dimensional code restoration model is improved.

[0080] The initial two-dimensional code restoration model is established by combining the auxiliary encoder network, the encoder network, the decoder network and the attention network, the encoder network, the decoder network and the attention network are trained and updated by using the simulation data set, and the two-dimensional code restoration model is established only by combining the updated encoder network, the updated decoder network and the updated attention network, so that various defective two-dimensional codes can be further efficiently restored.

[0081] In optional embodiments, the simulation data set is obtained, specifically including: generating at least one normal two-dimensional code image based on an open source algorithm library; for each normal two-dimensional code image, simulating defects on the normal two-dimensional code image according to a predefined two-dimensional code defect type to obtain at least one defective two-dimensional code image.

[0082] As an example, based on an open source algorithm library, at least one normal two-dimensional code image is generated, and the generated normal two-dimensional code image is taken as a label of a simulation data set.

[0083] According to actual application requirements, a two-dimensional code defect type is predefined. For each normal two-dimensional code image, a defect is simulated on the normal two-dimensional code image according to the predefined two-dimensional code defect type, and at least one defective two-dimensional code image is obtained.

[0084] For example, when the predefined two-dimensional code defect type includes defects such as scratches and breakage, random pixel erasing, grayscale covering and the like can be performed on a region where a two-dimensional code in the normal two-dimensional code image is located to simulate such defects, and at least one defective two-dimensional code image is obtained; when the predefined two-dimensional code defect type includes defects such as uneven illumination and dark light environment, random hue offset, random brightness and the like can be performed on the region where the two-dimensional code in the normal two-dimensional code image is located to simulate such defects, and at least one defective two-dimensional code image is obtained; when the predefined two-dimensional code defect type includes defects such as deformation and bending, affine transformation, cropping and splicing and the like can be performed on the region where the two-dimensional code in the normal two-dimensional code image is located to simulate such defects, and at least one defective two-dimensional code image is obtained; when the predefined two-dimensional code defect type includes defects such as imaging blur and wear, Gaussian blur, salt and pepper noise and the like can be added to the region where the two-dimensional code in the normal two-dimensional code image is located to simulate such defects, and at least one defective two-dimensional code image is obtained. These defect simulation methods can be used alone or in any combination to improve the diversity of the simulation data set.

[0085] The embodiment of the present application can quickly obtain a large number of defective two-dimensional code images of different defect types by generating at least one normal two-dimensional code image based on an open source algorithm library, simulating any type of defect predefined on each normal two-dimensional code image to obtain at least one defective two-dimensional code image, training an initial two-dimensional code repair model, ensuring the effectiveness and generality of model training, and further efficiently repairing various defective two-dimensional codes.

[0086] In the optional embodiment, the network loss of the initial two-dimensional code repair model is determined by combining the features of the normal two-dimensional code image, the features of the defective two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image, and specifically includes: calculating the feature difference degree between the defective two-dimensional code image and the normal two-dimensional code image according to the features of the normal two-dimensional code image and the features of the defective two-dimensional code image to obtain a first network loss; calculating the difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the defective area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a second network loss; calculating the difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the two-dimensional code functional area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a third network loss; and calculating the difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in the information encoding area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a fourth network loss.

[0087] For example, when the features of the normal two-dimensional code image and the features of the defective two-dimensional code image are obtained, the feature difference degree between the defective two-dimensional code image and the normal two-dimensional code image is calculated according to the features of the normal two-dimensional code image and the features of the defective two-dimensional code image to obtain a first network loss.

[0088] When the repaired two-dimensional code image is obtained, the difference degrees between the repaired two-dimensional code image and the normal two-dimensional code image in the defective area, the two-dimensional code functional area and the information encoding area are calculated respectively according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a second network loss, a third network loss and a fourth network loss.

[0089] The defective area is a defective area in the area where the two-dimensional code is located in the image, the two-dimensional code functional area is an area containing version information, timing patterns, alignment patterns, separators and other functional information of the two-dimensional code in the area where the two-dimensional code is located in the image, and the information encoding area is the remaining area in the area where the two-dimensional code is located in the image except the defective area and the two-dimensional code functional area.

[0090] By taking the feature difference degree between the defective two-dimensional code image and the normal two-dimensional code image as the first network loss, and taking the difference degrees between the repaired two-dimensional code image and the normal two-dimensional code image in the defective area, the two-dimensional code functional area and the information encoding area as the second network loss, the third network loss and the fourth network loss, the repair effects of different areas of the two-dimensional code can be focused on in the model training process in the embodiment of the application, the defective area of the repaired two-dimensional code is guaranteed, the non-defective area of the two-dimensional code is avoided from being tampered, the repaired two-dimensional code is consistent with the original two-dimensional code, and various defective two-dimensional codes can be repaired efficiently.

[0091] In the optional embodiment, the first network loss is:

[0092] Alternatively,

[0093]

[0094] wherein, L 1loss is the first network loss, gt k is the k-th dimensional feature value on the feature map corresponding to the normal QR code image, pre k is the k-th dimensional feature value on the feature map corresponding to the defect QR code image, k = 1, 2, …, K, K is the number of feature dimensions of the feature map;

[0095] The second network loss is:

[0096] or,

[0097]

[0098] wherein, L 2loss is the second network loss, gt l is the l-th pixel value on the simulated defect region in the normal QR code image, pre l is the l-th pixel value on the defect region in the repaired QR code image, l = 1, 2, …, L, L is the number of pixels on the defect region of the image;

[0099] The third network loss is:

[0100] or,

[0101]

[0102] wherein, L 3loss is the third network loss, gt m is the m-th pixel value on the QR code functional region in the normal QR code image, pre m is the m-th pixel value on the QR code functional region in the repaired QR code image, m = 1, 2, …, M, M is the number of pixels on the QR code functional region of the image;

[0103] The fourth network loss is:

[0104] or,

[0105]

[0106] wherein, L 4loss is the fourth network loss, gt n is the n-th pixel value on the information encoding region in the normal QR code image, pre nTo repair the n-th pixel value on the information encoding area in the two-dimensional code image, n = 1, 2,..., N, N is the number of pixels on the information encoding area of the image.

[0107] As an example, according to the characteristics of the normal two-dimensional code image, the feature map corresponding to the normal two-dimensional code image is generated, according to the characteristics of the defective two-dimensional code image, the feature map corresponding to the defective two-dimensional code image is generated, based on the feature map corresponding to the normal two-dimensional code image and the feature map corresponding to the defective two-dimensional code image, the feature difference between the defective two-dimensional code image and the normal two-dimensional code image is calculated, and the first network loss is obtained, the first network loss is:

[0108] Or,

[0109]

[0110] In formula (1), (2), L 1loss The first network loss is gt k The k-th dimensional feature value on the feature map corresponding to the normal two-dimensional code image is pre k The k-th dimensional feature value on the feature map corresponding to the defective two-dimensional code image is pre

[0111] According to the normal two-dimensional code image, the pixel value of the simulated defect area in the normal two-dimensional code image is determined, according to the repaired two-dimensional code image, the pixel value of the defect area in the repaired two-dimensional code image is determined, and the difference between the repaired two-dimensional code image and the normal two-dimensional code image in the defect area is calculated, and the second network loss is obtained, the second network loss is:

[0112] Or,

[0113]

[0114] In formula (3), (4), L 2loss The second network loss is gt l The l-th pixel value on the simulated defect area in the normal two-dimensional code image is pre l The l-th pixel value on the defect area in the repaired two-dimensional code image is pre

[0115] According to the normal two-dimensional code image, the pixel value of the two-dimensional code function area in the normal two-dimensional code image is determined, according to the repaired two-dimensional code image, the pixel value of the two-dimensional code function area in the repaired two-dimensional code image is determined, the difference degree between the repaired two-dimensional code image and the normal two-dimensional code image on the two-dimensional code function area is calculated by combining the pixel value of the two-dimensional code function area in the normal two-dimensional code image and the pixel value of the two-dimensional code function area in the repaired two-dimensional code image, and a third network loss is obtained, and the third network loss is:

[0116] Or,

[0117]

[0118] In formula (5), (6), L 3loss The third network loss is gt m The mth pixel value on the two-dimensional code function area in the normal two-dimensional code image is pre m The mth pixel value on the two-dimensional code function area in the repaired two-dimensional code image is pre

[0119] According to the normal two-dimensional code image, the pixel value of the two-dimensional code function area in the normal two-dimensional code image is determined, according to the repaired two-dimensional code image, the pixel value of the two-dimensional code function area in the repaired two-dimensional code image is determined, the difference degree between the repaired two-dimensional code image and the normal two-dimensional code image on the two-dimensional code function area is calculated by combining the pixel value of the two-dimensional code function area in the normal two-dimensional code image and the pixel value of the two-dimensional code function area in the repaired two-dimensional code image, and a third network loss is obtained, and the third network loss is:

[0120] Or,

[0121]

[0122] In formula (7), (8), L 4loss The fourth network loss is gt n The nth pixel value on the information coding area in the normal two-dimensional code image is pre n The nth pixel value on the information coding area in the repaired two-dimensional code image is pre

[0123] The first network loss, the second network loss, the third network loss and the fourth network loss are calculated according to the above calculation method, so that the four network losses of the initial two-dimensional code repair model can be quickly determined, and various defective two-dimensional codes can be further efficiently repaired.

[0124] In an optional embodiment, the updating the parameters of the encoder network, the decoder network and the attention network according to the network losses comprises: performing back propagation on the weighted sum of the first network loss, the second network loss, the third network loss and the fourth network loss to update the parameters of the encoder network, the decoder network and the attention network; and repeating the training operation until the latest first network loss, the latest second network loss, the latest third network loss and the latest fourth network loss all reach the corresponding preset threshold, to obtain the updated encoder network, the updated decoder network and the updated attention network.

[0125] For example, the weights of the first network loss, the second network loss, the third network loss and the fourth network loss are pre-allocated according to actual application requirements.

[0126] The first network loss, the second network loss, the third network loss and the fourth network loss are weighted and summed according to the weights of the first network loss, the second network loss, the third network loss and the fourth network loss, and then back propagation is performed to update the parameters of the auxiliary encoder network, the encoder network, the decoder network and the attention network.

[0127] After updating the encoder network, the decoder network and the attention network each time, it is determined whether the current first network loss, the current second network loss, the current third network loss and the current fourth network loss all reach the corresponding preset values, if yes, a two-dimensional code repair model is established combining the updated encoder network, the updated decoder network and the updated attention network, and if not, the training operation is performed again, that is, the simulated data set is input into the updated initial two-dimensional code repair model for training and updating, until the latest first network loss, the latest second network loss, the latest third network loss and the latest fourth network loss all reach the corresponding preset threshold, to obtain the updated encoder network, the updated decoder network and the updated attention network.

[0128] For example, the data flow graph of the training initial two-dimensional code repair model is as shown in Figure 2

[0129] The embodiment of the application can ensure that the two-dimensional code repair model quickly and accurately repairs the defect area and the non-defect area of the two-dimensional code, and further efficiently repairs various defective two-dimensional codes, by updating the parameters of the encoder network, the decoder network and the attention network according to the first network loss, the second network loss, the third network loss and the fourth network loss.

[0130] ​In an optional embodiment, before the based on the two-dimensional code repair model, the obtained two-dimensional code image is repaired to obtain the target two-dimensional code image, further comprising: using target detection technology, positioning the defective two-dimensional code according to the initial two-dimensional code image to obtain the positioning position of the defective two-dimensional code; according to the positioning position of the defective two-dimensional code, the region image in the initial two-dimensional code image is intercepted as the two-dimensional code image.

[0131] As an example, the initial two-dimensional code image is collected by a camera, a code scanning gun and the like, the target detection technology is used to locate the defective two-dimensional code in the initial two-dimensional code image to obtain the positioning position of the defective two-dimensional code, and the region image in the initial two-dimensional code image is intercepted as the two-dimensional code image according to the positioning position of the defective two-dimensional code.

[0132] The target detection technology can be two-dimensional code detection by using image binarization, edge detection, histogram statistics and the like, or two-dimensional code detection based on a YOLO, SSD and the like neural network model.

[0133] The two-dimensional code image is input into the two-dimensional code repair model, the features of the two-dimensional code image are extracted through the encoder network in the two-dimensional code repair model, the two-dimensional code is restored according to the features of the two-dimensional code image through the decoder network in the two-dimensional code repair model to obtain the target two-dimensional code image, and the features of the two-dimensional code image and the features of the target two-dimensional code image are fused through the attention network in the two-dimensional code repair model for learning to achieve better two-dimensional code repair effect.

[0134] For example, the data flow diagram of the two-dimensional code repair model is as shown in Figure 3 .

[0135] The embodiment of the application can make the two-dimensional code repair model accurately repair the defective two-dimensional code according to the two-dimensional code image, and further efficiently repair various defective two-dimensional codes by using the target detection technology to locate the defective two-dimensional code in the initial two-dimensional code image and intercepting the image of the region where the defective two-dimensional code is located in the initial two-dimensional code image as the two-dimensional code image.

[0136] Please refer to Figure 4 , Figure 4 The structure diagram of a two-dimensional code repair device provided by the second embodiment of the application. The second embodiment of the application provides a two-dimensional code repair device, which comprises: a model establishing module 201, configured to establish an initial two-dimensional code repair model according to an encoder-decoder structure, and train the initial two-dimensional code repair model to obtain a two-dimensional code repair model; and a two-dimensional code repair module 202, configured to repair an obtained two-dimensional code image based on the two-dimensional code repair model to obtain a target two-dimensional code image.

[0137] In optional embodiments, the training the initial two-dimensional code repair model to obtain a two-dimensional code repair model specifically comprises: obtaining a simulation data set; wherein the simulation data set comprises at least one normal two-dimensional code image and at least one defective two-dimensional code image; inputting the normal two-dimensional code image into the initial two-dimensional code repair model to extract features of the normal two-dimensional code image through an auxiliary encoder network; inputting the defective two-dimensional code image into the initial two-dimensional code repair model to extract features of the defective two-dimensional code image through an encoder network, restoring a two-dimensional code from the features of the defective two-dimensional code image through a decoder network to obtain a repaired two-dimensional code image, and learning through an attention network by fusing the features of the defective two-dimensional code image and the features of the repaired two-dimensional code image; wherein the initial two-dimensional code repair model comprises the auxiliary encoder network, the encoder network, the decoder network and the attention network; determining a network loss of the initial two-dimensional code repair model in combination with the features of the normal two-dimensional code image, the features of the defective two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image; updating parameters of the encoder network, the decoder network and the attention network according to the network loss, and establishing the two-dimensional code repair model in combination with the updated encoder network, the updated decoder network and the updated attention network.

[0138] In optional embodiments, the obtaining the simulation data set specifically comprises: generating at least one normal two-dimensional code image based on an open source algorithm library; for each normal two-dimensional code image, simulating a defect on the normal two-dimensional code image according to a predefined two-dimensional code defect type to obtain at least one defective two-dimensional code image.

[0139] In optional embodiments, the determining the network loss of the initial two-dimensional code repair model in combination with the features of the normal two-dimensional code image, the features of the defective two-dimensional code image, the normal two-dimensional code image and the repaired two-dimensional code image specifically comprises: calculating a feature difference degree between the defective two-dimensional code image and the normal two-dimensional code image according to the features of the normal two-dimensional code image and the features of the defective two-dimensional code image to obtain a first network loss; calculating a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in a defective area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a second network loss; calculating a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in a two-dimensional code functional area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a third network loss; and calculating a difference degree between the repaired two-dimensional code image and the normal two-dimensional code image in an information encoding area according to the normal two-dimensional code image and the repaired two-dimensional code image to obtain a fourth network loss.

[0140] In optional embodiments, the first network loss is:

[0141] or,

[0142]

[0143] wherein, L 1loss is the first network loss, gt k is the k-th dimensional feature value on the feature map corresponding to the normal QR code image, pre k is the k-th dimensional feature value on the feature map corresponding to the defect QR code image, k = 1, 2, …, K, K is the number of feature dimensions of the feature map;

[0144] The second network loss is:

[0145] or,

[0146]

[0147] wherein, L 2loss is the second network loss, gt l is the l-th pixel value on the simulated defect region in the normal QR code image, pre l is the l-th pixel value on the defect region in the repaired QR code image, l = 1, 2, …, L, L is the number of pixels on the defect region of the image;

[0148] The third network loss is:

[0149] or,

[0150]

[0151] wherein, L 3loss is the third network loss, gt m is the m-th pixel value on the QR code functional region in the normal QR code image, pre m is the m-th pixel value on the QR code functional region in the repaired QR code image, m = 1, 2, …, M, M is the number of pixels on the QR code functional region of the image;

[0152] The fourth network loss is:

[0153] or,

[0154]

[0155] wherein, L 4loss is the fourth network loss, gt n is the n-th pixel value on the information encoding region in the normal QR code image, pre n is the n-th pixel value on the information encoding region in the repaired QR code image, n = 1, 2, …, N, N is the number of pixels on the information encoding region of the image.

[0156] In an optional embodiment, the updating the parameters of the encoder network, the decoder network and the attention network according to the network losses comprises: performing back propagation on a weighted sum of the first network loss, the second network loss, the third network loss and the fourth network loss to update the parameters of the encoder network, the decoder network and the attention network; and repeating the training operation until the latest first network loss, the latest second network loss, the latest third network loss and the latest fourth network loss all reach the corresponding preset threshold, to obtain the updated encoder network, the updated decoder network and the updated attention network.

[0157] In an optional embodiment, the QR code repairing module 202 is further configured to, before the repairing of the obtained QR code image based on the QR code repairing model to obtain the target QR code image, locate the defective QR code according to the initial QR code image by using a target detection technology to obtain a positioning position of the defective QR code; and cut a region image in the initial QR code image as the QR code image according to the positioning position of the defective QR code.

[0158] The implementation process of the functions and roles of each module in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0159] Please refer to Figure 5 , Figure 5 A structural schematic diagram of an electronic device is provided for the third embodiment of the present application. The third embodiment of the present application provides an electronic device 30, which comprises a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301; the memory 302 is coupled to the processor 301, and the processor 301 implements the QR code repairing method as described in the first embodiment of the present application when executing the computer program, and can achieve the same beneficial effects.

[0160] The processor 301 can implement the method of any embodiment of the QR code repairing method as described in the first embodiment of the present application by reading the computer program from the memory 302 through the bus 303 and executing the computer program.

[0161] The processor 301 can process digital signals and can include various computing structures. For example, a complex instruction set computer structure, a reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, the processor 301 can be a microprocessor.

[0162] The memory 302 can be used to store instructions executed by the processor 301 or data related to the instructions during execution. The instructions and / or data can include code for implementing some or all of the functions of one or more modules described in the embodiments of the present application. The processor 301 of the embodiments of the present disclosure can be used to execute the instructions in the memory 302 to implement the two-dimensional code repairing method as described in the first embodiment of the present application. The memory 302 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memories well known to those skilled in the art.

[0163] The fourth embodiment of the present application provides a computer readable storage medium, which comprises a stored computer program; wherein the computer readable storage medium controls the device where the computer readable storage medium is located to perform the two-dimensional code repairing method as described in the first embodiment of the present application when the computer program runs, and the same beneficial effects can be achieved.

[0164] To sum up, the embodiments of the present application provide a two-dimensional code repairing method, device, electronic equipment and storage medium, the two-dimensional code repairing method comprises: establishing an initial two-dimensional code repairing model according to an encoder-decoder structure, and training the initial two-dimensional code repairing model to obtain a two-dimensional code repairing model; based on the two-dimensional code repairing model, performing repairing processing on the obtained two-dimensional code image to obtain a target two-dimensional code image. Through training the initial two-dimensional code repairing model established according to the encoder-decoder structure, the target two-dimensional code image is obtained by performing repairing processing on the two-dimensional code image based on the trained two-dimensional code repairing model, and various defective two-dimensional codes can be efficiently repaired.

[0165] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus embodiments described above are only schematic, for example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two consecutive blocks can actually be executed substantially in parallel, or they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system, or by a combination of special-purpose hardware and computer instructions.

[0166] In addition, each functional module in various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0167] If the functions are realized in the form of software functional modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A QR code repair method, characterized in that, include: An initial QR code restoration model is established according to the encoder-decoder structure, and the initial QR code restoration model is trained to obtain the QR code restoration model; The process of training the initial QR code restoration model to obtain the QR code restoration model specifically includes: Obtain a simulated dataset; wherein the simulated dataset includes at least one normal QR code image and at least one defective QR code image; The normal QR code image is input into the initial QR code repair model, and the features of the normal QR code image are extracted through the auxiliary encoder network. The defective QR code image is input into the initial QR code repair model. The features of the defective QR code image are extracted by the encoder network, and the QR code is restored by the decoder network based on the features of the defective QR code image to obtain the repaired QR code image. The features of the defective QR code image and the features of the repaired QR code image are fused by the attention network for learning. The initial QR code repair model includes the auxiliary encoder network, the encoder network, the decoder network, and the attention network. By combining the features of the normal QR code image, the features of the defective QR code image, the normal QR code image, and the repaired QR code image, the network loss of the initial QR code repair model is determined. The parameters of the encoder network, the decoder network, and the attention network are updated based on the network loss. The updated encoder network, the updated decoder network, and the updated attention network are then combined to establish the QR code repair model. Based on the QR code repair model, the acquired QR code image is repaired to obtain the target QR code image.

2. The QR code repair method according to claim 1, characterized in that, The acquisition of the simulated dataset specifically includes: Generate at least one normal QR code image based on an open-source algorithm library; For each normal QR code image, a defect is simulated on the normal QR code image according to a predefined QR code defect type to obtain at least one defective QR code image.

3. The QR code repair method according to claim 1, characterized in that, The process of combining the features of the normal QR code image, the features of the defective QR code image, the normal QR code image, and the repaired QR code image to determine the network loss of the initial QR code repair model specifically includes: Based on the features of the normal QR code image and the features of the defective QR code image, the feature difference degree between the defective QR code image and the normal QR code image is calculated to obtain the first network loss. Based on the normal QR code image and the repaired QR code image, the difference between the repaired QR code image and the normal QR code image in the defect area is calculated to obtain the second network loss; Based on the normal QR code image and the repaired QR code image, the difference between the repaired QR code image and the normal QR code image in the QR code functional area is calculated to obtain the third network loss. Based on the normal QR code image and the repaired QR code image, the difference between the repaired QR code image and the normal QR code image in the information encoding region is calculated to obtain the fourth network loss.

4. The QR code repair method according to claim 3, characterized in that, The first network loss is: ;or, ； in, For the first network loss, The k-th dimension feature value on the feature map corresponding to the normal QR code image. The k-th dimension feature value on the feature map corresponding to the defective QR code image. K is the number of feature dimensions of the feature map; The second network loss is: ;or, ； in, This is the second network loss. The first simulated defect area in the normal QR code image l pixel value, For the defective region in the repaired QR code image, the first l pixel value, L is the number of pixels in the defective region of the image; The third network loss is: ;or, ； in, For the third network loss, This refers to the value of the m-th pixel in the functional area of ​​the QR code in the normal QR code image. The value of the m-th pixel in the functional area of ​​the QR code in the repaired QR code image. M is the number of pixels in the QR code functional area of ​​the image; The fourth network loss is: ;or, ； in, For the fourth network loss, The value of the nth pixel in the information encoding region of the normal QR code image. The value of the nth pixel in the information encoding region of the repaired QR code image. N is the number of pixels in the image information encoding region.

5. The QR code repair method according to claim 3, characterized in that, The step of updating the parameters of the encoder network, the decoder network, and the attention network based on the network loss specifically includes: The first network loss, the second network loss, the third network loss, and the fourth network loss are weighted and summed, and then backpropagated to update the parameters of the encoder network, the decoder network, and the attention network. The training operation is repeated until the latest first network loss, the latest second network loss, the latest third network loss, and the latest fourth network loss all reach the corresponding preset thresholds, resulting in the updated encoder network, the updated decoder network, and the updated attention network.

6. The QR code repair method according to claim 1, characterized in that, Before performing repair processing on the acquired QR code image based on the QR code repair model to obtain the target QR code image, the method further includes: Using target detection technology, the defective QR code is located based on the initial QR code image, and the location of the defective QR code is obtained. Based on the location of the defective QR code, a region image is cropped from the initial QR code image to serve as the QR code image.

7. A QR code repair device, characterized in that, include: The model building module is used to build an initial QR code restoration model according to the encoder-decoder structure, and train the initial QR code restoration model to obtain the QR code restoration model; The process of training the initial QR code restoration model to obtain the QR code restoration model specifically includes: Obtain a simulated dataset; wherein the simulated dataset includes at least one normal QR code image and at least one defective QR code image; The normal QR code image is input into the initial QR code repair model, and the features of the normal QR code image are extracted through the auxiliary encoder network. The defective QR code image is input into the initial QR code repair model. The features of the defective QR code image are extracted by the encoder network, and the QR code is restored by the decoder network based on the features of the defective QR code image to obtain the repaired QR code image. The features of the defective QR code image and the features of the repaired QR code image are fused by the attention network for learning. The initial QR code repair model includes the auxiliary encoder network, the encoder network, the decoder network, and the attention network. By combining the features of the normal QR code image, the features of the defective QR code image, the normal QR code image, and the repaired QR code image, the network loss of the initial QR code repair model is determined. The parameters of the encoder network, the decoder network, and the attention network are updated based on the network loss. The updated encoder network, the updated decoder network, and the updated attention network are then combined to establish the QR code repair model. The QR code repair module is used to repair the acquired QR code image based on the QR code repair model to obtain the target QR code image.

8. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor, and the processor, when executing the computer program, implements the QR code repair method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the QR code repair method according to any one of claims 1 to 6.

Images