Two-dimensional code recognition method, device, equipment and storage medium based on deep learning

Abstract

The embodiment of the present invention relates to the field of artificial intelligence, and discloses a two-dimensional code recognition method, device, equipment and storage medium based on deep learning, the method comprising: constructing a sample image set; adding annotations to one or more two-dimensional codes of each sample training image in the sample image set, and clustering each sample training image after adding annotations according to the position coordinates of the position area of ​​each two-dimensional code; inputting each clustered sample training image into a deep learning network model for training to obtain a two-dimensional code detection model; inputting an image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes; identifying one or more two-dimensional codes in the image to be tested according to the position information, realizing the recognition of one or more two-dimensional codes in the image, and improving the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image. The present invention relates to blockchain technology, such as being able to write images into blockchains for use in scenarios such as data forensics.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a two-dimensional code recognition method, device, equipment and storage medium based on deep learning. Background Art

[0002] QR code is a very popular encoding method on mobile devices in recent years. It can store more information and represent more data types than traditional barcodes.

[0003] At present, the industry often uses grayscale processing and binarization methods in the OpenCV method to identify QR codes, and locates the QR code by finding the characteristic area of ​​the QR code. However, this method usually has white edges, black edges, black and white edges, and patterns around the QR code, which will affect the recognition result of the QR code. Sometimes, the QR code cannot be recognized, let alone multiple QR codes. Therefore, how to recognize QR codes more effectively has become the focus of research. Summary of the invention

[0004] The embodiments of the present invention provide a deep learning-based two-dimensional code recognition method, apparatus, device and storage medium, which can realize the recognition of one or more two-dimensional codes in an image, thereby improving the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image.

[0005] In a first aspect, an embodiment of the present invention provides a two-dimensional code recognition method based on deep learning, comprising:

[0006] Constructing a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes;

[0007] adding annotations to one or more two-dimensional codes of each sample training image in the sample image set, and clustering each sample training image in the sample image set after the annotations are added according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes;

[0008] Input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model;

[0009] Inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested;

[0010] One or more two-dimensional codes in the image to be tested are identified according to the position information of the one or more two-dimensional codes in the image to be tested.

[0011] Furthermore, the step of adding annotations to one or more QR codes of each sample training image in the sample image set includes:

[0012] Determine the location area corresponding to each two-dimensional code in each sample training image and the area object name corresponding to each two-dimensional code;

[0013] Use a designated marking tool to add a first annotation to the position coordinates of the position area corresponding to each QR code in each sample training image, and add a second annotation to the area object name corresponding to each QR code in each sample training image, and store each sample training image after adding the first annotation and the second annotation in a designated file.

[0014] Furthermore, clustering the sample training images in the labeled sample image set according to the position coordinates of the position area corresponding to each of the one or more two-dimensional codes includes:

[0015] Determine the height and width of each sample training image according to the position coordinates of the position area corresponding to each QR code;

[0016] Clustering is performed on the sample training images in the labeled sample image set according to the height and width of the sample training images.

[0017] Furthermore, the step of determining the height and width of each sample training image according to the position coordinates of the position area corresponding to each two-dimensional code includes:

[0018] Determine the minimum position coordinates and the maximum position coordinates of the position areas corresponding to the respective two-dimensional codes according to the position coordinates of the position areas corresponding to the respective two-dimensional codes;

[0019] The minimum position coordinate and the maximum position coordinate are input into a specified clustering algorithm model to obtain the height and width of each sample training image.

[0020] Furthermore, before inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, the method further includes:

[0021] Acquire a video memory size, and adjust the size of each sample training image in the clustered sample image set according to the video memory size;

[0022] The method of inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model includes:

[0023] Input each resized sample training image into the preset deep learning network model to obtain the loss function value;

[0024] When the loss function value does not meet the preset conditions, adjusting the model parameters of the preset deep learning network model according to the loss function value, and inputting the resized sample training images into the deep learning network model after adjusting the model parameters for iterative training;

[0025] When the loss function value obtained after iterative training meets the preset condition, it is determined that the two-dimensional code detection model is obtained through training.

[0026] Furthermore, the step of inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested includes:

[0027] Input the image to be tested into the two-dimensional code detection model to obtain the predicted position label and predicted area object name of the two-dimensional code of the image to be tested;

[0028] The number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code are determined according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested.

[0029] Further, the determining the number of the two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested includes:

[0030] Determining the confidence that the QR code exists in the image to be tested according to the predicted position label and the predicted area object name of the QR code in the image to be tested;

[0031] When the confidence is greater than a preset confidence threshold, it is determined that there is a QR code in the image to be tested, and the number of QR codes in the image to be tested and the position information of each QR code are determined based on the position annotation of the QR code in the image to be tested, the area object name and the confidence.

[0032] In a second aspect, an embodiment of the present invention provides a two-dimensional code recognition device based on deep learning, comprising:

[0033] A construction unit, configured to construct a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes;

[0034] a labeling unit, configured to label one or more two-dimensional codes of each sample training image in the sample image set, and cluster each sample training image in the labeled sample image set according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes;

[0035] A training unit, used for inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model;

[0036] A determination unit, used for inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested;

[0037] The recognition unit is used to recognize one or more two-dimensional codes in the image to be tested according to the position information of the one or more two-dimensional codes in the image to be tested.

[0038] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program includes a program, and the processor is configured to call the computer program to execute the method of the first aspect above.

[0039] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0040] The embodiment of the present invention can construct a sample image set, wherein the sample image set includes multiple sample training images, each sample training image includes one or more two-dimensional codes; annotate one or more two-dimensional codes of each sample training image in the sample image set, and cluster the sample training images in the sample image set after adding annotations according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model; input the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; and identify one or more two-dimensional codes in the image to be tested according to the position information of one or more two-dimensional codes in the image to be tested. In this way, the recognition of one or more two-dimensional codes in the image can be realized, and the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0042] Figure 1 is a schematic flow chart of a two-dimensional code recognition method based on deep learning provided by an embodiment of the present invention;

[0043] Figure 2 is a schematic block diagram of a two-dimensional code recognition device based on deep learning provided by an embodiment of the present invention;

[0044] Figure 3 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The deep learning-based two-dimensional code recognition method provided in the embodiments of the present invention can be applied to a deep learning-based two-dimensional code recognition device. In some embodiments, the deep learning-based two-dimensional code recognition device is set in a computer device. In some embodiments, the computer device includes but is not limited to one or more of a smart phone, a tablet computer, a laptop computer, etc.

[0047] The embodiment of the present invention can construct a sample image set, the sample image set includes multiple sample training images, each sample training image includes one or more two-dimensional codes; annotate one or more two-dimensional codes of each sample training image in the sample image set, and cluster the sample training images in the sample image set after annotating according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model; input the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; and identify one or more two-dimensional codes in the image to be tested according to the position information of one or more two-dimensional codes in the image to be tested. The embodiment of the present invention can realize the recognition of one or more two-dimensional codes in an image in this way, and improve the accuracy and efficiency of the recognition of one or more two-dimensional codes in an image.

[0048] The embodiments of the present application can acquire and process relevant data (such as sample training images and images to be tested) based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0049] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0050] The following is combined with Figure 1 The deep learning-based two-dimensional code recognition method provided in an embodiment of the present invention is schematically illustrated.

[0051] See also Figure 1 , Figure 1 is a schematic flow chart of a two-dimensional code recognition method based on deep learning provided by an embodiment of the present invention, such as Figure 1 As shown, the method can be executed by a two-dimensional code recognition device based on deep learning, and the two-dimensional code recognition device based on deep learning is set in a computer device. Specifically, the method of the embodiment of the present invention includes the following steps.

[0052] S101: constructing a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes.

[0053] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can construct a sample image set, where the sample image set includes multiple sample training images, and each sample training image includes one or more two-dimensional codes.

[0054] In one embodiment, when constructing a sample image set, the deep learning-based two-dimensional code recognition device can collect image data and store the collected image data as sample training images in the same folder to construct the sample image set, wherein each image in the collected image data includes one or more two-dimensional codes. In some embodiments, when collecting image data, images including one or more two-dimensional codes can be collected from different galleries.

[0055] In one embodiment, the sample image set may also include a sample image set and a test image set, wherein the sample image set is used to train a deep learning network model to obtain a two-dimensional code detection model; and the test image set is used to verify the trained two-dimensional code detection model. When constructing a sample image set, the two-dimensional code recognition device based on deep learning can collect image data and divide the image data according to a preset ratio to obtain a sample image set and a test image set. For example, assuming that the preset ratio is 9:1 for the ratio of the sample image set to the test image set, the collected image data can be divided into a sample image set and a test image set according to a ratio of 9:1.

[0056] In one embodiment, after determining the sample image set and the test image set, the image file name can be randomly extracted from the folder storing the sample image set and / or the test image set, and the suffix such as .jpg is removed and saved as an index in a txt file, which is saved in a single line, that is, it finally contains test.txt, train.txt, trainval.txt and val.txt. The index in the txt file helps to subsequently obtain the annotated content and the original image in the folder at the same time.

[0057] S102: Annotate one or more QR codes of each sample training image in the sample image set, and cluster the sample training images in the annotated sample image set according to the position coordinates of the position area corresponding to each QR code in the one or more QR codes.

[0058] In an embodiment of the present invention, a deep learning-based QR code recognition device can add annotations to one or more QR codes of each sample training image in the sample image set, and cluster the sample training images in the annotated sample image set according to the position coordinates of the position area corresponding to each QR code in the one or more QR codes.

[0059] In one embodiment, when the deep learning-based two-dimensional code recognition device adds annotations to one or more two-dimensional codes of each sample training image in the sample image set, it can determine the position area corresponding to each two-dimensional code in each sample training image and the area object name corresponding to each two-dimensional code; use a designated marking tool to add a first annotation to the position coordinates of the position area corresponding to each two-dimensional code in each sample training image, and add a second annotation to the area object name corresponding to each two-dimensional code in each sample training image, and store each sample training image after adding the first annotation and the second annotation in a designated file. In some embodiments, the designated marking tool includes but is not limited to labelImg.

[0060] In some embodiments, the designated file includes the image object name, image file name, image file path, image size, location coordinates of the location area corresponding to the QR code, area object name corresponding to the QR code, etc. of each sample training image after adding the first annotation and the second annotation.

[0061] In one embodiment, when a deep learning-based QR code recognition device adds annotations to one or more QR codes in each sample training image in the sample image set, it can add annotations to each QR code in each sample training image by manually framing the position area (such as a rectangle) corresponding to each QR code in each sample training image and the area object name (such as qrcode) corresponding to the QR code.

[0062] In one embodiment, when a deep learning-based QR code recognition device clusters the sample training images in the labeled sample image set according to the position coordinates of the position area corresponding to each QR code in the one or more QR codes, it can determine the height and width of each sample training image according to the position coordinates of the position area corresponding to each QR code; and cluster the sample training images in the labeled sample image set according to the height and width of each sample training image.

[0063] In one embodiment, when determining the height and width of each sample training image according to the position coordinates of the position area corresponding to each two-dimensional code, the two-dimensional code recognition device based on deep learning can determine the minimum position coordinates and maximum position coordinates of the position area corresponding to each two-dimensional code according to the position coordinates of the position area corresponding to each two-dimensional code; and input the minimum position coordinates and maximum position coordinates into a specified clustering algorithm model to obtain the height and width of each sample training image. In some embodiments, the minimum position coordinate includes a minimum horizontal coordinate and a minimum vertical coordinate, and the maximum position coordinate includes a maximum horizontal coordinate and a maximum vertical coordinate.

[0064] In some embodiments, the specified clustering algorithm includes, but is not limited to, K-means clustering algorithm, Hierarchical methods hierarchical clustering algorithm, Partition-based methods partition clustering algorithm, etc.

[0065] In one embodiment, when clustering each sample training image in the sample image set after adding annotations, the two-dimensional code recognition device based on deep learning can collect a sample data set, wherein the sample data set includes multiple sample data, and the sample data is image data with annotated categories; randomly select K sample data from the sample data set, and calculate the distance between the feature information of each sample training image and each sample data in the K sample data, where K is a positive integer greater than or equal to 1; determine that the category annotated by the sample data corresponding to the minimum distance is the category of the corresponding sample training image. In some embodiments, the feature information can be vector data, and the sample data can be vector data with annotated two-dimensional code categories.

[0066] In one embodiment, the designated clustering algorithm may be a hierarchical clustering algorithm. When a deep learning-based two-dimensional code recognition device clusters each sample training image in a sample image set after annotation, it may calculate the distance between the feature information of each sample training image and each sample data, and merge the sample data with the minimum distance with the feature information, and determine them as the same category, recalculate the distance between the merged data and each sample data, merge the sample data with the minimum distance with the merged data, and determine them as the same category, and repeat the calculation in this way to determine that the same category to which all the sample data finally merged belongs is the category of the corresponding sample training image.

[0067] S103: Input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model.

[0068] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model.

[0069] In some embodiments, the structure of the preset deep learning network model can be based on the YOLO framework, wherein YOLO is an object recognition and positioning algorithm based on a deep neural network. CSP is added to each large residual block of Darknet53, corresponding to layer0~layer104, i.e., layer 0 to layer 104, thereby forming a backbone network. The receptive field of the network is increased by adding spatial pyramid pooling, and the maximum pooling of layer107 is performed with 5*5, 9*9, and 13*13, respectively, to obtain layer108, ​​layer110, and layer112. After pooling, they are connected into a feature map layer114, and the dimension is reduced to 512 channels through 1*1.

[0070] In one embodiment, a deep learning-based QR code recognition device can obtain a video memory size and adjust the size of each sample training image in the clustered sample image set according to the video memory size before inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model.

[0071] In one example, when adjusting the size of each sample training image in the clustered sample image set according to the video memory size, V100 four-card training can be used to adapt the image size of the input deep network to 416*416 according to the video memory size.

[0072] In one embodiment, when a deep learning-based two-dimensional code recognition device inputs each sample training image in a clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, each sample training image after resizing can be input into the preset deep learning network model to obtain a loss function value; when the loss function value does not meet the preset conditions, the model parameters of the preset deep learning network model are adjusted according to the loss function value, and the each sample training image after resizing is input into the deep learning network model after the model parameters are adjusted for iterative training; when the loss function value obtained after the iterative training meets the preset conditions, it is determined that the two-dimensional code detection model is obtained through training.

[0073] S104: Input the image to be tested into the two-dimensional code detection model to obtain position information of one or more two-dimensional codes in the image to be tested.

[0074] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can input the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested.

[0075] S105: Identify one or more two-dimensional codes in the image to be tested according to position information of one or more two-dimensional codes in the image to be tested.

[0076] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can recognize one or more two-dimensional codes in the image to be tested according to the position information of one or more two-dimensional codes in the image to be tested.

[0077] In one embodiment, when a deep learning-based QR code recognition device inputs an image to be tested into the QR code detection model to obtain the position information of one or more QR codes in the image to be tested, the device can input the image to be tested into the QR code detection model to obtain the predicted position annotation and predicted area object name of the QR code in the image to be tested; and determine the number of QR codes in the image to be tested and the position information of each QR code based on the predicted position annotation and predicted area object name of the QR code in the image to be tested.

[0078] In one embodiment, when a deep learning-based QR code recognition device determines the number of QR codes in the image to be tested and the position information of each QR code based on the predicted position annotation and predicted area object name of the QR code in the image to be tested, it can determine the confidence level of the existence of the QR code in the image to be tested based on the predicted position annotation and predicted area object name of the QR code in the image to be tested; when the confidence level is greater than a preset confidence threshold, it is determined that there is a QR code in the image to be tested, and the number of QR codes in the image to be tested and the position information of each QR code are determined based on the position annotation, area object name and the confidence level of the QR code in the image to be tested.

[0079] In some embodiments, the confidence level is the probability that a location area (such as a bounding box) corresponding to a QR code contains an object and the accuracy of the location (ie, whether it just wraps the QR code).

[0080] In one embodiment, after determining the confidence that the test image contains a two-dimensional code, the probability of the test image containing a two-dimensional code under the confidence level can also be determined, and the predicted value of the two-dimensional code in the image to be tested is determined based on the confidence level and the category probability, that is, the predicted value = confidence level * category probability; the predicted values ​​under each category are sorted according to the size of the predicted values ​​under each category. A preliminary screening is performed by using the category probability greater than a preset parameter, and then the presence of a two-dimensional code in the image to be tested corresponding to the predicted value within the preset threshold range is determined based on the sorted predicted values.

[0081] In one embodiment, when determining the number of QR codes in the image to be tested based on the position annotation of the QR code in the image to be tested, the area object name and the confidence level, if it is determined that the number of QR codes in the image to be tested is 0, then it is determined that there is no QR code in the image to be tested; if it is determined that the number of QR codes in the image to be tested is N, then it is determined that the number of QR codes in the image to be tested is N.

[0082] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can construct a sample image set, wherein the sample image set includes multiple sample training images, each sample training image includes one or more two-dimensional codes; one or more two-dimensional codes of each sample training image in the sample image set are annotated, and each sample training image in the sample image set after the annotation is clustered according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; each sample training image in the clustered sample image set is input into a preset deep learning network model for training to obtain a two-dimensional code detection model; an image to be tested is input into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; and one or more two-dimensional codes in the image to be tested are identified according to the position information of one or more two-dimensional codes in the image to be tested. In this way, the recognition of one or more two-dimensional codes in the image can be realized, and the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image are improved.

[0083] The embodiment of the present invention further provides a two-dimensional code recognition device based on deep learning, which is used to execute a unit of any of the above methods. Figure 2 , Figure 2 2 is a schematic block diagram of a two-dimensional code recognition device based on deep learning provided by an embodiment of the present invention. The two-dimensional code recognition device based on deep learning of this embodiment includes: a construction unit 201, a labeling unit 202, a training unit 203, a determination unit 204 and a recognition unit 205.

[0084] A construction unit 201 is used to construct a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes;

[0085] A labeling unit 202 is used to label one or more two-dimensional codes of each sample training image in the sample image set, and cluster each sample training image in the sample image set after labeling according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes;

[0086] The training unit 203 is used to input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model;

[0087] A determination unit 204 is used to input the image to be tested into the two-dimensional code detection model to obtain position information of one or more two-dimensional codes in the image to be tested;

[0088] The recognition unit 205 is configured to recognize one or more two-dimensional codes in the image to be tested according to position information of the one or more two-dimensional codes in the image to be tested.

[0089] Furthermore, when the annotation unit 202 adds annotations to one or more two-dimensional codes of each sample training image in the sample image set, it is specifically used to:

[0090] Determine the location area corresponding to each two-dimensional code in each sample training image and the area object name corresponding to each two-dimensional code;

[0091] Use a designated marking tool to add a first annotation to the position coordinates of the position area corresponding to each QR code in each sample training image, and add a second annotation to the area object name corresponding to each QR code in each sample training image, and store each sample training image after adding the first annotation and the second annotation in a designated file.

[0092] Further, when the labeling unit 202 clusters the sample training images in the labeled sample image set according to the position coordinates of the position area corresponding to each of the one or more two-dimensional codes, it is specifically used to:

[0093] Determine the height and width of each sample training image according to the position coordinates of the position area corresponding to each QR code;

[0094] Clustering the sample training images in the labeled sample image set is performed according to the height and width of the sample training images.

[0095] Further, when the labeling unit 202 determines the height and width of each sample training image according to the position coordinates of the position area corresponding to each two-dimensional code, it is specifically used to:

[0096] Determine the minimum position coordinates and the maximum position coordinates of the position areas corresponding to the respective two-dimensional codes according to the position coordinates of the position areas corresponding to the respective two-dimensional codes;

[0097] The minimum position coordinate and the maximum position coordinate are input into a specified clustering algorithm model to obtain the height and width of each sample training image.

[0098] Furthermore, the training unit 203 inputs each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, and is also used for:

[0099] Acquire a video memory size, and adjust the size of each sample training image in the clustered sample image set according to the video memory size;

[0100] The training unit 203 inputs each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, which is specifically used for:

[0101] Input each resized sample training image into the preset deep learning network model to obtain the loss function value;

[0102] When the loss function value does not meet the preset conditions, adjusting the model parameters of the preset deep learning network model according to the loss function value, and inputting the resized sample training images into the deep learning network model after adjusting the model parameters for iterative training;

[0103] When the loss function value obtained after iterative training meets the preset condition, it is determined that the two-dimensional code detection model is obtained through training.

[0104] Furthermore, when the determination unit 204 inputs the image to be tested into the two-dimensional code detection model and obtains the position information of one or more two-dimensional codes in the image to be tested, it is specifically used to:

[0105] Input the image to be tested into the two-dimensional code detection model to obtain the predicted position label and predicted area object name of the two-dimensional code of the image to be tested;

[0106] The number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code are determined according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested.

[0107] Further, when the determination unit 204 determines the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested, it is specifically used to:

[0108] Determining the confidence that the QR code exists in the image to be tested according to the predicted position label and the predicted area object name of the QR code in the image to be tested;

[0109] When the confidence is greater than a preset confidence threshold, it is determined that there is a QR code in the image to be tested, and the number of QR codes in the image to be tested and the position information of each QR code are determined based on the position annotation of the QR code in the image to be tested, the area object name and the confidence.

[0110] In an embodiment of the present invention, a two-dimensional code recognition device based on deep learning can construct a sample image set, wherein the sample image set includes multiple sample training images, each sample training image includes one or more two-dimensional codes; one or more two-dimensional codes of each sample training image in the sample image set are annotated, and each sample training image in the sample image set after the annotation is clustered according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; each sample training image in the clustered sample image set is input into a preset deep learning network model for training to obtain a two-dimensional code detection model; an image to be tested is input into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; and one or more two-dimensional codes in the image to be tested are identified according to the position information of one or more two-dimensional codes in the image to be tested. In this way, the recognition of one or more two-dimensional codes in the image can be realized, and the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image are improved.

[0111] See also Figure 3 , Figure 3 is a schematic block diagram of a computer device provided by an embodiment of the present invention. In some embodiments, such as Figure 3 The computer device in the embodiment shown may include: one or more processors 301; one or more input devices 302, one or more output devices 303 and a memory 304. The processor 301, input device 302, output device 303 and memory 304 are connected via a bus 305. The memory 304 is used to store a computer program, the computer program includes a program, and the processor 301 is used to execute the program stored in the memory 304. The processor 301 is configured to call the program to execute:

[0112] Constructing a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes;

[0113] adding annotations to one or more two-dimensional codes of each sample training image in the sample image set, and clustering each sample training image in the sample image set after the annotations are added according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes;

[0114] Input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model;

[0115] Inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested;

[0116] One or more two-dimensional codes in the image to be tested are identified according to the position information of the one or more two-dimensional codes in the image to be tested.

[0117] Furthermore, when the processor 301 adds annotations to one or more two-dimensional codes of each sample training image in the sample image set, it is specifically configured to:

[0118] Determine the location area corresponding to each two-dimensional code in each sample training image and the area object name corresponding to each two-dimensional code;

[0119] Use a designated marking tool to add a first annotation to the position coordinates of the position area corresponding to each QR code in each sample training image, and add a second annotation to the area object name corresponding to each QR code in each sample training image, and store each sample training image after adding the first annotation and the second annotation in a designated file.

[0120] Further, when the processor 301 clusters the sample training images in the labeled sample image set according to the position coordinates of the position area corresponding to each of the one or more two-dimensional codes, it is specifically configured to:

[0121] Determine the height and width of each sample training image according to the position coordinates of the position area corresponding to each QR code;

[0122] Clustering the sample training images in the labeled sample image set is performed according to the height and width of the sample training images.

[0123] Further, when the processor 301 determines the height and width of each sample training image according to the position coordinates of the position area corresponding to each two-dimensional code, it is specifically used to:

[0124] Determine the minimum position coordinates and the maximum position coordinates of the position areas corresponding to the respective two-dimensional codes according to the position coordinates of the position areas corresponding to the respective two-dimensional codes;

[0125] The minimum position coordinate and the maximum position coordinate are input into a specified clustering algorithm model to obtain the height and width of each sample training image.

[0126] Furthermore, before the processor 301 inputs each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, it is also used to:

[0127] Acquire a video memory size, and adjust the size of each sample training image in the clustered sample image set according to the video memory size;

[0128] The processor 301 inputs each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, which is specifically used for:

[0129] Input each resized sample training image into the preset deep learning network model to obtain the loss function value;

[0130] When the loss function value does not meet the preset conditions, adjusting the model parameters of the preset deep learning network model according to the loss function value, and inputting the resized sample training images into the deep learning network model after adjusting the model parameters for iterative training;

[0131] When the loss function value obtained after iterative training meets the preset condition, it is determined that the two-dimensional code detection model is obtained through training.

[0132] Further, when the processor 301 inputs the image to be tested into the two-dimensional code detection model and obtains the position information of one or more two-dimensional codes in the image to be tested, it is specifically used to:

[0133] Input the image to be tested into the two-dimensional code detection model to obtain the predicted position label and predicted area object name of the two-dimensional code of the image to be tested;

[0134] The number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code are determined according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested.

[0135] Further, when the processor 301 determines the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested, it is specifically used to:

[0136] Determining the confidence that the QR code exists in the image to be tested according to the predicted position label and the predicted area object name of the QR code in the image to be tested;

[0137] When the confidence is greater than a preset confidence threshold, it is determined that there is a QR code in the image to be tested, and the number of QR codes in the image to be tested and the position information of each QR code are determined based on the position annotation of the QR code in the image to be tested, the area object name and the confidence.

[0138] In an embodiment of the present invention, a computer device can construct a sample image set, wherein the sample image set includes multiple sample training images, each sample training image includes one or more two-dimensional codes; annotate one or more two-dimensional codes of each sample training image in the sample image set, and cluster the sample training images in the sample image set after adding annotations according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model; input the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; and identify one or more two-dimensional codes in the image to be tested according to the position information of one or more two-dimensional codes in the image to be tested. In this way, the recognition of one or more two-dimensional codes in the image can be realized, and the accuracy and efficiency of the recognition of one or more two-dimensional codes in the image are improved.

[0139] It should be understood that in the embodiment of the present invention, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] The input device 302 may include a touch pad, a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0141] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0142] In a specific implementation, the processor 301, the input device 302, and the output device 303 described in the embodiment of the present invention can execute the Figure 1 The implementation described in the method embodiment can also implement the embodiment of the present invention. Figure 2 The implementation method of the described deep learning-based two-dimensional code recognition device will not be repeated here.

[0143] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, Figure 1 The two-dimensional code recognition method based on deep learning described in the corresponding embodiment can also realize the present invention Figure 2 The two-dimensional code recognition device based on deep learning of the corresponding embodiment will not be described in detail here.

[0144] The computer-readable storage medium may be an internal storage unit of the deep learning-based two-dimensional code recognition device described in any of the aforementioned embodiments, such as a hard disk or memory of the deep learning-based two-dimensional code recognition device. The computer-readable storage medium may also be an external storage device of the deep learning-based two-dimensional code recognition device, such as a plug-in hard disk, a smart memory card (SmarS Media Card, SMC), a secure digital (Secure DigiSal, SD) card, a flash card (Flash Card), etc. equipped on the deep learning-based two-dimensional code recognition device. Further, the computer-readable storage medium may also include both the internal storage unit of the deep learning-based two-dimensional code recognition device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the deep learning-based two-dimensional code recognition device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly contributed to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes. The computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application required for a function, etc.; the storage data area may store data created according to the use of blockchain nodes, etc.

[0146] It should be emphasized that in order to further ensure the privacy and security of the above data, the above data can also be stored in a node of a blockchain. Among them, the blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer.

[0147] The above descriptions are only partial implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A two-dimensional code recognition method based on deep learning, characterized in that: include: Constructing a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes; adding annotations to one or more two-dimensional codes of each sample training image in the sample image set, and clustering each sample training image in the sample image set after the annotations are added according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; Input each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model; Inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; Identify one or more two-dimensional codes in the image to be tested according to position information of one or more two-dimensional codes in the image to be tested; Before inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, the method further includes: Acquire a video memory size, and adjust the size of each sample training image in the clustered sample image set according to the video memory size; The method of inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model includes: Input each resized sample training image into the preset deep learning network model to obtain the loss function value; When the loss function value does not meet the preset conditions, adjusting the model parameters of the preset deep learning network model according to the loss function value, and inputting the resized sample training images into the deep learning network model after adjusting the model parameters for iterative training; When the loss function value obtained after iterative training meets the preset condition, it is determined that the two-dimensional code detection model is obtained through training; The step of inputting the image to be tested into the two-dimensional code detection model to obtain position information of one or more two-dimensional codes in the image to be tested includes: Input the image to be tested into the two-dimensional code detection model to obtain the predicted position label and predicted area object name of the two-dimensional code of the image to be tested; Determine the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested; The step of determining the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position labeling and the predicted area object name of the two-dimensional code in the image to be tested includes: Determining the confidence that the QR code exists in the image to be tested according to the predicted position label and the predicted area object name of the QR code in the image to be tested; When the confidence level is greater than a preset confidence threshold, it is determined that there is a two-dimensional code in the image to be tested, and the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code are determined according to the predicted position label of the two-dimensional code in the image to be tested, the predicted area object name and the confidence level; The clustering of the sample training images in the labeled sample image set includes: Calculate the distance between the feature information of each sample training image and each sample data, merge the sample data with the minimum distance with the feature information, and determine them as the same category, recalculate the distance between the merged data and each sample data, merge the sample data with the minimum distance with the merged data, and determine them as the same category, and repeat the calculation in this way, and determine that the same category to which all the sample data finally merged belongs is the category of the corresponding sample training image; When the confidence level is greater than a preset confidence threshold, determining that a two-dimensional code exists in the image to be tested includes: Determine the probability that the image to be tested contains a category of a two-dimensional code under the confidence level, and determine a prediction value of the existence of the two-dimensional code in the image to be tested according to the confidence level and the category probability, wherein the prediction value is the product of the confidence level and the category probability; According to the prediction values ​​under each category, the prediction values ​​under each category are sorted in order from large to small, and the prediction values ​​under each category whose category probability is greater than the preset parameter are screened out, and then from the screened prediction values, it is determined that there is a QR code in the image to be tested corresponding to the prediction value within the preset threshold range.

2. The method according to claim 1, characterized in that The adding annotations to one or more two-dimensional codes of each sample training image in the sample image set includes: Determine the location area corresponding to each two-dimensional code in each sample training image and the area object name corresponding to each two-dimensional code; Use a designated marking tool to add a first annotation to the position coordinates of the position area corresponding to each QR code in each sample training image, and add a second annotation to the area object name corresponding to each QR code in each sample training image, and store each sample training image after adding the first annotation and the second annotation in a designated file.

3. The method according to claim 1, characterized in that The clustering of the sample training images in the labeled sample image set according to the position coordinates of the position area corresponding to each of the one or more two-dimensional codes includes: Determine the height and width of each sample training image according to the position coordinates of the position area corresponding to each QR code; Clustering is performed on the sample training images in the labeled sample image set according to the height and width of the sample training images.

4. The method according to claim 3, characterized in that The determining the height and width of each sample training image according to the position coordinates of the position area corresponding to each two-dimensional code includes: Determine the minimum position coordinates and the maximum position coordinates of the position areas corresponding to the respective two-dimensional codes according to the position coordinates of the position areas corresponding to the respective two-dimensional codes; The minimum position coordinate and the maximum position coordinate are input into a specified clustering algorithm model to obtain the height and width of each sample training image.

5. A two-dimensional code recognition device based on deep learning, characterized in that: include: A construction unit, configured to construct a sample image set, wherein the sample image set includes a plurality of sample training images, and each sample training image includes one or more two-dimensional codes; a labeling unit, configured to label one or more two-dimensional codes of each sample training image in the sample image set, and cluster each sample training image in the labeled sample image set according to the position coordinates of the position area corresponding to each two-dimensional code in the one or more two-dimensional codes; A training unit, used for inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a QR code detection model; A determination unit, used for inputting the image to be tested into the two-dimensional code detection model to obtain the position information of one or more two-dimensional codes in the image to be tested; an identification unit, configured to identify one or more two-dimensional codes in the image to be tested according to position information of the one or more two-dimensional codes in the image to be tested; The training unit inputs each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model, and is also used for: Acquire a video memory size, and adjust the size of each sample training image in the clustered sample image set according to the video memory size; The method of inputting each sample training image in the clustered sample image set into a preset deep learning network model for training to obtain a two-dimensional code detection model includes: Input each resized sample training image into the preset deep learning network model to obtain the loss function value; When the loss function value does not meet the preset conditions, adjusting the model parameters of the preset deep learning network model according to the loss function value, and inputting the resized sample training images into the deep learning network model after adjusting the model parameters for iterative training; When the loss function value obtained after iterative training meets the preset condition, it is determined that the two-dimensional code detection model is obtained through training; When the determination unit inputs the image to be tested into the two-dimensional code detection model and obtains the position information of one or more two-dimensional codes in the image to be tested, it is specifically used to: Input the image to be tested into the two-dimensional code detection model to obtain the predicted position label and predicted area object name of the two-dimensional code of the image to be tested; Determine the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested; When the determination unit determines the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code according to the predicted position label and the predicted area object name of the two-dimensional code in the image to be tested, it is specifically used to: Determining the confidence that the QR code exists in the image to be tested according to the predicted position label and the predicted area object name of the QR code in the image to be tested; When the confidence level is greater than a preset confidence threshold, it is determined that there is a two-dimensional code in the image to be tested, and the number of two-dimensional codes in the image to be tested and the position information of each two-dimensional code are determined according to the predicted position label of the two-dimensional code in the image to be tested, the predicted area object name and the confidence level; When the labeling unit clusters the sample training images in the labeled sample image set, it is specifically used to: Calculate the distance between the feature information of each sample training image and each sample data, merge the sample data with the minimum distance with the feature information, and determine them as the same category, recalculate the distance between the merged data and each sample data, merge the sample data with the minimum distance with the merged data, and determine them as the same category, and repeat the calculation in this way, and determine that the same category to which all the sample data finally merged belongs is the category of the corresponding sample training image; When the confidence level is greater than a preset confidence threshold, the determining unit determines that a two-dimensional code exists in the image to be tested, specifically for: Determine the probability that the image to be tested contains a category of a two-dimensional code under the confidence level, and determine a prediction value of the existence of the two-dimensional code in the image to be tested according to the confidence level and the category probability, wherein the prediction value is the product of the confidence level and the category probability; According to the prediction values ​​under each category, the prediction values ​​under each category are sorted in order from large to small, and the prediction values ​​under each category whose category probability is greater than the preset parameter are screened out, and then from the screened prediction values, it is determined that there is a QR code in the image to be tested corresponding to the prediction value within the preset threshold range.

6. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is configured to call the computer program to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images