QR code image reconstruction method based on position coding
Through the QR code image reconstruction method based on location encoding, the problem of difficult QR code decoding in complex environments is solved, high-precision recognition and decoding is achieved, and the efficiency and reliability of logistics management and product tracking is improved. It is suitable for Qualcomm chip platform .
Patent Information
- Application Number
- CN202411274959.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2025-01-07
AI Technical Summary
The existing QR code decoding technology is difficult to accurately identify and decode in complex environments (such as insufficient lighting, long distances and large angles), which consumes a lot of time and resources and cannot meet the needs of warehousing and logistics and commodity tracking.
Using the QR code image reconstruction method based on location encoding, the location encoding and ReLU6 activation function were added to remove the entire system by generating simulation data sets, constructing ResNet-101 teacher model and convolutional neural network student model, using knowledge distillation strategies and minimalist models. The connection layer, which uses only a simple convolutional layer and pooling layer, is deployed on Qualcomm chip for testing.
High-precision QR code recognition and decoding are achieved in complex environments, improving the practicality and reliability of logistics management and product tracking, significantly improving work efficiency, saving human and material resources, and reducing error rates.
Smart Images
Figure CN119272781A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision and machine learning, and in particular relates to a QR code image reconstruction method based on position coding. Background Art
[0002] In the warehousing and logistics industry, we often encounter a situation where a large number of items are stacked together, and each item has several barcodes. The existing decoding technology can only scan one by one, which not only takes a lot of time, but also wastes manpower and material resources. Therefore, it is urgent to develop a system that can simultaneously recognize and decode multiple barcodes in one shot. Since the system needs to process multiple barcodes in one scan, it will inevitably cause the barcodes at the edge of the image to be unclear and deformed due to the large shooting angle. Traditional decoding algorithms are difficult to decode accurately in this case and cannot meet the needs of practical applications. Summary of the invention
[0003] The purpose of the present invention is to provide a QR code image reconstruction method based on position coding. The present invention can accurately identify and decode in complex environments, especially in the case of insufficient lighting, long distance and large angle, so as to improve the practicality and reliability of the system in the fields of logistics management and commodity tracking. The image reconstruction method based on position coding realizes complex image reconstruction tasks with a minimalist model. The application of this achievement on embedded platforms is of great significance, improving work efficiency and saving manpower and material resources.
[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0005] A QR code image reconstruction method based on position coding comprises the following steps:
[0006] S1, generate simulated QR code classification data set and regression data set;
[0007] S2, collect the real QR code dataset and annotate the dataset;
[0008] S3, uses ResNet-101 to build a teacher model of the classification model, uses the convolutional neural network method to build a student model of the classification model, and designs a QR code regression model based on position coding;
[0009] S4, uses the simulated QR code classification dataset to train the teacher model of the classification model, uses the simulated QR code classification dataset and the knowledge distillation strategy to train the student model of the classification model, and uses the simulated QR code regression dataset to train the regression model containing position encoding;
[0010] S5, deploy the model to Qualcomm chips to test the performance of the model in real scenarios.
[0011] Preferably, in S3, when designing a QR code regression model based on position coding, the method used is as follows:
[0012] A1, the model input image is a single-channel image, and two-dimensional position encoding is added before entering the network;
[0013] A2, the model uses the ReLU6 activation function after the convolutional layer;
[0014] A3, remove the fully connected layer of the regression model to optimize the network structure;
[0015] A4 uses a minimalist model, which only uses simple convolutional layers, pooling layers, and activation layers.
[0016] Preferably, in S4, the specific steps of using the simulated QR code classification data set and the knowledge distillation strategy to train the student model of the classification model are as follows:
[0017] S40201, set hyperparameters for knowledge distillation;
[0018] S40202, use the trained teacher model to predict the training data and generate soft labels;
[0019] S40203, combine soft labels and hard labels to define the distillation loss function;
[0020] S40204, training a student model of a classification model using a distillation loss function and a simulated QR code classification dataset generated in S1;
[0021] S40205, after training, the student model is evaluated and optimized.
[0022] Preferably, in S4, the specific steps of using the simulated QR code regression data set to train the regression model containing the position code are:
[0023] S40301, set hyperparameters of the regression model;
[0024] S40302, define the loss function, the specific formula is as follows:
[0025]
[0026]
[0027] Among them, σ(X ij ) is the sigmoid function, N is the number of black and white blocks in the QR code, W ij is the weight of the i-th row and j-th column, X ij is the probability value of the predicted black and white block in the i-th row and j-th column, Y ijis the element in the i-th row and j-th column of the target matrix Y, W ij The weight of the eighth row and the eighth column is 2, and the weight of the remaining elements is 1;
[0028] S40303, training a regression model containing position encoding using the loss function and the simulated QR code regression dataset generated in S1;
[0029] S40304, after training, evaluate and optimize the student model.
[0030] The beneficial effects of the present invention are:
[0031] This method aims to achieve high-precision reconstruction of damaged or distorted QR codes by combining deep learning and position coding technology. It can accurately identify and decode in complex environments, especially in low light conditions, long distances, and large angles, thereby improving the practicality and reliability of the system in areas such as logistics management and product tracking. The algorithm will significantly improve work efficiency in warehousing and logistics, save manpower and material resources, significantly improve the ability to process QR codes on mobile devices, and reduce error rates, especially on mobile devices with limited Qualcomm chips, showing wide practicality and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a flow chart of the present invention.
[0033] Figure 2 Schematic diagram for reconstructing QR code.
[0034] Figure 3 This is the actual detection effect diagram. DETAILED DESCRIPTION
[0035] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention.
[0036] like Figure 1-Figure 3 As shown, the present invention discloses a QR code image reconstruction method based on position coding, comprising the following steps:
[0037] S1, generate simulated QR code classification dataset and regression dataset.
[0038] Since the cost of collecting and annotating the real QR code data set is too high, the present invention uses a simulated QR code data set generated based on ray tracing technology to train the model. Practice has proved that the simulated QR code data set generated by this method can fully meet the requirements of training a high-precision deep learning model.
[0039] S101, making a simulated QR code classification data set required for a classification model, wherein the label of the data set is a QR code version number.
[0040] The QR code images in the QR code classification dataset include multiple different versions of images. Each version of the image simulates different scanning quality and environmental factors, such as pollution, too strong or too dark light, blur, etc. The label of the QR code classification dataset is the QR code version number. There are 11 versions in total, and each version contains 10,000 images. The data of the QR code classification dataset will be used to train the QR code classification model.
[0041] S102, making a simulated QR code regression data set required for the regression model, the label of the data set is a 01 matrix.
[0042] The QR code images in the QR code regression dataset cover all extreme situations encountered in daily life, such as: too large angle, distortion, stains, etc. There are 11 versions in total, 5,000 images in each version, and a total of 55,000 images. The label of the QR code regression dataset is a 01 matrix, where 0 represents black and 1 represents white.
[0043] The data of the QR code regression dataset will be used to train the QR code regression model, with the goal of enabling the model to reconstruct high-quality QR code images, with the labels being the corresponding 01 matrices, representing the precise two-dimensional layout of the QR code, ensuring that the model can accurately recover the complete data from any damaged or distorted QR code images.
[0044] S2, collect the real QR code dataset and annotate the dataset;
[0045] The specific steps for collecting a real QR code dataset include:
[0046] S201, collect QR code images of different versions under various real environment conditions, and annotate the QR code version number of each image to support the training of the classification model.
[0047] A total of 2,000 real QR code data sets were collected for the classification model. These data were used to test the generalization ability of the classification model trained using the simulated QR code data set on the real QR code data set.
[0048] S202, collecting 1100 real QR code data sets of the regression model to test the generalization ability of the regression model trained using the simulated QR code data set on the real QR code data set.
[0049] S3, uses ResNet-101 to build the teacher model of the classification model, uses the convolutional neural network (CNN) method to build the student model of the classification model, and designs the QR code regression model based on position coding.
[0050] 1. Use ResNet-101 to build the teacher model of the classification model, and use the convolutional neural network (CNN) method to build the student model of the classification model.
[0051] (1) Construct a teacher model for the classification model.
[0052] The teacher model is ResNet-101. The architecture of ResNet-101 relies on the idea of residual learning. Through skip connections, each layer can directly receive inputs from previous layers. This not only alleviates the gradient vanishing problem of deep networks, but also makes the training process more stable and efficient. Although ResNet-101 is a very deep network, it can converge in a reasonable time and performs well on classification tasks.
[0053] In order to be successfully deployed on resource-constrained mobile platforms, knowledge distillation technology can be used for model compression. It uses a large and complex teacher model to guide the learning process of a small and lightweight student model, thereby significantly reducing the size and computational complexity of the model while maintaining or approaching the performance of the teacher model.
[0054] (2) Construct a student model for the classification model.
[0055] Using deep learning technology, a convolutional neural network (CNN) method is used to build the student model of the classification model.
[0056] 2. When designing a QR code regression model based on position coding, the method used is as follows:
[0057] Since there are 40 versions of QR codes, this paper only designs regression models for the first 11 versions commonly used in daily life, which already include 99% of QR codes in daily life. The important structure of the regression model is to add two-dimensional position encoding before the convolution operation, and to add a residual structure during the convolution process, and to follow the following points in the design:
[0058] A1, the input image of the regression model is a single-channel image, and two-dimensional position encoding is added before entering the regression model. Position encoding can capture the spatial position information in the image and enhance the representation ability of the regression model. Experiments have shown that position encoding is the key to the success of this method.
[0059] A2, the regression model uses the ReLU6 activation function after the convolutional layer.
[0060] The regression model uses the ReLU6 activation function after the convolution layer to reduce the quantization error that may be generated during the quantization process of the regression model, thereby improving the performance of the model in a low-precision computing environment.
[0061] A3, remove the fully connected layer of the regression model to optimize the network structure.
[0062] When designing the regression model, the fully connected layer was removed because the state (black or white) of each black and white block in the QR code mainly depends on local features and does not require global connection. Therefore, removing the fully connected layer can effectively reduce the number of parameters of the regression model.
[0063] A4 uses a minimalist model, which only uses simple convolutional layers, pooling layers, and activation layers.
[0064] The regression model does not use adaptive average pooling layers and deconvolution operations, but only simple convolution layers, pooling layers, and activation layers. The complex image reconstruction task is achieved with a minimalist model.
[0065] Because the Qualcomm quantization platform has strict restrictions on the adaptive average pooling layer, there is no need to use the adaptive average pooling layer to ensure the smooth deployment and efficient execution of the regression model on the Qualcomm chip.
[0066] Because the deconvolution operation cannot be integrated with the normalization layer and the activation layer during quantization, which will result in a larger range of activation values for this layer and a larger quantization loss, the deconvolution operation cannot be used to avoid problems such as large quantization errors caused by these operations during quantization.
[0067] After the QR code classification model and QR code regression model are designed, the classification model and regression model can be used normally to reconstruct the QR code principle. Figure 2 As shown:
[0068] Classification model: used to identify the version of the QR code. First, the input QR code image is classified and its version information is identified through a neural network. After determining the version of the QR code, the system will output the version number and dynamically call the corresponding regression model.
[0069] Regression model: A dedicated regression model is designed for each QR code version. Each regression model has been optimized for a specific version of the QR code and can handle image distortion problems in various extreme environments. Through these specially designed regression models, the system can output a 01 matrix corresponding to the QR code structure, which reconstructs the QR code in binary form, thereby generating a clear and decodable QR code image.
[0070] S4, uses the simulated QR code classification dataset to train the teacher model of the classification model, uses the simulated QR code classification dataset and knowledge distillation strategy to train the student model of the classification model, and uses the simulated QR code regression dataset to train the regression model containing position encoding.
[0071] S401, the specific steps of using the simulated QR code classification data set to train the teacher model of the classification model are as follows:
[0072] S40101, set the hyperparameters of the teacher model of the classification model.
[0073] S40102, use the simulated QR code classification dataset generated by S1 to train the teacher model constructed by S3.
[0074] Model output: The model outputs the version number of the current image, which is used to select the corresponding regression model later.
[0075] Loss function and learning rate decay strategy: Use the cross entropy loss function to train the teacher model. When the loss does not decrease after 10 rounds, adjust the learning rate to half of the original value. When the loss does not decrease after 20 rounds, end the training.
[0076] S40103, performance verification: test the accuracy of the teacher model on the real QR code data set of the classification model collected in S201. If the accuracy does not meet the requirement, regenerate the simulated QR code data set and retrain until the accuracy meets the requirement.
[0077] S402, the specific steps of using the simulated QR code classification data set and the knowledge distillation strategy to train the student model of the classification model are as follows:
[0078] Knowledge distillation is a model compression technique used to transfer the knowledge of a large, complex model to a smaller and simpler model. Its purpose is to reduce the computational complexity and storage requirements of the model while maintaining predictive accuracy.
[0079] S40201, set the hyperparameters for knowledge distillation.
[0080] S40202, use the trained teacher model to predict the training data and generate soft labels.
[0081] The soft label is obtained by calculating the probability distribution of the output layer of the teacher model. In this classification model, the soft label is the probability of each category obtained using the softmax function.
[0082] S40203, combine soft labels and hard labels to define the distillation loss function.
[0083] Hard labels refer to the true category labels in the original training data set. During the training process, hard labels are used to calculate the cross entropy loss. The soft labels and hard labels are combined to define a new distillation loss function. The method is to weight the cross entropy loss of the soft labels and the cross entropy loss of the hard labels. The specific formula is as follows:
[0084] L= αL hard + βLsoft Formula (1)
[0085] Among them, L hard is the cross entropy loss using hard labels, L soft The cross entropy loss of the soft label is used, and α and β are weight coefficients used to balance the contribution of the two losses, which are 0.7 and 0.3 respectively.
[0086] S40204, trains a student model of the classification model using the distillation loss function and the simulated QR code classification dataset generated by S1.
[0087] S40205, after training, the student model is evaluated and optimized.
[0088] After the training is completed, the student model is evaluated to check whether its performance meets the requirements on the real QR code data set of the classification model collected in S201. If necessary, the weight coefficient of the distillation loss function can be adjusted, or the model structure can be improved to further optimize the performance of the student model.
[0089] S403, the specific steps of using the simulated QR code regression data set to train a regression model containing position coding are as follows:
[0090] S40301, set the hyperparameters of the regression model.
[0091] Determine the hyperparameters needed during training, including learning rate, batch size, number of training rounds, select an optimizer, and set its initial learning rate.
[0092] S40302, define the loss function, the specific formula is as follows:
[0093]
[0094] Among them, σ(X ij ) is the sigmoid function, N is the number of black and white blocks in the QR code, W ij is the weight of the i-th row and j-th column, X ij is the probability value of the predicted black and white block in the i-th row and j-th column, Y ij is the element in the i-th row and j-th column of the target matrix Y, W ij The weight of the eighth row and the eighth column is 2, and the weight of the remaining elements is 1.
[0095] Because the eighth row and the eighth column of the QR code are the most important functional graphics, which are used to determine the confidence of the mask used by the QR code and are the basis for successful decoding, the weight of the eighth row and the eighth column is 2, and the weight of the remaining elements is 1.
[0096] S40303, train a regression model containing position encoding using the loss function and the simulated QR code regression dataset generated by S1.
[0097] S4030301, iterate multiple training epochs, traversing the training data in each epoch.
[0098] S4030302, for each batch of data, perform the following steps:
[0099] a. Load the input data and target values onto the computing device.
[0100] b. Pass the input data through the position encoding module to generate an input tensor with position information.
[0101] c. Pass the position-encoded input tensor to the model to get the predicted value.
[0102] d. Use the loss function to calculate the loss between the predicted value and the target value.
[0103] e. Perform back propagation and calculate gradients.
[0104] f. Use the optimizer to update the model parameters.
[0105] S4030303 At the end of each training round, evaluate the performance of the model on the validation set, record the loss and other evaluation indicators, and stop training when the loss does not decrease after 20 rounds.
[0106] S40304, after training, evaluate and optimize the regression model.
[0107] After the training is completed, the regression model is evaluated to check whether its performance meets the requirements on the real QR code data set of the regression model collected in S202. If necessary, the weight coefficient of the loss function can be adjusted, or the model structure can be improved to further optimize the performance of the regression model.
[0108] S5, deploy the model to Qualcomm chips to test the performance of the model in real scenarios.
[0109] After the quantization deployment platform of Qualcomm chips, the trained model was quantized and successfully deployed on Qualcomm chips for actual testing. The actual test results are as follows: Figure 3 As shown in the figure, the image reconstruction method based on position coding realizes the complex image reconstruction task with a minimalist model. This result is of great significance in the application of embedded platforms.
Claims
1. A QR code image reconstruction method based on position coding, characterized in that: The following steps are involved: S1, generate simulated QR code classification data set and regression data set; S2, collect the real QR code dataset and annotate the dataset; S3, uses ResNet-101 to build a teacher model of the classification model, uses the convolutional neural network method to build a student model of the classification model, and designs a QR code regression model based on position coding; S4, uses the simulated QR code classification dataset to train the teacher model of the classification model, uses the simulated QR code classification dataset and the knowledge distillation strategy to train the student model of the classification model, and uses the simulated QR code regression dataset to train the regression model containing position encoding; S5, deploy the model to Qualcomm chips to test the performance of the model in real scenarios.
2. The QR code image reconstruction method based on position coding according to claim 1, characterized in that: In S3, when designing a QR code regression model based on position coding, the method used is as follows: A1, the model input image is a single-channel image, and two-dimensional position encoding is added before entering the network; A2, the model uses the ReLU6 activation function after the convolutional layer; A3, remove the fully connected layer of the regression model to optimize the network structure; A4 uses a minimalist model, which only uses simple convolutional layers, pooling layers, and activation layers.
3. The QR code image reconstruction method based on position coding according to claim 1, characterized in that: In S4, the specific steps of using the simulated QR code classification data set and the knowledge distillation strategy to train the student model of the classification model are as follows: S40201, set hyperparameters for knowledge distillation; S40202, use the trained teacher model to predict the training data and generate soft labels; S40203, combine soft labels and hard labels to define the distillation loss function; S40204, training a student model of a classification model using a distillation loss function and a simulated QR code classification dataset generated in S1; S40205, after training, the student model is evaluated and optimized.
4. The QR code image reconstruction method based on position coding according to claim 1, characterized in that: In S4, the specific steps of using the simulated QR code regression data set to train the regression model containing position coding are: S40301, set hyperparameters of the regression model; S40302, define the loss function, the specific formula is as follows: Among them, σ(X ij ) is the sigmoid function, N is the number of black and white blocks in the QR code, W ij is the weight of the i-th row and j-th column, X ij is the probability value of the predicted black and white block in the i-th row and j-th column, Y ij is the element in the i-th row and j-th column of the target matrix Y, W ij The weight of the eighth row and the eighth column is 2, and the weight of the remaining elements is 1; S40303, training a regression model containing position encoding using the loss function and the simulated QR code regression dataset generated in S1; S40304, after training, evaluate and optimize the student model.