Method for recognizing damaged quick-response code

Abstract

The present application relates to the technical field of damaged codes, and specifically relates to a method for recognizing a damaged quick-response code. The method comprises: first acquiring a sample data set, inputting the sample data set into a CenterNet target detection model to be trained, and training the CenterNet target detection model; and then acquiring a target damaged quick-response code to be recognized, and recognizing the target damaged quick-response code by means of the trained CenterNet target detection model, so as to acquire a damage rate of the target damaged quick-response code. In the solution, a target damaged quick-response code is recognized by using a trained CenterNet target detection model, thereby reducing the amount of and complexity of calculation; and the CenterNet target detection model is optimized by means of an activation function, and the performance of the trained CenterNet target detection model is then evaluated, thereby improving the accuracy and efficiency of damage rate recognition results.

Description

A method for identifying damaged QR codes

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on August 22, 2023, with application number 202311063816.4 and invention name “A Method for Identifying a Damaged QR Code”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the technical field of damaged codes, and in particular to a method for identifying damaged QR codes. Background Art

[0004] In recent years, with the development of industrial production automation, the emergence of mobile robot (Automated Guided Vehicle, AGV) technology has brought about fundamental changes in industrial production.

[0005] At present, in indoor positioning, QR code (Quick Response code, QR code) recognition technology is often used in SLAM technology (Simultaneous Localization and Mapping) due to its advantages such as simple operation, low price and high precision. That is, QR codes are affixed to key points and indicate the absolute position in world coordinates, providing reliable positioning tags for visual positioning, and the map is updated back to the initial navigation coordinates based on these positioning tags. However, since the QR codes attached to the ground are often rubbed by AGV wheels or pedestrians, they are prone to damage and blur after long-term operation, which makes the AGV posture information prone to deviation. At this time, it is necessary to enable the AGV to identify and analyze abnormal QR codes,

[0006] Currently, there are two main types of research on abnormal QR code recognition. One is the recognition of abnormal QR codes in images, including those with motion blur, image rotation, and image distortion; the other is the recognition of QR codes under unusual lighting conditions, including uneven lighting, complex lighting, and reflections. For both of these situations, researchers at home and abroad have proposed various solutions, such as reducing the impact of blurred images caused by camera shake by estimating uniform camera blur, locating distorted or rotated QR codes based on contour tracking and contour screening, correction algorithms based on background grayscale estimation, adaptive thresholding algorithms, and an improved adaptive thresholding method that combines the histogram bimodal method, the Otsu method, and the Niblack thresholding method to obtain the optimal threshold under different lighting conditions.

[0007] Although the above method can improve the processing effect, it also increases the complexity of the algorithm and requires a large number of floating-point operations. This affects the recognition speed of portable QR code readers for AGV positioning, making it difficult to achieve real-time and fast requirements. There is still room for improvement.

[0008] Summary of the Invention

[0009] In view of this, an embodiment of the present application provides a method for identifying damaged QR codes to solve the problem that the existing damaged QR code recognition has large computational complexity and complex algorithms, which affects the recognition speed of the damage rate of the damaged QR code and makes it difficult to achieve real-time and fast requirements.

[0010] In a first aspect, an embodiment of the present application provides a method for identifying a damaged QR code, the method comprising:

[0011] Obtain a sample data set; the sample data set includes a QR code image sample and a QR code label sample;

[0012] Inputting the sample data set into a CenterNet target detection model to be trained, and training the CenterNet target detection model;

[0013] A target damaged QR code to be identified is obtained, and the target damaged QR code is identified using the trained CenterNet target detection model to obtain a damage rate of the target damaged QR code.

[0014] According to the above technical means, the embodiment of the present application uses the trained CenterNet target detection model to identify the target damaged QR code. The damage rate of the obtained target damaged QR code is more accurate, the calculation amount is small, and the algorithm is simple.

[0015] In combination with the first aspect, in one embodiment, the QR code label sample includes at least one of the QR code file name, the QR code file address, the type and location of the defacement pattern contained in the QR code, and the image size of the QR code.

[0016] In combination with the first aspect, in one embodiment, the sample data set is divided into a training sample set, a validation sample set, and a test sample set according to a specified ratio;

[0017] The training sample set is classified into multiple categories according to the contaminated color of the damaged two-dimensional code image, and the category information is recorded.

[0018] In conjunction with the first aspect, in one embodiment, training parameters and specification data of the CenterNet object detection model to be trained are obtained; the training parameters and specification data include at least one of a backbone feature extraction network, pre-training weights, training image size, freezing phase parameters, thawing phase parameters, model confidence, maximum and minimum values ​​of a model learning rate, optimizer parameters, and evaluation parameters;

[0019] The CenterNet target detection model is trained according to the training sample set, the training parameters and the specification data.

[0020] According to the above technical means, by setting the parameters of each model, the accuracy of model training is guaranteed, thereby improving the recognition efficiency and accuracy of damaged QR codes.

[0021] In combination with the first aspect, in one embodiment, an activation function of the CenterNet target detection model is obtained, and the CenterNet target detection model in the training phase is optimized according to the activation function.

[0022] According to the above technical means, the CenterNet target detection model in the training stage is optimized through the activation function to obtain the optimal CenterNet target detection model, thereby ensuring that the progress of the trained CenterNet target detection model is higher.

[0023] In conjunction with the first aspect, in one embodiment, a performance evaluation is performed on the trained CenterNet object detection model based on the verification sample set;

[0024] Based on the test sample set, the performance of the trained CenterNet target detection model is tested.

[0025] According to the above technical means, after the training is completed, the trained CenterNet target detection model is evaluated, and then the CenterNet target detection model with the best performance is selected.

[0026] In conjunction with the first aspect, in one embodiment, target prediction is performed on the verification sample set using the trained CenterNet target detection model to obtain a contamination pattern corresponding to the verification sample set;

[0027] Obtaining a contamination prediction frame on the original two-dimensional code image of the verification sample set according to the contamination pattern;

[0028] Obtaining the mean average precision of the trained CenterNet object detection model based on the contaminated prediction frame and the corresponding contaminated real frame on the original two-dimensional code image of the verification sample set;

[0029] The performance of the trained CenterNet object detection model is evaluated based on the mean average precision.

[0030] Based on the above technical means, the mean average precision can be used to evaluate the recognition effect of the trained CenterNet object detection model to decide whether to retrain the model or select another algorithm. The best-performing model is selected based on the recognition results of the validation sample set to ensure the accuracy of the final damage rate recognition.

[0031] In conjunction with the first aspect, in one embodiment, the average recognition accuracy of the trained CenterNet object detection model for all types of damaged QR codes is obtained based on the degree of overlap between the damaged prediction frame and the corresponding damaged real frame on the original QR code image of the verification sample set;

[0032] According to the recognition average precision, the mean average precision of the trained CenterNet target detection model is obtained.

[0033] Based on the above technical means, the mean average precision can be used to evaluate the recognition effect of the trained CenterNet object detection model to decide whether to retrain the model or select another algorithm. The best-performing model is selected based on the recognition results of the validation sample set to ensure the accuracy of the final damage rate recognition.

[0034] In conjunction with the first aspect, in one embodiment, based on the degree of influence of the damage rate, the distribution of the damage area, the damage color, and the damage recognition algorithm, the damage rate thresholds of QR codes with different damage colors, the damage rate thresholds of QR codes with different damage areas, and the damage rate threshold of the AGV positioning QR code are obtained;

[0035] According to the damage rate thresholds of the QR codes of different damaged colors, the damage rate thresholds of the QR codes of different damaged areas, the damage rate thresholds of the AGV positioning QR code and the damage rate of the target damaged QR code, the target damaged QR code is warned and repaired.

[0036] According to the above technical means, by setting a reasonable QR code breakage rate threshold, it is ensured that when the breakage rate is lower than the QR code breakage rate threshold, the QR code can be successfully identified, and early warning and repair are carried out; thereby improving the robustness and reliability of QR code recognition.

[0037] In conjunction with the first aspect, in one embodiment, a data set comprising different original QR code images, different damage rates, different damage areas, and different damage colors is constructed;

[0038] Creating a detector; the detector is used to identify the data set to record the identification result of each sample in the data set;

[0039] According to the recognition result of each sample, the average values ​​of the damage rates of different damaged colors and different damaged areas are obtained respectively, so as to obtain the damage rate thresholds of the QR codes of different damaged colors and the damage rate thresholds of the QR codes of different damaged areas;

[0040] Obtaining the recognition rate of the damaged image for each damage rate according to the recognition result of each sample;

[0041] The damage rate threshold of the AGV positioning QR code is obtained according to the relationship curve between the recognition rate of the damaged image of each damage rate and the damage rate of the corresponding damaged image.

[0042] According to the above technical means, by setting a reasonable QR code breakage rate threshold, it is ensured that when the breakage rate is lower than the QR code breakage rate threshold, the QR code can be successfully identified, and early warning and repair are carried out; thereby improving the robustness and reliability of QR code recognition.

[0043] In a second aspect, an embodiment of the present application provides a device for identifying a damaged QR code, the device comprising:

[0044] A sample data set acquisition device is used to acquire a sample data set; the sample data set includes a QR code image sample and a QR code label sample;

[0045] A model training device, configured to input the sample data set into a CenterNet target detection model to be trained and train the CenterNet target detection model;

[0046] The damage rate identification device is used to obtain a target damaged QR code to be identified, and identify the target damaged QR code through the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code.

[0047] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned method for identifying a damaged QR code.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned method for identifying a damaged QR code.

[0049] The technical solution provided by this application may have the following beneficial effects:

[0050] This application first obtains a sample data set, inputs the sample data set into the CenterNet target detection model to be trained, and trains the CenterNet target detection model; then obtains the target damaged QR code to be identified, and uses the trained CenterNet target detection model to identify the target damaged QR code to obtain the damage rate of the target damaged QR code. The above scheme uses the trained CenterNet target detection model to identify the target damaged QR code, reducing the amount of calculation and the complexity of the calculation. During the training phase, the model parameters are set, and the CenterNet target detection model is optimized through the activation function; after the training is completed, the performance of the trained CenterNet target detection model is evaluated through the verification sample set, thereby improving the accuracy and efficiency of the recognition results of the CenterNet target detection model.

[0051] In addition, this application sets a reasonable QR code breakage rate threshold to ensure that when the breakage rate is lower than the QR code breakage rate threshold, the QR code can be successfully identified, and early warning and repair can be performed; thereby improving the robustness and reliability of QR code recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] FIG1 is a flowchart showing an application of a method for identifying a damaged two-dimensional code according to an exemplary embodiment.

[0054] FIG2 is a flowchart showing an application of a method for identifying a damaged two-dimensional code according to an exemplary embodiment.

[0055] FIG3 is a schematic diagram of mask images showing an 8% damage rate, a damaged area No. 0, and different damaged colors according to an exemplary embodiment.

[0056] FIG4 is a schematic diagram of a damaged mask image showing a 5% damage rate, 180 contamination colors, and different damaged areas according to an exemplary embodiment.

[0057] FIG5 is a schematic diagram of mask images showing 120 contamination colors, a damaged area No. 0, and different damage rates according to an exemplary embodiment.

[0058] FIG6 is a schematic diagram of target prediction of a CenterNet target detection model according to an exemplary embodiment.

[0059] FIG7 is a schematic diagram of the mean average precision of the CenterNet target detection model according to an exemplary embodiment.

[0060] FIG8 is a schematic diagram of recognition rate curves corresponding to two-dimensional codes with different degrees of damage according to an exemplary embodiment.

[0061] FIG9 is a flowchart showing an application of a method for identifying a damaged two-dimensional code according to an exemplary embodiment.

[0062] FIG10 is a schematic diagram of damaged mask images generated by damaged area No. 2 and different damage rates according to an exemplary embodiment.

[0063] FIG. 11 is a schematic diagram of a damaged mask image generated by different damaged areas with a damage rate of 2% according to an exemplary embodiment.

[0064] FIG. 12 is a schematic diagram showing a mask operation according to an exemplary embodiment.

[0065] Fig. 13 shows a device for identifying a damaged two-dimensional code according to an exemplary embodiment.

[0066] FIG14 shows a structural block diagram of a computer device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solution of this application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0068] It should be understood that the "indication" mentioned in the embodiments of this application can be a direct indication, an indirect indication, or an indication of an association. For example, "A indicates B" can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association between A and B.

[0069] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between the two, or an association relationship between the two, or a relationship between indication and being indicated, configuration and being configured, etc.

[0070] In an embodiment of the present application, "predefinition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including a terminal device and a network device). This application does not limit its specific implementation method.

[0071] Currently, there are two main types of research on abnormal QR code recognition. One is the recognition of QR codes with abnormal images, including motion blur, image rotation, and image distortion; the other is the recognition of QR codes under special lighting conditions, including uneven lighting, complex lighting, and reflections.

[0072] This application generates and identifies damaged QR codes in batches to adapt to problems that may arise in actual application scenarios; these include wear, breakage, missing, etc., and take into account the existence of motion blur, image rotation, image distortion, reflection and other problems. In order to solve the above problems, the QR code recognition technology is summarized and analyzed, and in response to the needs of AGV positioning, this application proposes an efficient and accurate damaged QR code recognition solution; by comprehensively considering the characteristics of QR codes under different degrees of wear, this application performs QR code recognition calculations based on deep learning and evaluates its performance. The results show that this application can better deal with problems such as wear, breakage, missing, and has a certain degree of adaptability; it can improve the efficiency and accuracy of QR code recognition technology for AGV positioning.

[0073] FIG1 is a flowchart of a method for identifying a damaged QR code according to an exemplary embodiment. The method is executed by a computer device. As shown in FIG1 , the identification method may include the following steps:

[0074] Step S101: Acquire a sample data set; the sample data set includes a QR code image sample and a QR code label sample.

[0075] In one possible implementation, in order to identify damaged QR codes, the present application proposes a QR code recognition algorithm based on deep learning. Before model training, it is necessary to obtain a sample data set, and the model training and evaluation are performed using the sample data set. The sample data set of the present application is generated by a computer device based on a damaged code generation algorithm. Compared with the historical sample data used in related technologies, the sample data set of the present application has a larger data volume and is simpler to collect.

[0076] Step S102: input the sample data set into the CenterNet target detection model to be trained, and train the CenterNet target detection model.

[0077] In one possible implementation, the training model used in this application is the CenterNet target detection model; the target detection algorithm in current related technologies usually uses a priori boxes to predict targets, that is, a large number of boxes are set on the picture, and they are adjusted through the network to obtain predicted boxes. Although this method improves the detection ability of the network, it is limited by the size of the object and is not applicable to this field, and its recognition efficiency is low. Unlike this method, the CenterNet target detection model used in this application regards the target as a point, and the detector of the model uses key point estimation to find this center point, and then obtains other target attributes. It is not limited by the size of the object, the application is more flexible, and the recognition efficiency is higher. After determining the sample data set and the CenterNet target detection model to be trained, the CenterNet target detection model to be trained can be trained using the sample data set.

[0078] Step S103: Obtain a target damaged QR code to be identified, and identify the target damaged QR code using the trained CenterNet target detection model to obtain a damage rate of the target damaged QR code.

[0079] In a possible implementation, after the CenterNet target detection model is trained, the target damaged QR code to be identified is input into the trained CenterNet target detection model, and the trained CenterNet target detection model can generate the damage rate corresponding to the target damaged QR code.

[0080] In summary, this application first obtains a sample data set, inputs the sample data set into the CenterNet target detection model to be trained, and trains the CenterNet target detection model; then obtains the target damaged QR code to be identified, and identifies the target damaged QR code through the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code. The above scheme uses the trained CenterNet target detection model to identify the target damaged QR code, reducing the amount of calculation and the complexity of the calculation. During the training phase, the model parameters are set, and the CenterNet target detection model is optimized through the activation function; after the training is completed, the performance of the trained CenterNet target detection model is evaluated through the verification sample set, thereby improving the accuracy and efficiency of the recognition results of the CenterNet target detection model.

[0081] In addition, this application sets a reasonable QR code breakage rate threshold to ensure that when the breakage rate is lower than the QR code breakage rate threshold, the QR code can be successfully identified, and early warning and repair can be performed; thereby improving the robustness and reliability of QR code recognition.

[0082] FIG2 is a flowchart of a method for identifying a damaged QR code according to an exemplary embodiment. The method is executed by a computer device. As shown in FIG2 , the identification method may include the following steps:

[0083] Step S201: Acquire a sample data set; the sample data set includes a QR code image sample and a QR code label sample.

[0084] In a possible implementation, the two-dimensional code image sample includes an original two-dimensional code image and a damaged two-dimensional code image after a mask operation; the damaged two-dimensional code image includes damaged samples with different stain colors, different damaged areas, and different damage rates;

[0085] The QR code label sample includes at least one of the file name of the QR code, the file address of the QR code, the type and position of the defacement pattern contained in the QR code, and the image size of the QR code.

[0086] In a possible implementation, after obtaining the sample data set, the sample data set is divided into a training sample set, a validation sample set, and a test sample set according to a specified ratio;

[0087] According to the contaminated color of the damaged two-dimensional code image, the training sample set is classified into multiple categories, and the type information is recorded.

[0088] Optionally, the sample dataset can be a VOC format sample, including images (i.e., the aforementioned QR code image samples) and labels (i.e., the aforementioned QR code label samples). The QR code image samples are the original QR code image (the intact AprilTag QR code) and the damaged QR code image after the masking operation; the original QR code image corresponds one-to-one to the damaged QR code image after the masking operation.

[0089] For example, please refer to the schematic diagram of the 8% damage rate, damaged area No. 0, and mask images of different damaged colors shown in FIG3. The sample data set of this application selects 10 binary damaged colors, whose RGB values ​​are (255, 255, 255), (230, 230, 230), (210, 210, 210), (180, 180, 180), (150, 150, 150), (120, 120, 120), (90, 90, 90), (60, 60, 60), (30, 30, 30), and (0, 0, 0), which are used to distinguish damaged images of different colors, corresponding to colors 0 to 255 in FIG3 respectively; among them, "0" represents "damaged area No. 0"; "800" represents a damage rate of 8% and 800 damaged black pixels;

[0090] For the damaged area, please refer to Figure 4 showing a schematic diagram of a damaged mask image with a 5% damage rate, 180 stain colors, and different damaged areas; the sample data set of this application selects 10 different damaged areas, from area number 0 to area number 9.

[0091] Regarding the damage rate, please refer to FIG5 , which shows a schematic diagram of 120 stain colors, a damaged area No. 0, and mask images with different damage rates. The sample data set of this application selects 20 different damage rates, ranging from 1% to 20%.

[0092] The above-mentioned QR code label sample can be an XML format file, the content of which mainly includes the file name, file address, image size, and the type and location of the contaminated pattern contained in the corresponding QR code.

[0093] The following is a simple example to illustrate how to obtain the above sample dataset:

[0094] A total of 1,174,000 samples were generated in the embodiment of the present application, including 587 different original QR code images, 20 different damage rates, 10 different damage areas, and 10 different damage colors. Their distribution is shown in Tables 1, 2, and 3, where Table 1 shows the sample distribution of different damage rates; Table 2 shows the sample distribution of different damage areas; and Table 3 shows the sample distribution of different damage colors.

[0095] Table 1

[0096] Table 2

[0097] Table 3

[0098] 90% of the sample data set is randomly selected as the training sample set and the validation sample set, and the remaining 10% is used as the test sample set. That is, the sample data set is divided into the training sample set, the validation sample set, and the test sample set according to the specified ratio. 90% of the training sample set and the validation sample set are the training sample set, and the remaining 10% are the validation sample set.

[0099] Based on the stain color, all training samples are classified into 8 categories, labeled Stain_30, Stain_60, Stain_90, Stain_120, Stain_150, Stain_180, Stain_210, and Stain_230. The type information is written to voc_classes.txt to record the type information.

[0100] According to the above classification method, all sample data sets are divided into test, train, trainval, and val based on the QR code image samples and QR code label samples, where trainval is the training sample set and the verification sample set, test is the test sample set, train is the training sample set, and val is the verification sample set.

[0101] Step S202: input the sample data set into the CenterNet target detection model to be trained, and train the CenterNet target detection model.

[0102] In a possible implementation, step S202 includes:

[0103] Obtaining training parameters and specification data of the CenterNet object detection model to be trained; the training parameters and specification data include at least one of a backbone feature extraction network, pre-training weights, training image size, freezing phase parameters, thawing phase parameters, model confidence, maximum and minimum values ​​of the model learning rate, optimizer parameters, and evaluation parameters;

[0104] The CenterNet target detection model is trained according to the training sample set, the training parameters and the specification data.

[0105] Optionally, this application is based on the damaged QR code recognition of the CenterNet object detection model. The application principle of the CenterNet object detection model is explained in detail below:

[0106] The backbone network of the CenterNet target detection model:

[0107] The CenterNet object detection model uses a variety of backbone feature networks, primarily including the Hourglass Network, the DLANet Network, and the ResNet Network. Optionally, this embodiment uses ResNet as the backbone feature extraction network. ResNet50 is a commonly used convolutional neural network in deep learning, employing two basic blocks: the Conv Block and the Identity Block. The Conv Block contains three convolutional layers: a 1x1 convolutional layer, a 3x3 convolutional layer, and another 1x1 convolutional layer. These three convolutional layers reduce and increase the dimensionality of the input data, while the nonlinear activation function ReLU is added to enhance the network's nonlinear representation capabilities. Following the last 1x1 convolutional layer is a Batch Normalization layer that normalizes the feature maps output by the convolution, helping to improve training stability and network generalization. The Identity Block also consists of three convolutional layers, but lacks dimensionality reduction and increase. The first and third convolutional layers have the same number of output channels as the input channels, while the middle 3x3 convolutional layer has four times the number of output channels as the input channels. The Identity Block enhances the expressiveness of features while maintaining the size of the feature map, deepening and widening the network. Similarly, this block also includes a Batch Normalization layer. Both the Conv Block and the Identity Block are residual network structures.

[0108] Prediction implementation of CenterNet object detection model:

[0109] (1) Obtaining preliminary features

[0110] When the input image is 512*512*3, convolution is first performed on the input image with a convolution kernel size of 7*7, a stride of 2*2, and 64 channels. The input image is then normalized and activated, and then max pooling is performed with a pooling kernel size of 3*3 and a stride of 2*2. Finally, multiple Conv Blocks and Identity Blocks are stacked to obtain an effective feature layer of 16*16*2048.

[0111] (2) Using preliminary features to obtain high-resolution feature maps

[0112] For the last feature layer of Resnet50 obtained in the previous step, the CenterNet target detection model uses a method similar to pyramid pooling to further process and fuse low-resolution feature maps at different scales to obtain a set of feature maps with different sizes and semantic information.

[0113] The CenterNet object detection model then uses these feature maps to predict the location of each object and generate a Gaussian heatmap centered on the object's center. These Gaussian heatmaps are then added to the original low-resolution feature map and upsampled to produce an effective feature layer, or high-resolution feature map, which is used to obtain the final prediction result.

[0114] (3) Center Head obtains prediction results from features

[0115] The Center Head in the CenterNet object detection model is used to obtain information about the location and size of the target from the high-resolution feature map. Specifically, the Center Head extracts features at each location from the high-resolution feature map, processes them through several consecutive convolutional layers, and outputs three matrices: objectness maps, center-ness maps, and regression maps.

[0116] Among them, objectness maps is a binary matrix used to indicate whether there is an object at that location; center-ness maps is a scalar matrix used to evaluate whether the location is the center point of the object; regression maps is a two-dimensional vector matrix used to predict the distance from the center point of the object to the upper left corner and the lower right corner of the corresponding bounding box.

[0117] In its implementation, the Center Head first performs convolution operations and activation functions, splitting the results into three components: objectness, centerness, and regression. The Center Head then employs specialized network structures to further extract features for each component. For objectness and centerness, the Center Head uses a convolutional layer and a fully connected layer, respectively. For regression, two convolutional layers are used, with a sigmoid activation function added to the final layer to constrain the result to a range between 0 and 1.

[0118] Finally, Center Head calculates the coordinates of the upper left and lower right corners of the predicted bounding box based on the generated objectness maps, center-ness maps, and regression maps, and transforms them into the original image to obtain the target detection result on the original image.

[0119] (4) Decoding of prediction results

[0120] In the CenterNet object detection model, the prediction decoding process consists of two steps: non-maximum suppression (NMS) and decoding. Its purpose is to eliminate duplicate detections and convert Gaussian heatmaps into bounding boxes. First, in the non-maximum suppression (NMS) stage, for each object center point, it is necessary to retain the output with the maximum response and filter out other redundant outputs with low confidence. The specific implementation of this step usually adopts the following process:

[0121] a) Sort all center points by objectness scores from high to low;

[0122] b) Select the center point with the highest score and retain the target prediction box formed by this point as the output result;

[0123] c) Remove all other center points and their corresponding target boxes whose IoU (intersection over union) with the selected prediction box is greater than a given threshold;

[0124] d) Repeat the above process, select the center point with the second highest score, and retain the target prediction box formed by this point as the output result;

[0125] e) Until all center points are processed, all retained prediction boxes are obtained;

[0126] Secondly, during the decoding phase, for each retained prediction box, the Gaussian heatmap is converted into a bounding box. Specifically, for each retained center point, a bounding box is calculated using its spatial position and the predicted regression offset. Following this prediction method, a set of prediction boxes can be obtained for each category.

[0127] Training implementation of CenterNet target detection model:

[0128] (1) Real frame processing

[0129] In the CenterNet target detection model, the processing of the real box includes the following steps:

[0130] 1. For each training sample, first determine the actual target bounding box (ground-truth boxes); this can be achieved by calculating the IoU (intersection / union) between the ground-truth box and the output heat map. If the IoU between a ground-truth bounding box and the heat map is greater than a preset threshold, the target is considered to exist at that location and further processing is performed;

[0131] 2. After determining which objects exist, these objects need to be mapped to specific high-resolution feature maps. At this point, a regression module is applied to calculate the center point offset and width and height scaling factors, and the corresponding Gaussian heat map is trained together with the output to improve model accuracy. Specifically, the target center point coordinates are mapped to the feature map coordinate system. Then, for each object, a Gaussian function is used to generate a heat map at a specific center point, where the variance of the Gaussian function is related to the object size;

[0132] 2. After generating the Gaussian heatmap, we need to calculate and generate regression targets. Specifically, for each target, we need to calculate the offset from the target center point to the heatmap grid and the width and height scaling factors. These values ​​will be used to train the regression network.

[0133] (2) Loss value calculation

[0134] The loss value loss includes heat map loss, reg center point loss, and wh width and height loss.

[0135] Find the center of the true frame, and find the corresponding feature point through the center of the true frame. The heat map loss adopts the Focal loss idea:

[0136] Among them, L k Represents the confidence loss of the target center point prediction, α and β are the hyperparameters of the Focal function; optionally, α and β are 2 and 4 respectively; N is the number of positive samples (positive samples are predicted boxes and the overlap between the real box is greater than the target value), which is used for normalization. is the predicted value, Y xyc is the true value.

[0137] Like CornerNet, in order to eliminate the deviation of the predicted coordinate results caused by sampling, the CenterNet target detection model also predicts and outputs the offset of each coordinate position:

[0138] Among them, L off is the offset regression loss; N is the number of positive samples (positive samples are those where the overlap between the predicted box and the true box is greater than the target value); represents the center coordinate of the predicted target, Indicates the offset of the prediction result, represents the true coordinate position of the target after downsampling, p represents the true coordinate position of the target in the original image, and R represents the downsampling multiple.

[0139] The Reg center point loss and wh width and height loss are obtained using the common L1 loss function through the following formula:

[0140] Among them, L size is the regression loss of the target width and height prediction results; N is the number of positive samples (positive samples are those where the overlap between the predicted box and the true box is greater than the target value), is the predicted value, S k is the true value.

[0141] The total loss value is obtained by the following formula: det =L k +λ size L size +λ off L off ;

[0142] Among them, L det Represents the total loss value, L k represents the confidence loss of target center point prediction, λ size and λ off are coefficients, L size represents the regression loss of target width and height prediction results, L off represents the offset regression loss.

[0143] Based on the above principles, this embodiment inputs the sample data set into the CenterNet target detection model to be trained and trains the CenterNet target detection model.

[0144] During the training process, it is necessary to confirm the training parameters and specification data of the CenterNet target detection model to be trained. First, for the backbone feature extraction network, pre-training weights and training image size, this embodiment uses ResNet50 as the backbone feature extraction network, centernet_resnet50_voc.pth as the pre-training weights, and the training image size is 512*512.

[0145] Secondly, regarding the parameters in the freezing and thawing phases, this application divides the training process of the CenterNet object detection model into two phases: the first phase is the freezing phase, and the second phase is the thawing phase. During the freezing phase, the backbone of the CenterNet object detection model is frozen, and the backbone feature extraction network remains unchanged. Due to the small number of model training parameters, the CenterNet object detection model occupies less video memory at this time, and only fine-tuning of the network is performed. The initial training epoch Init_Epoch is set to 0, the number of frozen training iterations Freeze_Epoch is set to 500, the Freeze_batch_size is set to 16, and the learning rate Freeze_lr is set to 1e-3.

[0146] In the unfreezing stage, the backbone feature extraction network of the CenterNet target detection model is not frozen, and the feature extraction network will change. At this time, the model training parameters are large, the CenterNet target detection model occupies a large amount of video memory, and all network parameters will change; set the total number of training iterations UnFreeze_Epoch to 1000, UnFreeze_batch_size to 8, and the learning rate UnFreeze_lr to 1e-4.

[0147] Regarding the maximum and minimum values ​​of the model confidence and the model learning rate, the confidence of the CenterNet target detection model is set to 0.3, and only the prediction boxes with scores greater than the confidence are retained; the nms_iou size used for non-maximum suppression is set to 0.3; the maximum learning rate Init_lr of the CenterNet target detection model is set to 5e-4, and the minimum learning rate Min_lr of the CenterNet target detection model is set to 0.01 times the maximum learning rate.

[0148] For the optimizer parameters, the CenterNet target detection model uses the adam optimizer, the momentum parameter used in the optimizer is set to 0.9, the weight decay weight_decay is set to 0, and the learning rate descent method lr_decay_type is set to cos.

[0149] Regarding the evaluation parameters, the CenterNet object detection model is set to save weights every 10 generations and perform an evaluation every 50 generations. The mAP obtained here is the mAP of the validation set. The evaluation parameters are set conservatively to speed up the evaluation.

[0150] Step S203: Obtain an activation function of the CenterNet target detection model, and optimize the CenterNet target detection model in the training phase according to the activation function.

[0151] Optionally, to improve the speed and recognition accuracy of the CenterNet object detection model, this embodiment uses a ResNet network as the model's backbone feature extraction network. ResNet is a deep neural network used for image classification that can increase network depth and improve network model accuracy without increasing network complexity.

[0152] The ResNet network has network structures of different depths, such as 18, 34, 50, 101, and 152 layers. Taking into account the network depth, the number of floating-point operations, and the error rate on the ImageNet database, this embodiment uses a 50-layer deep neural network ResNet as the backbone network and as the backbone network extraction network to extract the features of the QR code.

[0153] In order to use nonlinear functions to describe the features extracted by the neural network, the neural network needs to be followed by a nonlinear activation function after the affine transformation. Common activation functions of neural networks include Sigmoid function, Tanh function, ReLU function, LeakyReLU, etc. Optionally, the activation function of the CenterNet target detection algorithm used in this embodiment is the ReLU function. The expression of the ReLU function is as follows: y = max(0, x);

[0154] Among them, x represents the input weight of the neuron, and y represents the actual weight used by the neuron.

[0155] The ReLU function can effectively solve the problem of gradient disappearance, and has fast convergence speed and efficient computing power. It can also avoid the interference of useless information, introduce sparsity to the network, and further improve computing efficiency.

[0156] Step S204: Perform a performance evaluation on the trained CenterNet object detection model based on the validation sample set in the sample data set.

[0157] In a possible implementation, step S204 includes:

[0158] Step S2041: Perform target prediction on the verification sample set using the trained CenterNet target detection model to obtain a contamination pattern corresponding to the verification sample set;

[0159] Step S2042: Obtain a contamination prediction frame on the original two-dimensional code image of the verification sample set according to the contamination pattern;

[0160] Step S2043: Obtain the mean average precision of the trained CenterNet object detection model based on the defacement prediction frame and the corresponding defacement real frame on the original QR code image of the verification sample set;

[0161] Step S2044: Perform a performance evaluation on the trained CenterNet target detection model based on the mean average precision.

[0162] In a possible implementation, step S2043 includes:

[0163] Based on the overlap between the predicted damaged frame and the corresponding real damaged frame on the original QR code image of the verification sample set, the average recognition accuracy of the trained CenterNet object detection model for all types of damaged QR codes is obtained;

[0164] According to the recognition average precision, the mean average precision of the trained CenterNet target detection model is obtained.

[0165] Optionally, the following is a detailed description of the performance evaluation definition of the CenterNet object detection model:

[0166] In the field of machine learning, accuracy, recall, precision, and F1 Score are often used to evaluate the performance of classification models:

[0167] (1) Accuracy

[0168] The accuracy rate represents the ratio of the number of samples correctly classified by the classifier to the total number of samples. Its calculation formula is:

[0169] Among them, True Positive represents true positive examples (referring to the number of samples correctly predicted as positive examples by the model, that is, the number of samples predicted by the model as positive examples and the true label is also positive); True Negative represents true negative examples (referring to the number of samples incorrectly predicted as positive examples by the model, that is, the number of samples predicted by the model as positive examples but the true label is negative); False Positive represents false positive examples (referring to the number of samples incorrectly predicted as positive examples by the model, that is, the number of samples predicted by the model as positive examples but the true label is negative); False Negative represents false negative examples (referring to the number of samples incorrectly predicted as negative examples by the model, that is, the number of samples predicted by the model as negative examples but the true label is positive).

[0170] When the number of categories in the sample dataset is similar, accuracy is a good way to measure model performance. In some cases, such as when the categories are unbalanced in a binary classification problem, accuracy may not be applicable because only by considering other performance indicators can the model performance be more comprehensively evaluated.

[0171] (2) Recall

[0172] Recall is also called recall rate, which represents the ratio of the number of positive class samples correctly predicted by the classifier to the total number of positive instance samples. Its calculation formula is:

[0173] When the CenterNet target detection model is required to classify all true positive examples as positive examples, the recall rate is also more important.

[0174] (3) Precision

[0175] Precision refers to the ratio of the number of positive class samples correctly predicted by the classifier to the total number of samples predicted as positive instances. Its calculation formula is:

[0176] (4)F1 Score

[0177] F1 Score is a trade-off between precision and recall, and is the harmonic mean of these two indicators. Its calculation formula is:

[0178] It is generally believed that F1 Score is more suitable for situations where the sample categories are unbalanced or the ratio of positive and negative samples is large compared to accuracy.

[0179] Based on this, this application evaluates the performance of the trained CenterNet target detection model. First, the target prediction of the CenterNet target detection model is performed:

[0180] After the CenterNet target detection model training is completed, the detect_image function of the CenterNet target detection model is used to perform target prediction on the verification sample set. After detecting the stain pattern of the verification sample set, a prediction box is drawn on the original QR code image, and the area of ​​the stain prediction box is divided by the total area as the predicted damage rate. At this time, please refer to the target prediction diagram of the CenterNet target detection model shown in Figure 6. As shown in Figure 6, Stain_60 (the classification of stain color) is identified as damage, and its confidence is 0.79.

[0181] Optionally, the MINOVERLAP (minimum overlap) set in this embodiment is 0.5, and the prediction confidence confidence is set to 0.02; when the overlap between a certain prediction box and the true box is greater than MINOVERLAP, the prediction box is considered to be a positive sample, otherwise it is a negative sample. In addition, the size of the non-maximum suppression value nms_iou used in the prediction is set to 0.5. The larger the value, the less strict the non-maximum suppression; and since this embodiment uses the mean average precision (mAP) to evaluate the performance of the trained CenterNet target detection model, because calculating mAP requires obtaining almost all prediction boxes, the prediction confidence confidence defined above cannot be changed at will, so this embodiment specifically defines a score_threhold to represent the threshold value, and then finds the Recall and Precision values ​​corresponding to the threshold value when calculating mAP. The threshold value score_threhold set in this embodiment is 0.5.

[0182] Regarding mean average precision (mAP), when applied to target detection tasks, mAP is usually one of the important indicators for evaluating the quality of detection algorithms. Therefore, this example uses mAP to evaluate the performance of the CenterNet target prediction model in the AprilTag QR code damage rate recognition task.

[0183] mAP is the average of all class mean average precision (AP). For each class, average precision (AP) is a measure of accuracy that takes into account the degree of match (intersection over union) between the predicted and true labels. Therefore, it effectively measures model performance such as accuracy, recall, and precision. mAP can be calculated by adding up the AP values ​​for all classes and dividing by the number of classes.

[0184] The entire mean average precision (mAP) calculation process includes obtaining prediction results, obtaining true bounding boxes, and calculating VOC_mAP. At this point, please refer to the mean average precision diagram of the CenterNet object detection model shown in Figure 7. Figure 7 follows the above example and evaluates 117,400 samples to obtain the average precision (AP) of the eight types (Stain_30, Stain_60, Stain_90, Stain_120, Stain_150, Stain_180, Stain_210, and Stain_230) classified by stain color. Finally, the average of the eight types of average precision (AP) is taken to obtain a mean average precision (mAP) of 98.95% for the CenterNet object detection model, indicating that this CenterNet object detection model has high performance in identifying the damage rate of AprilTag QR codes.

[0185] Step S205: Perform a performance test on the trained CenterNet object detection model based on the test sample set in the sample data set.

[0186] Optionally, the validation sample set is used to evaluate the effectiveness of the trained model and decide whether to retrain the model or choose another algorithm. The best-performing model is selected based on the results of the validation sample set. The implementation steps of the test sample set are the same as those of the validation sample set. It is used to perform performance testing on the trained CenterNet object detection model, that is, to test the model's stability, robustness, and generalization error. The test sample set does not participate in the establishment and preparation of any model.

[0187] Step S206: Obtain a target damaged QR code to be identified, and identify the target damaged QR code using the trained CenterNet target detection model to obtain a damage rate of the target damaged QR code.

[0188] Step S207: Based on the damage rate impact, damage area distribution, damage color, and damage recognition algorithm, obtain the damage rate thresholds for QR codes with different damage colors, the damage rate thresholds for QR codes with different damage areas, and the damage rate threshold for AGV positioning QR codes.

[0189] In a possible implementation, step S207 includes:

[0190] Construct a dataset containing different original QR code images, different damage rates, different damage areas, and different damage colors;

[0191] Create a detector; the detector is used to identify the data set to record the identification result of each sample in the data set;

[0192] According to the recognition result of each sample, the average damage rate of different damage colors and different damage areas is obtained respectively, so as to obtain the damage rate threshold of the QR code with different damage colors and the damage rate threshold of the QR code with different damage areas;

[0193] According to the recognition result of each sample, the recognition rate of the damaged image of each damage rate is obtained;

[0194] The AGV positioning QR code damage rate threshold is obtained based on the relationship curve between the recognition rate of the damage image of each damage rate and the damage rate of the corresponding damage image.

[0195] Step S208: Based on the damage rate thresholds of the QR codes of different damaged colors, the damage rate thresholds of the QR codes of different damaged areas, the damage rate thresholds of the AGV positioning QR code, and the damage rate of the target damaged QR code, an early warning and repair of the target damaged QR code are performed.

[0196] Optionally, the following details the QR code damage rate threshold:

[0197] The QR code damage rate threshold refers to the threshold at which recognition of a QR code fails when the degree of damage to the QR code image reaches a certain threshold. This threshold typically varies depending on the application scenario and the characteristics of the recognition algorithm. When a QR code is damaged, some information may be missing, distorted, or blurred, affecting recognition performance. To improve the robustness and reliability of QR code recognition, it is necessary to set a reasonable damage rate threshold to ensure that the QR code can be successfully recognized when the damage rate is below the threshold.

[0198] Since the QR code itself has a certain error correction mechanism, a certain degree of damage and contamination will not affect the normal operation of the QR code; however, as the damage rate of the QR code gradually increases, the QR code will inevitably become damaged and unrecognizable; the minimum damage rate at which the QR code is damaged and unrecognizable is the QR code damage rate threshold.

[0199] In practical applications, selecting a QR code damage rate threshold often requires consideration of multiple factors, such as the damage rate's impact on QR code recognition, the distribution of the damaged areas, the damage color, and the characteristics of the recognition algorithm used. Sometimes, to achieve higher QR code recognition accuracy, multiple different damage rate thresholds can be used to process the QR code image and their recognition results can be combined to improve overall recognition performance.

[0200] This application uses the QR code breakage rate threshold for QR code breakage warning in AGV positioning. When the AGV recognizes that the QR code is close to the breakage rate threshold, it can notify the maintenance personnel in advance to maintain the QR code to avoid the system from not being able to operate normally due to the inability to recognize the QR code.

[0201] If the damage rate threshold is set too low, maintenance personnel will need to frequently maintain the QR code, resulting in increased labor costs and system downtime. If the damage rate threshold is set too high, the QR code may be damaged and unrecognizable, but no warning will be issued, causing system downtime and affecting system efficiency.

[0202] Based on this, this application proposes an algorithm for the QR code damage rate threshold. This application uses the pupil_apriltags library in the Python language to identify and judge AprilTag QR codes. This library is an AprilTag QR code processing function library; the calculation steps include:

[0203] (1) Calculate the damage rate thresholds of QR codes with different damage colors and QR code damage rate thresholds with different damage areas;

[0204] To obtain damage rate thresholds for AprilTag QR codes with different damage colors and damage areas, this embodiment takes the following steps:

[0205] 1. Construct a dataset containing labels for different original QR codes, different damage rates, different damage areas, and different damage colors. For example, this example generates 1,174,000 samples, including 587 different original QR code images, 20 different damage rates, 10 different damage areas, and 10 different damage colors.

[0206] 2. Use the apriltag.Detector function to create an AprilTag detector, use the detector to identify all data sets, and record the recognition results of each sample;

[0207] 3. Record the minimum damage rate at which the QR code cannot be recognized for different damage colors and different damage areas, and calculate the average value to obtain the damage rate threshold of the QR code for different damage colors and different damage areas.

[0208] The QR code damage thresholds corresponding to different damage colors are shown in Table 4:

[0209] Table 4

[0210] The QR code damage rate thresholds corresponding to different damage areas are shown in Table 5:

[0211] Table 5

[0212] (2) AGV positioning QR code damage rate threshold;

[0213] In order to calculate the AprilTag QR code damage rate threshold in AGV positioning and implement QR code maintenance warning in AGV positioning, this application adopts the following calculation method:

[0214] 1. Construct a dataset containing labels for different original QR codes, different damage rates, different damage areas, and different damage colors. For example, this example generates 1,174,000 samples, including 587 different original QR code images, 20 different damage rates, 10 different damage areas, and 10 different damage colors.

[0215] 2. Use the apriltag.Detector function to create an AprilTag detector, use the detector to identify all data sets, and record the recognition results of each sample;

[0216] 3. For each damaged image with a damage rate, calculate the proportion of correctly identified images to obtain the accuracy rate at that damage level.

[0217] 4. Draw a recognition rate (accuracy)-damage rate curve and select an appropriate threshold value so that when the accuracy rate is below this threshold, it is considered that the QR code with this damage rate cannot be correctly identified. Please refer to the recognition rate curve diagram corresponding to QR codes with different damage levels shown in Figure 8. The horizontal axis of the recognition rate (accuracy)-damage rate curve is the damage rate, and the vertical axis is the recognition rate. As can be seen from Figure 8, based on the requirements for QR code accuracy and the consideration of the false recognition rate in AGV positioning, the accuracy threshold selected in this embodiment is 70%, that is, the damage rate with an identification accuracy rate below 70% is considered to be a damage rate that cannot be accurately identified. Therefore, the damage rate threshold that needs to be set for the calculated damaged QR code recognition model based on AGV positioning technology is 7.54%.

[0218] In summary, this application first obtains a sample data set, inputs the sample data set into the CenterNet target detection model to be trained, and trains the CenterNet target detection model; then obtains the target damaged QR code to be identified, and identifies the target damaged QR code through the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code. The above scheme uses the trained CenterNet target detection model to identify the target damaged QR code, reducing the amount of calculation and the complexity of the calculation. During the training phase, the model parameters are set, and the CenterNet target detection model is optimized through the activation function; after the training is completed, the performance of the trained CenterNet target detection model is evaluated through the verification sample set, thereby improving the accuracy and efficiency of the recognition results of the CenterNet target detection model.

[0219] In addition, this application sets a reasonable QR code damage rate threshold to ensure that when the damage rate is lower than the QR code damage rate threshold, the QR code is successfully identified, and early warning and repair are performed; thereby improving the robustness and reliability of QR code recognition.

[0220] FIG9 is a flowchart illustrating a method for identifying a damaged QR code according to an exemplary embodiment. In actual application, the method uses the original QR code image (AprilTag QR code) and the damaged image corresponding to the original QR code image after masking as a sample data set to train the CenterNet target detection model to be trained, and the trained CenterNet target detection model is used to recognize the target damaged QR code; the method is executed by a computer device. As shown in FIG9, the method may include the following steps:

[0221] Step S901: Obtain the original QR code image.

[0222] In a possible implementation, step S201 includes: generating a plurality of original two-dimensional code images; the original two-dimensional code images include coding information and graphic information; and the original two-dimensional code images are used to indicate location information.

[0223] Optionally, the original QR code image of this application is an AprilTag QR code, which has higher real-time performance, robustness, and reliability. Compared with the QR code in the related art, the AprilTag QR code has the following characteristics:

[0224] The AprilTag QR code is positioned differently within an image than a QR code. While a QR code uses a special positioning pattern to identify the QR code's position and orientation, the AprilTag uses an internal black and white pattern to identify its position and orientation. This design enables the AprilTag to reliably identify and locate the QR code at varying angles, distances, and lighting conditions.

[0225] The encoding method of the AprilTag QR code is also different from that of the QR code. QR codes use an error detection and correction technology based on Hamming codes to ensure the reliability of data transmission, while the AprilTag QR code uses a compression encoding technology based on hash functions, which can store more information in a small storage space and can also be decoded quickly.

[0226] AprilTag QR codes also feature graduated resolution. At long distances or in low-resolution situations, larger tag sizes can be used for identification, which is one of its advantages over QR codes.

[0227] This application generates a total of 587 AprilTag QR codes as original images, with a size of 100*100 pixels.

[0228] Step S902: Acquire damage information of the original two-dimensional code image; the damage information includes at least one of a damage rate, a damage area, and a damage color.

[0229] In one possible implementation, in practical application scenarios, since damage rate analysis of actual QR code images is required, it is desirable to be able to directly use the input QR code image to quickly determine its quality. Therefore, the core issue is how to achieve reference-free evaluation of QR code image quality using only the input image. Therefore, this application introduces a masking operation to generate corresponding masks based on different damage rates, damage areas, damage colors, and so on. The damage information in this application is preset based on actual needs. Compared to conventional QR code recognition reference images, the mask image in this application can be considered a pseudo-reference image, meaning that the usability information contained in the image is almost negligible. For any input image, in the absence of a high-quality reference, a pseudo-reference mask image can be used as a comparison benchmark. The smaller the difference with the mask image, the lower the usability quality, while the larger the difference, the higher the usability quality. This allows for reference-free evaluation of the QR code image using only the input QR code image, resulting in a usability quality representation of the QR code image and, consequently, the actual QR code damage rate.

[0230] Step S903: According to the damage information, determine the starting damage point from the initial mask image, and add the starting damage point to the target damage array; the initial mask image is an image with a white background.

[0231] Step S904: continuously obtain neighboring pixel points of the starting damage point, and add the neighboring pixel points to the target damage array until the number of elements in the target damage array meets the target condition.

[0232] In one possible implementation, step S904 includes: taking the starting damage point as the center, continuously acquiring neighboring pixel points of the starting damage point around the starting damage point, and adding the neighboring pixel points of the starting damage point to the target damage array until the number of elements in the target damage array meets the target threshold.

[0233] In a possible implementation, after the initial damage point is determined from the initial mask image, the initial damage point is set to black;

[0234] After continuously acquiring neighboring pixels of the initial damaged point, the neighboring pixels are set to black.

[0235] Step S905: constructing a damaged mask image corresponding to the original two-dimensional code image according to the initial mask image and the target damaged array; the target damaged array indicates the damaged area.

[0236] Optionally, since damaged patterns often have clustered characteristics, the present application adopts the following method to construct a damaged mask image; first, for a 100*100 pixel white background image (that is, the initial mask image of the white background mentioned above), where the RGB value of each pixel is (255, 255, 255), the white background image has a total of 10,000 pixels.

[0237] Afterwards, based on the damage information (such as the preset damage rate or damage area), randomly select one pixel on the white background image as the starting damage point, set the RGB value of the starting damage point to black (0, 0, 0), and add it to the black_pixels array (i.e., the target damage array). Since the target damage array represents the size of the damage area, based on the preset damage rate or damage area, if the number of black pixels in the black_pixels array is less than the target threshold,

[0238] Repeat the following steps: Randomly select a pixel in the black_pixels array (prioritizing the initial damage point, searching for its neighbors first, then searching for its neighbors). During the search, randomly select a neighboring pixel from the top, bottom, left, or right directions and visit that neighboring pixel. If the neighboring pixel is not in the black_pixels array, set its RGB value to black (0, 0, 0) and add it to the black_pixels array. If the number of elements in the black_pixels array meets the target threshold, save the image at that point. If the neighboring pixel is already in the black_pixels array, continue the above steps. Since a small damage rate is insignificant, a damage rate of 1% is a suitable range. Therefore, when presetting the damage rate in the damage information, it is usually set to above 1%. Therefore, when setting the target threshold, ensure that the number of elements in the black_pixels array is divisible by 100.

[0239] Furthermore, since the identification of damaged QR codes has a certain threshold, when the damage rate is too high, the QR code is no longer meaningful for damage rate calculation. Therefore, this application optionally sets a default value of 2000 black pixels, indicating an expected maximum damage rate of 20% and a damage rate interval of 1%. Therefore, when setting the target threshold, it is necessary to ensure that the number of elements in the black_pixels array does not exceed 2000.

[0240] At this time, please refer to the schematic diagram of the damaged mask image generated by the damaged area No. 2 and different damage rates shown in Figure 10, and the schematic diagram of the damaged mask image generated by the damaged area with a damage rate of 2% and different damage areas shown in Figure 11; as shown in Figure 10, Figure 10 shows the damaged mask image generated when the number of elements in the black_pixels array ranges from 100 to 1000; that is, the damaged mask image generated when the preset damage rate is from 1% to 10%, and the "2" in Figure 3 refers to the "damaged area No. 2"; Figure 11 shows the damaged mask image generated when the number of elements in the black_pixels array is 200 and the damaged area ranges from "damaged area No. 0" to "damaged area No. 9"; that is, the preset damage rate is 2%, and the damaged area ranges from "damaged area No. 0" to "damaged area No. 9".

[0241] Step S906: Obtain a damaged image corresponding to the original two-dimensional code image according to each pixel value of the damaged area of ​​the damaged mask image.

[0242] In a possible implementation, step S906 includes: sequentially calculating the weighted average values ​​of each pixel point in the damaged area of ​​the damaged mask image, and replacing the pixel values ​​of the corresponding areas in the original QR code image and the damaged area of ​​the damaged mask image according to the weighted average values ​​to obtain a damaged image corresponding to the original QR code image.

[0243] Optionally, the present application adopts two methods to obtain the damaged image corresponding to the original QR code image. The first is a filtering-based image processing method; it uses the damaged mask image and the pixels in the original QR code image to perform operations one by one to obtain a weighted average or other specific mathematical statistics, and then uses the calculated result as a new pixel value to replace the original pixel value, thereby achieving image changes and optimization.

[0244] In a specific application, when the mask template of this embodiment (i.e., the damaged mask image described above) is applied to the original QR code image, the center of the mask template is aligned with the center of the original QR code image. A weighted average of the pixels within the damaged area of ​​the mask template is calculated, and the calculated result is used as a new pixel value to replace the corresponding pixel value in the original QR code image. In this way, each pixel in the damaged area will receive a new pixel value based on the different characteristics of its surrounding pixels. This implements damage mask processing of the original QR code image, thereby obtaining a damaged image.

[0245] In a possible implementation, step S906 further includes: performing a bitwise AND operation on each pixel of the original QR code image and the damaged mask image corresponding to the original QR code image, and saving the operation result to an output image, which is the damaged image corresponding to the original QR code image.

[0246] Optionally, the second way to obtain the damaged image corresponding to the original QR code image is to process the mask operation of the original QR code image based on the Python language to generate a damaged image; the cv2.bitwise_and() function is a bitwise AND operation function in the OpenCV library; when using the cv2.bitwise_and() function, its input parameters are two binary images to be processed, namely the original QR code image and the damaged mask image. The cv2.bitwise_and() function will perform a bitwise AND operation on each pixel on the two binary images and save the result to the output image. When performing a bitwise AND operation, only when the pixel values ​​of the pixels at the same position on the two input binary images are both 255 (that is, each bit of the binary number is 1), the pixel value at the corresponding position on the output image will be set to 255, otherwise it will be set to 0. Use the cv2.bitwise_and() function to multiply the original QR code image and the damage mask image to generate a damaged QR code that has been masked. Please refer to the mask operation diagram shown in Figure 12. As shown in Figure 12, the original QR code image is processed by the 600_10_mask damage mask to obtain a damaged image, where 600 represents that the damage mask has 600 damaged pixels, that is, a damage rate of 6%, and 10 represents that the damaged area of ​​the damage mask is number 10.

[0247] Step S907: Acquire a sample data set; the sample data set includes a QR code image sample and a QR code label sample.

[0248] Step S908: Input the sample data set into the CenterNet target detection model to be trained, and train the CenterNet target detection model.

[0249] Step S909: Obtain the activation function of the CenterNet target detection model, and optimize the CenterNet target detection model in the training phase according to the activation function.

[0250] Step S910: Perform performance evaluation on the trained CenterNet object detection model based on the verification sample set in the sample data set.

[0251] Step S910: Perform a performance test on the trained CenterNet target detection model based on the test sample set in the sample data set.

[0252] Step S911: Obtain a target damaged QR code to be identified, and identify the target damaged QR code using the trained CenterNet target detection model to obtain a damage rate of the target damaged QR code.

[0253] Step S912: Based on the damage rate impact, damage area distribution, damage color, and damage recognition algorithm, obtain the damage rate thresholds for QR codes with different damage colors, the damage rate thresholds for QR codes with different damage areas, and the damage rate threshold for AGV positioning QR codes.

[0254] Step S913: Based on the damage rate thresholds of the QR codes of different damaged colors, the damage rate thresholds of the QR codes of different damaged areas, the damage rate thresholds of the AGV positioning QR code, and the damage rate of the target damaged QR code, an early warning and repair of the target damaged QR code are performed.

[0255] In summary, the core issue of this application is how to identify damaged and unusable QR codes in AGV positioning technology, and promptly determine whether maintenance is necessary. To address this challenge, this application provides an in-depth summary and analysis of QR code recognition and detection technologies, as well as AGV positioning technologies, and proposes a feasible method for identifying damaged QR codes for AGV positioning. Its efficiency and feasibility are verified through experiments. Furthermore, this method has good adaptability and can effectively address issues that may arise in practical applications, such as wear, tear, and missing codes.

[0256] The research results of this application provide an efficient and accurate damaged QR code recognition solution for AGVs based on QR code positioning in SLAM real-time positioning and map building technology, and have certain theoretical and practical significance. This application verifies the feasibility and efficiency of this method through experiments. Therefore, this application can provide an efficient and accurate damaged QR code recognition solution for AGVs based on QR code positioning in SLAM real-time positioning and map building technology.

[0257] This application uses the CenterNet target detection model as the training model and the ResNet50 network as the model backbone feature extraction network to train the AprilTag QR code after damage mask processing.

[0258] In the above scheme, this application first generates all AprilTag QR codes (i.e., original QR code images), and then generates a variety of damaged mask images for different damage rates, different damaged areas, and different damaged colors; then uses the CenterNet target detection model to train the damaged images after mask processing, and uses the ResNet network to replace the network in the original model as the model backbone feature extraction network, and also performs performance evaluation on the trained CenterNet target detection model; finally, the QR code damage thresholds corresponding to different damage colors and different damage areas and the QR code damage rate threshold based on AGV positioning technology are calculated respectively.

[0259] FIG13 is a diagram showing a device for identifying a damaged QR code according to an exemplary embodiment, the device comprising:

[0260] The sample data set acquisition module 1301 is used to acquire a sample data set; the sample data set includes a QR code image sample and a QR code label sample;

[0261] A model training module 1302 is configured to input the sample data set into a CenterNet object detection model to be trained and train the CenterNet object detection model;

[0262] The damage rate identification module 1303 is used to obtain a target damaged QR code to be identified, and identify the target damaged QR code using the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code.

[0263] In a possible implementation, the two-dimensional code image sample includes an original two-dimensional code image and a damaged two-dimensional code image after a mask operation; the damaged two-dimensional code image includes damaged samples with different stain colors, different damaged areas, and different damage rates;

[0264] The QR code label sample includes at least one of the file name of the QR code, the file address of the QR code, the type and position of the defacement pattern contained in the QR code, and the image size of the QR code.

[0265] In one possible embodiment, the device is further used for:

[0266] Divide the sample data set into a training sample set, a validation sample set, and a test sample set according to the specified ratio;

[0267] According to the contaminated color of the damaged two-dimensional code image, the training sample set is classified into multiple categories, and the type information is recorded.

[0268] In one possible implementation, the model training module 1302 is further configured to:

[0269] Obtaining training parameters and specification data of the CenterNet object detection model to be trained; the training parameters and specification data include at least one of a backbone feature extraction network, pre-training weights, training image size, freezing phase parameters, thawing phase parameters, model confidence, maximum and minimum values ​​of the model learning rate, optimizer parameters, and evaluation parameters;

[0270] The CenterNet target detection model is trained according to the training sample set, the training parameters and the specification data.

[0271] In one possible embodiment, the device is further used for:

[0272] Obtain the activation function of the CenterNet target detection model and optimize the CenterNet target detection model in the training phase according to the activation function.

[0273] In one possible embodiment, the device is further used for:

[0274] Based on the validation sample set, the performance of the trained CenterNet target detection model is evaluated;

[0275] Based on the test sample set, the performance of the trained CenterNet target detection model is tested.

[0276] In one possible embodiment, the device is further used for:

[0277] Perform target prediction on the verification sample set using the trained CenterNet target detection model to obtain the contamination pattern corresponding to the verification sample set;

[0278] Obtaining a contamination prediction frame on the original two-dimensional code image of the verification sample set according to the contamination pattern;

[0279] Obtain the mean average precision of the trained CenterNet object detection model based on the contaminated prediction box and the corresponding contaminated real box on the original QR code image of the verification sample set;

[0280] The performance of the trained CenterNet target detection model is evaluated based on the mean average precision.

[0281] In one possible embodiment, the device is further used for:

[0282] Based on the overlap between the predicted damaged frame and the corresponding real damaged frame on the original QR code image of the verification sample set, the average recognition accuracy of the trained CenterNet object detection model for all types of damaged QR codes is obtained;

[0283] According to the recognition average precision, the mean average precision of the trained CenterNet target detection model is obtained.

[0284] In one possible embodiment, the device is further used for:

[0285] Based on the degree of damage rate impact, the distribution of damage areas, damage colors, and damage recognition algorithms, the damage rate thresholds for QR codes with different damage colors, QR code damage rate thresholds for different damage areas, and the damage rate threshold for AGV positioning QR codes are obtained.

[0286] According to the damage rate thresholds of the QR codes of different damaged colors, the damage rate thresholds of the QR codes of different damaged areas, the damage rate thresholds of the AGV positioning QR code and the damage rate of the target damaged QR code, an early warning and repair of the target damaged QR code are performed.

[0287] In one possible embodiment, the device is further used for:

[0288] Construct a dataset containing different original QR code images, different damage rates, different damage areas, and different damage colors;

[0289] Create a detector; the detector is used to identify the data set to record the identification result of each sample in the data set;

[0290] According to the recognition result of each sample, the average damage rate of different damage colors and different damage areas is obtained respectively, so as to obtain the damage rate threshold of the QR code with different damage colors and the damage rate threshold of the QR code with different damage areas;

[0291] According to the recognition result of each sample, the recognition rate of the damaged image of each damage rate is obtained;

[0292] The AGV positioning QR code damage rate threshold is obtained based on the relationship curve between the recognition rate of the damage image of each damage rate and the damage rate of the corresponding damage image.

[0293] In summary, this application first obtains a sample data set, inputs the sample data set into the CenterNet target detection model to be trained, and trains the CenterNet target detection model; then obtains the target damaged QR code to be identified, and identifies the target damaged QR code through the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code. The above scheme uses the trained CenterNet target detection model to identify the target damaged QR code, reducing the amount of calculation and the complexity of the calculation. During the training phase, the model parameters are set, and the CenterNet target detection model is optimized through the activation function; after the training is completed, the performance of the trained CenterNet target detection model is evaluated through the verification sample set, thereby improving the accuracy and efficiency of the recognition results of the CenterNet target detection model.

[0294] In addition, this application sets a reasonable QR code breakage rate threshold to ensure that when the breakage rate is lower than the QR code breakage rate threshold, the QR code can be successfully identified, and early warning and repair can be performed; thereby improving the robustness and reliability of QR code recognition.

[0295] Please refer to Figure 14, which is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application. The computer device includes a memory and a processor. The memory is used to store a computer program. When the computer program is executed by the processor, the above-mentioned method for identifying a damaged QR code is implemented.

[0296] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0297] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various functional applications and data processing of the processor, thereby implementing the methods in the above-mentioned method embodiments.

[0298] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0299] In an exemplary embodiment, a computer-readable storage medium is further provided, storing at least one computer program, which is loaded and executed by a processor to implement all or part of the steps of the above method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, or an optical data storage device.

[0300] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0301] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for identifying a damaged two-dimensional code, characterized in that: The method comprises: Acquire a sample data set; the sample data set includes a two-dimensional code image sample and a two-dimensional code label sample; Inputting the sample data set into a CenterNet target detection model to be trained, and training the CenterNet target detection model; The target damaged QR code to be identified is obtained, and the target damaged QR code is identified by using the trained CenterNet target detection model to obtain the damage rate of the target damaged QR code.

2. The method according to claim 1, characterized in that: The two-dimensional code image samples include original two-dimensional code images and damaged two-dimensional code images after masking operations; the damaged two-dimensional code images include damaged samples with different contamination colors, different damaged areas and different damage rates; The two-dimensional code label sample includes at least one of the file name of the two-dimensional code, the file address of the two-dimensional code, the type and position of the defacement pattern contained in the two-dimensional code, and the image size of the two-dimensional code.

3. The method according to claim 2, characterized in that After obtaining the sample data set, the method further includes: Dividing the sample data set into a training sample set, a validation sample set and a test sample set according to a specified ratio; According to the contaminated color of the damaged two-dimensional code image, the training sample set is classified into multiple categories, and the type information is recorded.

4. The method according to claim 3, characterized in that: Inputting the sample data set into a CenterNet target detection model to be trained, and training the CenterNet target detection model, comprises: Obtaining training parameters and specification data of the CenterNet target detection model to be trained; the training parameters and specification data include at least one of a backbone feature extraction network, pre-training weights, training picture size, freezing phase parameters, thawing phase parameters, model confidence, maximum and minimum values ​​of model learning rate, optimizer parameters, and evaluation parameters; The CenterNet target detection model is trained according to the training sample set, the training parameters and the specification data.

5. The method according to claim 4, characterized in that After the training phase of the CenterNet object detection model, the method further includes: An activation function of the CenterNet target detection model is obtained, and the CenterNet target detection model in the training phase is optimized according to the activation function.

6. The method according to claim 5, characterized in that After training the CenterNet target detection model, the method further includes: According to the verification sample set, the performance of the trained CenterNet target detection model is evaluated; According to the test sample set, the performance of the trained CenterNet target detection model is tested.

7. The method according to claim 6, characterized in that The performance evaluation of the trained CenterNet target detection model is performed according to the verification sample set, including: Performing target prediction on the verification sample set by using the trained CenterNet target detection model to obtain a contamination pattern corresponding to the verification sample set; According to the contamination pattern, obtaining a contamination prediction box on the original two-dimensional code image of the verification sample set; Obtaining the mean average precision of the trained CenterNet object detection model according to the contaminated prediction frame and the corresponding contaminated real frame on the original two-dimensional code image of the verification sample set; The performance of the trained CenterNet target detection model is evaluated according to the mean average precision.

8. The method according to claim 7, characterized in that The step of obtaining the mean average precision of the trained CenterNet target detection model according to the contaminated prediction frame and the contaminated real frame corresponding to the original two-dimensional code image of the verification sample set includes: According to the overlap between the damage prediction frame and the corresponding damage real frame on the original two-dimensional code image of the verification sample set, the average recognition accuracy of the trained CenterNet object detection model for all types of damaged two-dimensional codes is obtained; According to the recognition average precision, the mean average precision of the trained CenterNet target detection model is obtained.

9. The method according to any one of claims 3 to 8, characterized in that: After obtaining the damage rate of the target damaged two-dimensional code, the method further includes: According to the influence degree of the damage rate, the distribution of the damage area, the damage color and the damage recognition algorithm, the damage rate thresholds of the QR codes with different damage colors, the damage rate thresholds of the QR codes with different damage areas and the damage rate thresholds of the AGV positioning QR codes are obtained; According to the damage rate thresholds of the two-dimensional codes of different damaged colors, the damage rate thresholds of the two-dimensional codes of different damaged areas, the damage rate thresholds of the AGV positioning two-dimensional code and the damage rate of the target damaged two-dimensional code, the target damaged two-dimensional code is warned and repaired.

10. The method according to claim 9, characterized in that According to the degree of influence of the damage rate, the distribution of the damage area, the damage color and the damage recognition algorithm, the damage rate thresholds of the two-dimensional codes with different damage colors, the damage rate thresholds of the two-dimensional codes with different damage areas and the damage rate threshold of the two-dimensional code for AGV positioning are obtained, including: Construct a set of data sets containing different original QR code images, different damage rates, different damage areas, and different damage colors; Creating a detector; the detector is used to identify the data set to record the identification result of each sample of the data set; According to the recognition result of each sample, the average values ​​of the damage rates of different damaged colors and different damaged areas are respectively obtained to obtain the damage rate thresholds of the two-dimensional codes with different damaged colors and the damage rate thresholds of the two-dimensional codes with different damaged areas; According to the recognition result of each sample, obtaining the recognition rate of the damaged image of each damage rate; The damage rate threshold of the AGV positioning two-dimensional code is obtained according to the relationship curve between the recognition rate of the damaged image of each damage rate and the damage rate of the corresponding damaged image.

Images