An anti-fouling QR code detection and positioning method

Abstract

The present invention belongs to the technical field of image processing, and particularly relates to a method for detecting and positioning a stain-resistant two-dimensional code. Based on the mainstream object detection neural network model, it is improved according to the requirements of deployment on an embedded platform to obtain a two-dimensional code neural network model; and in the two-dimensional code neural network model, classification criteria are set according to its stain condition and angle information, and the classification channels of the detection head output are changed according to the set criteria to make it consistent with the set classification criteria; in the inference learning of the model, based on the preset classification criteria, the corresponding calculation module is selected according to the classification result obtained in step 4.1 to calculate the two-dimensional code angle information; then the two-dimensional code angle information is verified to obtain the final two-dimensional code position and angle information output of the image to be detected. Compared with the prior art, the present invention improves the detection and positioning accuracy of the two-dimensional code in a changing scenario, has a small amount of calculation, and supports implementation and operation on an embedded system.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method for detecting and positioning a stain-resistant two-dimensional code. Background Art

[0002] The detection and positioning of two-dimensional codes refer to identifying and positioning the position and orientation of two-dimensional codes in an image, and this technology is commonly used in environmental positioning of unmanned systems. According to the position and orientation information of the two-dimensional code in the image, as well as the environmental position information contained in the two-dimensional code, the positioning information of the unmanned system relative to the environment can be obtained, thereby enabling the unmanned system to move autonomously in the environment. Currently, the two-dimensional code detection and positioning technology is usually implemented based on the object detection method of neural networks or image segmentation technology. Among them, the object detection method based on neural networks can achieve high-precision detection of two-dimensional codes and is robust in complex scenarios. However, the method based on neural networks often comes with a heavy computational burden, and lightweight neural networks are difficult to fit the high-precision information of multiple labels (such as position and orientation), which makes this method difficult to be applied to the edge platform of unmanned systems. Compared with the object detection method based on neural networks, the method based on image segmentation technology is computationally simpler and can achieve relatively high precision for typical scenarios. However, in changing scenarios, such as the inevitable material dirt, wear, and tear in industrial scenarios, the method based on image segmentation often fails in this case, resulting in the failure of two-dimensional code detection. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for detecting and positioning a stain-resistant two-dimensional code, which can improve the detection and positioning accuracy of two-dimensional codes in changing scenarios while supporting implementation and operation on an embedded system.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions;

[0005] A method for detecting and positioning a stain-resistant two-dimensional code includes the following steps:

[0006] Step 1, construct a training data set for two-dimensional code images; the construction of the training data set includes collecting two-dimensional code image data and preprocessing the collected two-dimensional code image data. The collected two-dimensional code image data is two-dimensional code image data in an actual scenario, including two-dimensional codes with different codewords, different positions, different angles, different illuminations, and different environments of the two-dimensional code in the image;

[0007] The preprocessing includes two parts: data annotation and data cleaning; data annotation refers to the bounding of the position of the two-dimensional code in each image, the angle annotation of the two-dimensional code on the plane, and the classification category annotation of the two-dimensional code; data cleaning refers to converting the annotation file into a unified format for subsequent input training;

[0008] Step 2: Build a QR code detection neural network model. Based on the object detection neural network model, it is improved according to the requirements of embedded platform deployment to obtain the QR code detection neural network model. The QR code detection neural network model includes a feature extraction module, a detection head, a calculation unit, and a trainer;

[0009] Step 3: Based on the training dataset constructed in Step 1, train the QR code detection neural network model;

[0010] Step 4: Use the trained QR code detection neural network model to perform inference on the image to be detected:

[0011] 4.1 Input the image to be detected into the trained QR code detection neural network model. Use the feature extraction module to extract features and output them to the detection head for detection to obtain the QR code detection result. The QR code detection result includes the QR code positioning frame and the QR code classification result;

[0012] 4.2 According to the QR code positioning frame obtained in Step 4.1, crop out the area containing the QR code and having redundant space from the original image. This area is called the cropped area;

[0013] Step 5: Based on the area obtained in Step 4, use the QR code position information and the QR code classification result obtained in Step 4. Use the angle regression module and the angle calculation module to calculate the angle information of the QR code in the image to be detected, so as to obtain the final QR code position output of the image to be detected.

[0014] Further, the QR code classification category labeling reference rules in Step 1 are as follows:

[0015] 0 indicates that the QR code is intact and undamaged; 1 indicates that the QR code has internal damage and clear boundaries; 2-5 indicate that the QR code has damage and a relatively large degree of damage, with damaged boundaries, and it is difficult to determine whether it can be decoded. Among them, 2 indicates that the QR code angle range is 0-90°, 3 indicates that the QR code angle range is 90-180°, 4 indicates that the QR code angle range is 180-270°, 5 indicates that the QR code angle range is 270-360°, and 6 indicates that the QR code is severely damaged, most of the codewords are invisible, and it cannot be decoded or the angle cannot be calculated.

[0016] Further, the improvement in Step 2 according to the requirements of embedded platform deployment to obtain the QR code detection neural network model includes the following steps:

[0017] 2.1 Reduce the number of network channels and depth, and reduce the network complexity and calculation amount on the basis of maintaining the inference effect and accuracy to meet the operation requirements of the embedded platform;

[0018] 2.2 Modify the input classification channels of the detection head so that the number of channels is consistent with the total number of set classification categories.

[0019] Further, before using the trained QR code detection neural network model for inference in step 4, the trained QR code detection neural network model is cropped and fine-tuned to reduce the number of parameters while maintaining the detection effect.

[0020] Further, the step of calculating the angle information of the QR code in the to-be-detected image using the calculation module based on the cropped area obtained in step 4.2 in step 5 includes:

[0021] The calculation module includes an angle regression module and an angle calculation module. The angle regression module includes a first angle regression module and a second angle regression module; based on a preset classification standard, the corresponding angle regression module and angle calculation module are sequentially selected for calculation according to the classification result obtained in step 4.1;

[0022] When the classification result obtained in step 4.1 is 0 or 1, the cropped area obtained in step 4.2 is used as the input image and input into the first angle regression module and the first angle calculation module for calculation;

[0023] When the classification result obtained in step 4.1 is 2, 3, 4, or 5, the cropped area obtained in step 4.2 is used as the input image and input into the second angle regression module and the first angle calculation module for calculation;

[0024] When the classification result obtained in step 4.1 is 6, the QR code coordinate information and the angle None value are directly output according to the QR code positioning frame obtained in step 4.1, and the output QR code coordinate information is used as the final QR code position of the to-be-detected image.

[0025] Furthermore, the first angle regression module is also called a four-point angle regression module, and its calculation method is as follows:

[0026] (1) Unify the input image into a grayscale image; after performing Gaussian denoising on the grayscale image, perform binaryzation processing using an adaptive Gaussian threshold; at the same time, estimate the maximum and minimum values of the QR code size according to the size of the grayscale image;

[0027] (2) Use morphological operations to perform erosion and dilation processing on the binaryzation-processed image to eliminate noise and enhance the QR code features;

[0028] (3) Perform contour regression on the image obtained in (2);

[0029] (4) Perform morphological convex hull detection on the contour regressed in (3) and exclude contours that do not meet the conditions;

[0030] (5) Traverse the boundary contours obtained in (4), calculate the maximum inscribed rectangle for each of them, calculate the area of the rectangle, and determine the rectangular boundary that meets the QR code size range obtained in (1) as the outer boundary of the QR code, thereby obtaining the side lengths and four corner points of the outer boundary of the QR code;

[0031] (6) Calculate the minimum grid resolution of the QR code based on the side length of the outer boundary of the QR code obtained in (5);

[0032] (7) Calculate the proximity search range of the QR code corner points based on the minimum grid resolution obtained in (6);

[0033] (8) Traverse the 4 corner points of the outer boundary of the QR code obtained in (7), and based on the proximity search range of the QR code corner points, judge the brightness values near the corner points; then, based on the brightness values, determine 3 corner points among the 4 corner points of the outer boundary of the QR code that belong to the "L" - shaped vertex of the QR code recognition mark (finder pattern);

[0034] (9) Obtain the coordinate information for QR code positioning. Based on the coordinates of the 4 corner points obtained in (8), perform an average process to obtain a QR code center coordinate. When the difference between its value and the neural network regression coordinate is greater than 5 pixels, update the QR code center coordinate with the four - point positioning coordinates. When the difference is less than or equal to 5 pixels, no processing is performed;

[0035] (10) Based on the 3 corner points that belong to the "L" - shaped vertex of the QR code recognition mark determined in (8), calculate the vectors wb1 and wb2 of the two sides of the "L" - shaped QR code recognition mark; define the vectors nv1 and nv2 of the two sides when the QR code is at 0°, and use the first - angle calculation module to calculate the deflection angle of the QR code.

[0036] Furthermore, the steps for the first - angle calculation module to calculate the deflection angle of the QR code include:

[0037] Define a very small quantity eps, whose value is 0.02;

[0038] Perform a regularization operation on the four vectors obtained in step (8) by the first - angle regression module, that is, divide them by their 2 - norms respectively to obtain nv1_norm, nv2_norm, wb1_norm, and wb2_norm;

[0039] Calculate the cosine similarity between the two pairs of vectors nv1_norm and wb1_norm, and the cosine similarity between nv2_norm and wb2_norm respectively. The two cosine similarities are cos1 and cos2;

[0040] Calculate the cross products between two pairs of vectors respectively, that is, calculate the cross product between nv1_norm and wb1_norm, and the cross product between nv2_norm and wb2_norm respectively. The two groups of vectors are cross1 and cross2 respectively;

[0041] Use conditional judgment. If the conditions (abs(cos1 - cos2) < eps) & (cross1 * cross2 >= 0) are satisfied, then calculate the angle angle between each pair of vectors in the two pairs of vectors as the arccosine value of the average of the two cosine similarities; if the conditions (abs(cos1 - cos2) < eps) & (cross1 * cross2 >= 0) are not satisfied, then output a warning message and calculate the angle angle as the arccosine value of the first cosine similarity;

[0042] According to the formula angle = angle · (cross1 / |cross1|) if cross1 ≠ 0 else angle, adjust the direction of the angle angle. The finally obtained is the QR code angle.

[0043] Furthermore, the second angle regression module is also called the hough angle regression module, and its calculation method is as follows:

[0044] (1) Uniformly input the image as a grayscale image and perform binaryzation processing after alignment;

[0045] (2) Calculate the adaptive hough threshold according to the input image size;

[0046] (3) Use the adaptive hough threshold transformation calculated in (2) to detect the straight lines in the binaryzation processed image;

[0047] (4) Sort all the straight lines detected in (3): Each straight line corresponds to an angle of it in the image, and output the two angle values that appear the most times;

[0048] (5) Input the two obtained angle values into the second angle calculation module to calculate the QR code angle;

[0049] (6) Generate a QR code recognition template according to the QR code positioning frame coordinates obtained in step 4.1 of the second dimension and the QR code angle obtained by the second angle calculation module, and match and verify it with the original image. When the coincidence degree is lower than the threshold, output an alarm message. When the coincidence degree is higher than the threshold, the verification passes, and output the final QR code position and angle; The QR code recognition template is an "L" - shaped recognition mark of the recognition mark at a specified position and angle.

[0050] Furthermore, the method for the second angle calculation module to calculate the QR code angle includes:

[0051] Determine the difference between two angles. If the difference is less than 3°, then weighted average the two angle values according to the number of occurrences of the straight line at that angle to obtain an angle of a straight line cluster in the range of 0-90°; if the difference is greater than or equal to 3°, then select the angle with a higher number of occurrences as the angle of the straight line cluster.

[0052] Based on the obtained angle of the straight line cluster, and in combination with the preset classification criteria and the two-dimensional code classification results, convert the angle of the straight line cluster into a two-dimensional code angle on the plane of 0-360°.

[0053] Furthermore, the target detection neural network model is a YOLO target detection network, an SSD target detection network, or a target detection network based on Transformer, etc.

[0054] A method for detecting and positioning a stain-resistant two-dimensional code provided by the present invention is based on a target detection neural network model, improved according to the requirements of deployment on an embedded platform to obtain a two-dimensional code detection neural network model; and in the two-dimensional code detection neural network model, set classification criteria according to its stain condition and angle information, and change the classification channels output by the detection head according to the set classification criteria to make it consistent with the set classification criteria; in the inference learning of the model, based on the preset classification criteria, select the corresponding angle regression module and angle calculation module according to the classification results obtained in step 4.1, calculate the two-dimensional code angle information; then verify the two-dimensional code angle information to obtain the final output of the two-dimensional code position in the image to be detected.

[0055] Adopting the above technical solutions, the present invention has the following advantages:

[0056] 1. It has high robustness and solves the problem of positioning failure of traditional methods in the case of dirty and worn two-dimensional codes.

[0057] 2. Thanks to the pixel-level image processing method, it maintains a high-precision two-dimensional code detection effect in the case of dirty and worn two-dimensional codes, manifested in extremely low coordinate and angle deviations (<1mm, 1°).

[0058] 3. It is lightweight. While maintaining high precision and high robustness, it consumes less computing resources and supports real-time operation on an embedded system. Description of the Drawings

[0059] Figure 1 is the overall flowchart of a method for detecting and positioning a stain-resistant two-dimensional code provided by the embodiment;

[0060] Figure 2 is the calculation flowchart of the first angle regression module in the embodiment;

[0061] Figure 3 is the calculation flowchart of the first angle calculation module in the embodiment;

[0062] Figure 4 is the calculation flowchart of the second angle regression module in the embodiment;

[0063] Figure 5 is the calculation flowchart of the second angle calculation module in the embodiment;

[0064] Figure 6 is the definition of two pairs of vectors in the first angle regression module in the embodiment. Detailed implementation manners

[0065] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0066] As Figure 1 shown, a method for detecting and positioning an anti-fouling two-dimensional code provided in this embodiment includes the following steps:

[0067] Step 1, construct a two-dimensional code image training data set. In this embodiment, two-dimensional code image data in an actual scene is collected to construct a two-dimensional code image training data set. The method for constructing the training data set includes the following steps:

[0068] 1.1, obtain two-dimensional code image data in an actual scene;

[0069] 1.2, preprocess the two-dimensional code image data obtained in 1.1. The preprocessing includes two parts: data annotation and data cleaning:

[0070] The data annotation includes: selecting the position of the two-dimensional code in each image, annotating the angle of the two-dimensional code in the plane (the standard plane angle range is 0-360°), and annotating the classification category of the two-dimensional code: the two-dimensional codes in the images of this embodiment are divided into seven categories, and the seven categories are respectively represented by 0, 1, 2, 3, 4, 5, 6. The more detailed classification criteria are set as follows:

[0071] 0 means: the two-dimensional code is intact and free of fouling. 1 means: the two-dimensional code has internal fouling and clear boundaries. 2-5 means: the two-dimensional code has fouling and the fouling degree is relatively large, the boundary is damaged, and it is difficult to judge whether it can be decoded; among them, 2: the angle range of the two-dimensional code is 0-90°; 3: the angle range of the two-dimensional code is 90-180°; 4: the angle range of the two-dimensional code is 180-270°; 5: the angle range of the two-dimensional code is 270-360°.

[0072] 6: the two-dimensional code is severely damaged, most of the codewords are invisible, and it is impossible to decode or calculate the angle.

[0073] The process of the data cleaning is to convert the annotation file into a unified format for subsequent input training.

[0074] Step 2: Build a QR code detection neural network model, including a feature extraction network, a detection head, a calculation unit, and a trainer. The QR code detection neural network model is based on an object detection neural network model and is improved according to the requirements of embedded platform deployment. In this embodiment, the object detection neural network model used is one of the object detection networks such as the YOLO object detection network, the SSD object detection network, and the object detection network based on Transformer. The improvements made to the object detection neural network model according to the requirements of embedded platform deployment include:

[0075] According to the requirements of embedded platform deployment, reduce the number of channels and the depth of the network;

[0076] According to the classification annotation set in Step 1, modify the input classification channels of the detection head so that the number of channels is consistent with the total number of set classification categories, so as to be applicable to this application.

[0077] Step 3: Train the QR code detection neural network model based on the training dataset constructed in Step 1.

[0078] In this embodiment, the training dataset constructed in Step 1 is divided into a training set and a validation set, and the batch size, epoch, anchor box, and hyperparameters are set. The QR code detection neural network model is trained based on the training set, and the above training parameters are adjusted according to the actual situation during training, so as to obtain a trained QR code detection neural network.

[0079] Step 4: Use the trained QR code detection neural network model to perform inference on the image to be detected to obtain the angle information of the QR code in the image to be detected. Before inference, in this embodiment, the trained QR code detection neural network is first cropped and fine-tuned to reduce the number of parameters while maintaining the detection effect. The detailed inference process is as follows:

[0080] 4.1: Input the image to be detected into the trained QR code detection neural network model, use the feature extraction module to extract features and output them to the detection head for detection, and obtain the QR code detection result. The QR code detection result includes the QR code positioning frame and the QR code classification result;

[0081] 4.2: According to the QR code positioning frame obtained in Step 4.1, crop out the area that contains the QR code and has redundant space from the original image, and this area is called the cropped area.

[0082] Step 5: Based on the region obtained in Step 4, using the QR code position information and QR code classification result obtained in Step 4, the calculation module calculates the angle information of the QR code in the image to be detected, so as to obtain the final QR code position output of the image to be detected. The calculation module used in this embodiment includes an angle regression module and an angle calculation module. The angle regression module includes a first angle regression module and a second angle regression module. When calculating, based on the preset classification standard, the corresponding angle regression module and angle calculation module are sequentially selected for calculation according to the classification result obtained in Step 4.1:

[0083] When the classification result obtained in Step 4.1 is 0 or 1, the cropped region obtained in Step 4.2 is used as the input image and input into the first angle regression module and the first angle calculation module for calculation;

[0084] When the classification result obtained in Step 4.1 is 2, 3, 4, or 5, the cropped region obtained in Step 4.2 is used as the input image and input into the second angle regression module and the first angle calculation module for calculation;

[0085] When the classification result obtained in Step 4.1 is 6, the QR code coordinate information and the angle None value are directly output according to the QR code positioning frame obtained in Step 4.1, and the output QR code coordinate information is used as the final QR code position of the image to be detected.

[0086] The first angle regression module is also called a four-point angle regression module, and its calculation method is as Figure 2 shown, including the following steps:

[0087] (1) Unify the input image into a grayscale image; after performing Gaussian denoising on the grayscale image, perform binaryzation processing using an adaptive Gaussian threshold; at the same time, estimate the maximum and minimum values of the QR code size according to the size of the grayscale image;

[0088] (2) Use morphological operations to perform erosion and dilation processing on the binaryzation-processed image to eliminate noise and enhance the QR code features;

[0089] (3) Perform contour regression on the image obtained in (2);

[0090] (4) Perform morphological convex hull detection on the contours regressed in (3), and exclude the contours that do not meet the conditions;

[0091] (5) Traverse the boundary contours obtained in (4), perform the calculation of the maximum inscribed rectangle on them, and calculate the rectangle area. The rectangle boundary that meets the QR code size range obtained in (1) is determined as the QR code outer boundary, so as to obtain the side length and four corner points of the QR code outer boundary;

[0092] (6) Calculate the minimum grid resolution of the QR code according to the side length of the QR code outer boundary obtained in (5);

[0093] (7) Calculate the adjacent search range of the QR code corner points based on the minimum grid resolution obtained in (6).

[0094] (8) Traverse the 4 corner points of the outer boundary of the QR code obtained in (7). Based on the adjacent search range obtained in (7), judge the brightness values near the corner points. Then, according to the brightness values, determine the 3 corner points of the "L" shape that belong to the QR code identification mark (finder pattern) among the 4 corner points of the outer boundary of the QR code.

[0095] (9) Based on the 4 corner point coordinates obtained in (8), perform an averaging process to obtain a QR code center coordinate. When the difference between its value and the neural network regression coordinate is greater than 5 pixels, update the QR code center coordinate with the four-point positioning coordinate. When the difference is less than or equal to 5 pixels, no processing is performed.

[0096] (10) Calculate the vectors wb1 and wb2 of the two sides of the "L" shape based on the 3 corner points of the "L" shape vertices that belong to the QR code identification mark determined in (8). Define the vectors of the two sides when the QR code is at 0° as nv1 and nv2, and use the first angle calculation module to calculate the deflection angle of the QR code. Calculate the vectors wb1 and wb2 of the two sides of the "L" shape. Define the vectors of the two sides when the QR code is at 0° as nv1 and nv2. See Figure 6 。

[0097] The calculation steps of the first angle calculation module are as follows Figure 3 shown, including the following steps:

[0098] Define a very small quantity (epsilon, abbreviated as eps), whose value is 0.02;

[0099] Perform a regularization operation on the four vectors obtained in step (8) of the first angle regression module, that is, divide them by their 2-norms respectively to obtain nv1_norm, nv2_norm, wb1_norm, and wb2_norm;

[0100] Calculate the cosine similarity between the two pairs of vectors nv1_norm and wb1_norm, and the cosine similarity between nv2_norm and wb2_norm respectively. The two cosine similarities are cos1 and cos2 respectively;

[0101] Calculate the cross product between the two pairs of vectors respectively, that is, calculate the cross product between nv1_norm and wb1_norm, and the cross product between nv2_norm and wb2_norm respectively. The cross products between the two groups of vectors are cross1 and cross2 respectively;

[0102] Use conditional judgment. If the conditions (abs(cos1 - cos2) < eps) & (cross1 * cross2 >= 0) are met, then calculate the angle between each pair of vectors in the two pairs of vectors, where angle is the arccosine of the average of the two cosine similarities; if the conditions (abs(cos1 - cos2) < eps) & (cross1 * cross2 >= 0) are not met, then output a warning message and calculate the angle as the arccosine of the first cosine similarity.

[0103] According to the formula angle = angle · (cross1 / |cross1|) if cross1 ≠ 0 else angle, adjust the direction of the angle, and the finally obtained is the QR code angle.

[0104] The second angle regression module is also called the hough angle regression module, and its calculation method is as Figure 4 shown, including the following steps:

[0105] (1) Uniformly input the image as a grayscale image and perform alignment and binarization processing.

[0106] (2) Calculate the adaptive hough threshold according to the input image size.

[0107] (3) Use the adaptive hough threshold transformation calculated in (2) to detect the lines in the binarized image.

[0108] (4) Organize all the lines detected in (3): Each line corresponds to an angle of it in the image, and output the two angle values that appear the most times.

[0109] (5) Input the two obtained angle values into the second angle calculation module to calculate the QR code angle.

[0110] (6) Generate a QR code recognition template based on the QR code positioning frame coordinates obtained in step 4.1 and the QR code angle obtained by the second angle calculation module, and match and verify it with the original image. When the coincidence degree is lower than the threshold, output an alarm message. When the coincidence degree is higher than the threshold, the verification passes, and output the final QR code position and angle; the QR code recognition template refers to the "L"-shaped recognition mark (finder pattern) that determines the position and angle.

[0111] The method for the second angle calculation module to calculate the QR code angle is as Figure 5 shown, including the following steps:

[0112] Determine the difference between two angles. If the difference is less than 3°, then weighted average the two angle values according to the number of occurrences of the angle corresponding to the line to obtain a line cluster angle within 0 - 90°; if the difference is greater than or equal to 3°, then select the angle with a higher number of occurrences as the line cluster angle.

[0113] Based on the obtained line cluster angle, and in combination with the preset classification criteria and the two-dimensional code classification result, convert the angle of the line cluster into a two-dimensional code plane angle within 0 - 360°.

[0114] It can be understood that the present invention is described through some embodiments. Those skilled in the art know that without departing from the spirit and scope of the present invention, various changes or equivalent replacements can be made to these features and embodiments. Additionally, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A method for detecting and locating a two-dimensional code that is resistant to fouling, characterized in that: The following steps are involved: Step 1: Construct a QR code image training dataset; The construction of the training data set includes collecting two-dimensional QR code image data and preprocessing the collected QR code image data. The collected QR code image data is QR code image data in actual scenes, including QR codes with different code words, QR codes at different positions, different angles, different lighting and different environments in the image; Preprocessing includes data labeling and data cleaning. Data labeling refers to the selection of the QR code position in each image, the angle labeling of the QR code on the plane, and the classification of the QR code. Data cleaning refers to converting the labeling file into a unified format for subsequent input training. Step 2: Build a two-dimensional code detection neural network model, based on the target detection neural network model, improve it according to the requirements of embedded platform deployment, and obtain a two-dimensional code detection neural network model; the two-dimensional code detection neural network model includes a feature extraction module, a detection head, a computing unit and a trainer; Step 3: Based on the training data set constructed in step 1, train a neural network model for QR code detection; Step 4: Use the trained QR code detection neural network model to infer the image to be detected: 4.

1. Input the image to be detected into the trained QR code detection neural network model, use the feature extraction module to extract features and output them to the detection head for detection, and obtain the QR code detection result, which includes the QR code positioning frame and the QR code classification result; 4.

2. According to the QR code positioning frame obtained in step 4.1, a region containing the QR code and having redundant space is cropped from the original image, and this region is called the cropping region; Step 5: Based on the area obtained in step 4, the angle information of the QR code of the image to be detected is calculated using the angle regression module and the angle calculation module using the QR code position information and the QR code classification result obtained in step 4, thereby obtaining the final QR code position output of the image to be detected; Based on the cropped area obtained in step 4.2, the step of using the calculation module to calculate the angle information of the two-dimensional code of the image to be detected includes: The calculation module includes an angle regression module and an angle calculation module, and the angle regression module includes a first angle regression module and a second angle regression module; wherein the first angle regression module is a four-point angle regression module, and the second angle module is a hough angle regression module; based on the preset classification standard, the corresponding angle regression module and angle calculation module are selected in turn according to the classification result obtained in step 4.1 for calculation; When the classification result obtained in step 4.1 is 0 or 1, the cropped area obtained in step 4.2 is used as the input image and input into the first angle regression module and the first angle calculation module for calculation; When the classification result obtained in step 4.1 is 2, 3, 4 or 5, the cropped area obtained in step 4.2 is used as the input image and input into the second angle regression module and the first angle calculation module for calculation; When the classification result obtained in step 4.1 is 6, the two-dimensional code coordinate information and the angle None value are directly output according to the two-dimensional positioning frame obtained in step 4.1, and the output two-dimensional code coordinate information is used as the final two-dimensional code position of the image to be detected.

2. The anti-fouling two-dimensional code detection and positioning method according to claim 1, characterized in that: The reference rules for the QR code classification category marking in step 1 are as follows: 0 means the QR code is intact; 1 means the QR code has internal damage but clear boundaries; 2-5 means the QR code is severely damaged, with damaged boundaries, and it is difficult to determine whether it can be decoded. 2 means the QR code angle range is 0-90°, 3 means the QR code angle range is 90-180°, 4 means the QR code angle range is 180-270°, 5 means the QR code angle range is 270-360°, and 6 means the QR code is severely damaged, most of the code words are invisible, and it is impossible to decode or calculate the angle.

3. The anti-fouling two-dimensional code detection and positioning method according to claim 2, characterized in that: The step 2 is improved according to the requirements of embedded platform deployment to obtain a two-dimensional code detection neural network model, including the steps of: 2.

1. Reduce the number of network channels and depth, reduce network complexity and computational complexity while maintaining inference effect and accuracy, so as to meet the computing requirements of embedded platforms; 2.

2. Modify the input classification channels of the detection head so that the number of channels is consistent with the total number of set classification categories.

4. The anti-fouling two-dimensional code detection and positioning method according to claim 1, characterized in that: The step 4 also includes cutting and fine-tuning the trained two-dimensional code detection neural network model before using the trained two-dimensional code detection neural network model for reasoning.

5. The anti-fouling two-dimensional code detection and positioning method according to claim 1, characterized in that: The first angle regression module is also called a four-point angle regression module, and its calculation method is as follows: (1) The input image is unified into a grayscale image; after Gaussian denoising, the grayscale image is binarized using an adaptive Gaussian threshold; and the maximum and minimum values ​​of the QR code size are estimated based on the size of the grayscale image; (2) Use morphological operations to perform corrosion and expansion processing on the binary image to eliminate noise and enhance the characteristics of the QR code; (3) Perform contour regression on the image obtained in (2); (4) Perform morphological convex hull detection on the contours regressed in (3) and exclude contours that do not meet the conditions; (5) Traverse the boundary contour obtained in (4), calculate the maximum inscribed rectangle, and calculate the area of ​​the rectangle. Determine the rectangular boundary that meets the size range of the two-dimensional code obtained in (1) as the outer boundary of the two-dimensional code, thereby obtaining the side length and four corner points of the outer boundary of the two-dimensional code; (6) Calculate the minimum grid resolution of the QR code based on the length of the outer boundary of the QR code obtained in (5); (7) According to the minimum grid resolution obtained in (6), the neighboring search range of the QR code corner point is calculated; (8) Traverse the four corner points of the outer boundary of the two-dimensional code obtained by (7), and determine the brightness value near the corner point based on the adjacent search range of the two-dimensional code corner point; then, according to the brightness value, determine the three corner points of the "L"-shaped mark vertices of the identification mark of the two-dimensional code among the four corner points of the outer boundary of the two-dimensional code; (9) Based on the coordinates of the four corner points obtained in (8), average processing is performed to obtain a QR code center coordinate. When the difference between its value and the neural network regression coordinate is greater than 5 pixels, the four-point positioning coordinate is used to update the QR code center coordinate. When the difference is less than or equal to 5 pixels, no processing is performed; (10) According to the three corner points of the "L"-shaped vertices of the two-dimensional code recognition mark determined by (8), calculate the vectors wb1 and wb2 on both sides of the "L" shape; define the vectors nv1 and nv2 on both sides of the "L" shape when the two-dimensional code is 0°, and use the first angle calculation module to calculate the deflection angle of the two-dimensional code.

6. The anti-fouling two-dimensional code detection and positioning method according to claim 5, characterized in that: The step of calculating the deflection angle of the two-dimensional code by the first angle calculation module comprises: Define a very small amount eps, whose value is 0.02; Regularize the four vectors obtained in the calculation step (8) of the first angle regression module, that is, divide them by their 2 norms to obtain nv1_norm, nv2_norm, wb1_norm, and wb2_norm; Calculate the cosine similarity between two pairs of vectors nv1_norm and wb1_norm, and the cosine similarity between nv2_norm and wb2_norm, the two cosine similarities are cos1 and cos2 respectively; Calculate the cross product between two pairs of vectors, that is, the cross product between nv1_norm and wb1_norm, and the cross product between nv2_norm and wb2_norm, respectively. The two sets of vectors are cross1 and cross2 respectively. Use conditional judgment, if the condition (abs(cos1-cos2) is met<eps)&(cross1*cross2> =0), the angle between each pair of vectors is calculated as the arccosine value of the average value of the two cosine similarities; if the condition (abs(cos1-cos2)) is not met,<eps)&(cross1*cross2> =0), a warning message is output and the angle angle is calculated as the arccosine value of the first cosine similarity; According to the formula angle = angle·(cross1 / |cross1|)if cross1≠0else angle, adjust the direction of the angle, and the final result is the QR code angle.

7. The anti-fouling two-dimensional code detection and positioning method according to claim 1, characterized in that: The second angle regression module is also called the hough angle regression module, and its calculation method is as follows: (1) Unify the input images into grayscale images and align them for binary processing; (2) Calculate the adaptive hough threshold according to the input image size; (3) Using the adaptive Hough threshold transform calculated in (2) to detect straight lines in the binarized image; (4) Arrange all the straight lines detected in (3): each straight line corresponds to an angle in the image, and output the two angle values ​​that appear the most times; (5) Input the two obtained angle values ​​into the second angle calculation module to calculate the angle of the QR code; (6) Generate a QR code recognition template based on the QR code positioning frame coordinates obtained in step 4.1 and the QR code angle obtained by the second angle calculation module, and match and verify it with the original image. When the overlap is lower than the threshold, an alarm message is output. When the overlap is higher than the threshold, the verification is passed and the final QR code position and angle are output. The QR code recognition template is an "L"-shaped recognition mark of the specified position angle.

8. The anti-fouling two-dimensional code detection and positioning method according to claim 7, characterized in that: The method for calculating the angle of the two-dimensional code by the second angle calculation module includes: Determine the difference between the two angles. If the difference is less than 3°, then weight the two angle values ​​according to the number of times the angle line appears to obtain a 0-90° line cluster angle; if the difference is greater than or equal to 3°, then select the angle with a higher number of occurrences as the line cluster angle; According to the obtained straight line cluster angle, based on the preset classification standard and combined with the QR code classification result, the straight line cluster angle is converted into a plane QR code angle of 0-360°.

9. The anti-fouling two-dimensional code detection and positioning method according to claim 1, characterized in that: The target detection neural network model is a YOLO target detection network, an SSD target detection network or a transformer-based target detection network.

Images