DM code positioning and recognition method based on combination of deep learning and pixel-level detection

Abstract

The present invention relates to the technical field of DM code recognition, and provides a DM code positioning and recognition method based on a combination of deep learning and pixel-level detection. The method comprises: firstly, using a coarse positioning model to extract a roughly positioned DM code image from an original image to be detected; then performing extraction on the roughly positioned DM code image respectively by means of a pixel-wise traversal method and a semantic segmentation method to obtain a plurality of candidate corner point combinations; and finally sequentially traversing all the obtained candidate corner point combinations, and performing pixel-wise parsing on the roughly positioned DM code image, so as to complete positioning and recognition. According to the method, by means of a combination of a pixel-wise traversal method and a deep learning method, candidate corner point combinations are respectively extracted, and the two methods are complementary, thereby ensuring the recognition speed while improving the accuracy and success rate of DM code positioning and recognition, and achieving a good recognition effect.

Description

A DM code positioning and recognition method based on deep learning combined with pixel-level detection Technical Field

[0001] The present application relates to the technical field of DM code recognition, and in particular to a DM code positioning and recognition method that combines deep learning with pixel-level detection. Background Art

[0002] Due to their high information density and confidentiality, QR codes hold an irreplaceable position in product traceability and identity verification, making them widely used in daily life and industrial production. DataMatrix codes, also known as DM codes, are a subset of QR codes. DM codes feature a small footprint, high information storage density, large encoding capacity, and strong error correction capabilities, making them particularly useful in industrial production and logistics.

[0003] However, the background in industrial production and logistics environments is usually complex, which places higher requirements on the positioning and recognition of DM codes. Traditional two-dimensional code recognition methods such as morphological methods and Hough line detection have poor results in positioning and detecting DM codes, affecting the recognition accuracy of DM codes. Technical issues

[0004] Traditional two-dimensional code recognition methods such as morphological methods and Hough line detection have poor results in locating and detecting DM codes, which affects the recognition accuracy of DM codes. Technical Solutions

[0005] In response to the above-mentioned problems and technical requirements, this application proposes a DM code positioning and recognition method that combines deep learning with pixel-level detection. The technical solution of this application is as follows:

[0006] A DM code positioning and recognition method combining deep learning with pixel-level detection, the DM code positioning and recognition method comprising:

[0007] The coarse positioning model is used to extract the image of the DM code area from the original image to be detected to obtain the DM code coarse positioning image. The coarse positioning model is obtained based on the target detection network training;

[0008] After completing the binary preprocessing of the DM code coarse positioning image, it is traversed pixel by pixel. Based on the pixel value changes of the traversed pixels, several candidate corner point combinations are extracted. Each candidate corner point combination includes the candidate coordinates of each positioning corner point of the DM code.

[0009] The semantic segmentation model is used to perform semantic segmentation on the DM code coarse positioning image to obtain the DM code outline, and several candidate corner point combinations are obtained based on the vertex coordinates of the minimum circumscribed oblique rectangle of the DM code outline;

[0010] All candidate corner point combinations obtained are traversed in sequence. For each candidate corner point combination traversed, the DM code coarse positioning image is parsed pixel by pixel based on the candidate coordinates of each positioning corner point contained in the candidate corner point combination to obtain the corresponding candidate standard DM code image. When the candidate standard DM code image is decoded successfully, the positioning and recognition of the DM code is completed. When the candidate standard DM code image is decoded unsuccessfully, the next candidate corner point combination is traversed.

[0011] A further technical solution is to extract several candidate corner point combinations based on the pixel value changes of the traversed pixels, including:

[0012] The pixels in the DM code coarse positioning image are traversed row by row, and the horizontal separation points in each row of pixels are extracted. The horizontal separation points whose row coordinates are within the same interval in each pixel row are fitted with the least squares method to obtain a vertical straight line. Several vertical straight lines are fitted in total. The horizontal separation points are black pixels adjacent to white pixels in the same row.

[0013] The pixels in the DM code coarse positioning image are traversed column by column and the vertical separation points in each column of pixels are extracted. The vertical separation points whose column coordinates are within the same interval in each pixel column are fitted with the least squares method to obtain a horizontal straight line. Several horizontal straight lines are obtained by fitting. The vertical separation points are black pixels adjacent to white pixels in the same column.

[0014] A plurality of L-shaped candidate positioning marks are obtained based on the obtained plurality of longitudinal straight lines and the obtained plurality of transverse straight lines, and a corresponding candidate corner point combination is extracted according to each candidate positioning mark.

[0015] A further technical solution is to obtain a plurality of L-shaped candidate positioning marks based on the obtained plurality of longitudinal straight lines and the obtained plurality of transverse straight lines, including:

[0016] For each longitudinal line obtained by fitting, when the proportion of white pixels in each of the plurality of parallel scanning columns on one side of the longitudinal line of the DM code coarse positioning image reaches a first threshold, and the proportion of black pixels in each of the plurality of parallel scanning columns on the other side of the longitudinal line of the DM code coarse positioning image reaches a second threshold, the longitudinal line is determined to be a candidate longitudinal boundary line; otherwise, the longitudinal line is filtered out;

[0017] For each horizontal line obtained by fitting, when the proportion of white pixels in each of the several parallel scanning lines on one side of the horizontal line of the DM code coarse positioning image reaches a third threshold, and the proportion of black pixels in each of the several parallel scanning lines on the other side of the horizontal line of the DM code coarse positioning image reaches a fourth threshold, the horizontal line is determined to be a candidate horizontal boundary line; otherwise, the horizontal line is filtered out;

[0018] A plurality of L-shaped candidate positioning marks are obtained based on the retained plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines.

[0019] A further technical solution is to obtain a plurality of L-shaped candidate positioning marks based on the retained plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines, including:

[0020] A plurality of boundary line combinations are obtained by screening the plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines, each boundary line combination including a longitudinal candidate boundary line and a transverse candidate boundary line, and the angle between the longitudinal candidate boundary line and the transverse candidate boundary line in the same boundary line combination is within an angle range of 90°±θ, where θ is an error angle;

[0021] For each boundary line combination, a longitudinal boundary line segment is extracted based on the pixel values ​​of the pixel points on the longitudinal candidate boundary line in the boundary line combination of the DM code coarse positioning image, and a transverse boundary line segment is extracted based on the pixel values ​​of the pixel points on the transverse candidate boundary line in the boundary line combination of the DM code coarse positioning image, thereby obtaining an L-shaped candidate positioning marker composed of the longitudinal boundary line segment and the transverse boundary line segment.

[0022] A further technical solution is that the extracted longitudinal boundary line segments and transverse boundary line segments include:

[0023] Connect the black pixels on the candidate longitudinal boundary line in the DM code coarse positioning image and whose pixel distance does not exceed the distance threshold into a line segment, and use the continuous line segment formed by the black pixels on the candidate longitudinal boundary line as the longitudinal boundary segment;

[0024] The black pixels in the DM code coarse positioning image that are located on the horizontal boundary line segment and whose pixel distance does not exceed the distance threshold are connected into a line segment, and the continuous line segment formed by the black pixels on the horizontal boundary line segment is used as the horizontal boundary line segment.

[0025] A further technical solution is to obtain a corresponding candidate standard DM code image based on each candidate corner point combination, including:

[0026] Perform affine transformation on the DM code coarse positioning image according to the candidate coordinates of the four positioning corner points in the candidate corner point combination to obtain the attitude calibrated image;

[0027] After completing the binary preprocessing of the posture calibrated image, a pixel-by-pixel traversal is performed. Based on the changes in the pixel values ​​of the traversed pixels, an L-shaped positioning mark composed of a longitudinal positioning line segment and a transverse positioning line segment is extracted. The image of the posture calibrated image after the binary preprocessing is located within the rectangular area formed by the positioning mark as the target segmentation image;

[0028] Determine the code element width based on the longitudinal positioning line segment and the transverse positioning line segment, and divide the target partition image into grids according to the code element width to obtain a plurality of grids;

[0029] All pixels in a grid where the number of black pixels is greater than the number of white pixels are set as black pixels, and all pixels in a grid where the number of white pixels is greater than the number of black pixels are set as white pixels to obtain a candidate standard DM code image.

[0030] A further technical solution is to determine the code element width and perform grid division according to the code element width, including:

[0031] Starting from the longitudinal positioning line segment, sequentially detecting each pixel column adjacent to the longitudinal positioning line segment along the direction toward the other end of the transverse positioning line segment; when the proportion of black pixels in the detected pixel column reaches a second threshold, determining that the currently detected pixel column is a valid positioning column and continuing to detect the next pixel column along the detection direction; otherwise, terminating the detection and using the total number of the longitudinal positioning line segment and all valid positioning columns as the first code element width;

[0032] Starting from the horizontal positioning line segment, sequentially detecting each continuous pixel row adjacent to the horizontal positioning line segment along a direction toward the other end of the vertical positioning line segment; when the proportion of black pixels in the detected pixel row reaches a fourth threshold, determining that the currently detected pixel row is a valid positioning row and continuing to detect the next pixel row along the detection direction; otherwise, terminating the detection and using the total number of the horizontal positioning line segment and all valid positioning rows as the second code element width;

[0033] The target divided image is divided into a number of columns along the row direction according to the first code element width, and the target divided image is divided into a number of rows along the column direction according to the second code element width, thereby completing the grid division.

[0034] A further technical solution is to perform binarization preprocessing on any one of the images to be processed, the DM code coarse positioning image and the attitude calibrated image, including:

[0035] The grayscale histogram of the pixels in the central area of ​​the image to be processed is calculated and histogram equalization is performed; the image to be processed after histogram equalization is subjected to USM sharpening and then divided into several local areas, and adaptive binarization processing is performed on each local area according to the pixel values ​​of the pixels in each local area.

[0036] A further technical solution is to obtain several candidate corner point combinations based on the coordinates of the minimum oblique rectangle circumscribing the outline of the DM code, including:

[0037] The coordinates of the four vertices of the minimum oblique rectangle circumscribing the outline of the DM code are staggered and arranged in a clockwise order to obtain four candidate corner point combinations.

[0038] A further technical solution is to sequentially traverse all candidate corner point combinations including:

[0039] The traversal is performed in the order of first traversing each candidate corner point combination obtained by pixel-by-pixel traversal, and then traversing each candidate corner point combination obtained by the semantic segmentation model. Beneficial effects

[0040] This application discloses a DM code positioning and recognition method that combines deep learning with pixel-level detection. The method first uses a target detection algorithm to perform coarse positioning to extract a DM code coarse positioning image. Then, a pixel-by-pixel traversal method and a deep learning method are combined to extract candidate corner point combinations. DM codes are then recognized based on the candidate corner point combinations obtained by the two methods. By identifying corner point coordinates, DM codes can be positioned efficiently and accurately. The combination of the pixel-by-pixel traversal method and the deep learning method allows the candidate corner point combinations extracted by the two methods to complement each other, thereby ensuring recognition speed while improving the accuracy and success rate of DM code positioning and recognition, and achieving better recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a standard structure diagram of DM code.

[0042] FIG2 is a flow chart of a DM code positioning and identification method according to an embodiment of the present application.

[0043] FIG3 is a flowchart of a method for extracting candidate corner point combinations by traversing pixel by pixel in one embodiment.

[0044] FIG4 is a schematic diagram of extracting horizontal separation points and vertical separation points from a local image in an example.

[0045] FIG5 is a flow chart of a method for obtaining a candidate standard DM code image based on candidate corner point combination analysis in one embodiment. Modes for Carrying Out the Invention

[0046] The specific implementation of this application will be further described below with reference to the accompanying drawings.

[0047] This application discloses a method for locating and identifying DM codes using deep learning combined with pixel-level detection. This method is based on the standard format of DM codes, as shown in Figure 1. A DM code includes a data region, a border-seeking region, and a blank region. The data region of a DM code, as shown in (a) of Figure 1, consists of a two-dimensional grid of rectangular code blocks of identical dimensions. All code blocks within the data region have a horizontal side length of a and a vertical side length of b. Black and white code blocks are randomly distributed within the data region.

[0048] The edge-seeking area, shown in (b) of Figure 1, is located around the data area and includes a positioning marker. The positioning marker is an L-shaped structure consisting of a longitudinal positioning edge 1 and a transverse positioning edge 2. The transverse length of longitudinal positioning edge 1 is also a, covering the entire longitudinal extent of the data area. The transverse positioning edge 2 has a longitudinal length b, covering the entire transverse extent of the data area. Therefore, longitudinal positioning edge 1 is equivalent to a column consisting of several symbol blocks, and transverse positioning edge 2 is equivalent to a row consisting of several symbol blocks. All symbol blocks within longitudinal positioning edge 1 and transverse positioning edge 2 are black.

[0049] The edge-finding area also includes a clock mark, an L-shaped structure consisting of a longitudinal clock edge 3 and a transverse clock edge 4. The longitudinal positioning edge 1 and the longitudinal clock edge 3 form the two transverse boundaries of the data area, while the transverse positioning edge 2 and the transverse clock edge 4 form the two transverse boundaries of the data area. The transverse length of the longitudinal clock edge 3 is also a, and its longitudinal length covers the entire longitudinal extent of the data area. The transverse clock edge 4 has a longitudinal length b, and its transverse length covers the entire transverse extent of the data area. Therefore, the longitudinal clock edge 3 is equivalent to a column consisting of several symbol blocks, and the transverse clock edge 4 is equivalent to a row consisting of several symbol blocks. Within the longitudinal clock edge 3, black symbol blocks and white symbol blocks alternate, and the number of black symbol blocks and white symbol blocks are equal. Within the transverse clock edge 4, black symbol blocks and white symbol blocks alternate, and the number of black symbol blocks and white symbol blocks are equal.

[0050] The sum of the number of black and white symbol blocks on the horizontal clock edge 4 is the horizontal dimension of the DM code, while the sum of the number of black and white symbol blocks on the vertical clock edge 4 is the vertical dimension of the DM code. Based on the above standard structure of the DM code, when the data area includes M rows and N columns of symbol blocks, the vertical dimension of the DM code is M+2 and the horizontal dimension is N+2. Most DM codes have the same vertical and horizontal dimensions, but some have different vertical and horizontal dimensions.

[0051] The blank area is shown in (c) in Figure 1. The blank area is located around the periphery of the edge-seeking area. The blank area covers the area with a horizontal interval d1 and a vertical interval d2 from the edge-seeking area. The intervals d1 and d2 are both custom known parameters, which are generally fixed default values ​​in the industry.

[0052] The standard form of a complete DM code, consisting of a data area, a search area, and a blank area, is shown in Figure 1 (d). The four vertices of the search area of ​​the DM code serve as the four positioning corners of the DM code. Generally, the vertex where the horizontal search edge 2 intersects with the vertical clock edge 3 is designated as positioning corner A. The other positioning corners are determined in a clockwise direction. As shown in Figure 1 (d), the vertex where the two search edges intersect is designated as positioning corner B, the vertex where the vertical search edge 1 intersects with the horizontal clock edge 4 is designated as positioning corner C, and the vertex where the two clock edges intersect is designated as positioning corner D.

[0053] Based on the standard structure of the DM code as shown in FIG1 , the DM code positioning and identification method includes the following steps, please refer to the flowchart shown in FIG2 :

[0054] Step 1: Use a coarse positioning model to extract an image of the area where the DM code is located from the original image to be detected to obtain a DM code coarse positioning image.

[0055] The coarse positioning model is pre-trained based on the target detection network, and the coarse positioning model extracts the DM code coarse positioning image from the original image to be detected by the target detection method. In one embodiment, the coarse positioning model is pre-trained based on the picodet model.

[0056] In one embodiment, extracting a DM code coarse positioning image using a coarse positioning model includes:

[0057] The original image to be detected is fed into the coarse positioning model for object detection. This generates several detection boxes and their confidence scores. Detection boxes with confidence scores that meet the threshold are retained, while those with confidence scores that do not meet the threshold are deleted. The threshold can be customized based on actual needs.

[0058] The retained detection frames are further screened using non-maximum suppression. This involves first determining the detection frame with the highest confidence score, then traversing each of the remaining detection frames and calculating the overlap area between the image within the traversed detection frame and the image within the detection frame with the highest confidence score. If the overlap area reaches the overlap area threshold, the traversed detection frame is deleted; otherwise, it is retained. The overlap area threshold can be customized based on actual needs.

[0059] After traversing all detection frames, the screening is completed, and the image in each remaining detection frame is the extracted DM code coarse positioning image.

[0060] This step requires the use of a coarse positioning model, so a method for training the coarse positioning model is also included before using the coarse positioning model. In one embodiment, training the coarse positioning model includes the following process: first, a sample image containing a DM code is collected, and then the area where the DM code is located in the sample image is annotated to construct a training data set. The training data set is then used to train the model based on the target detection network to obtain the required coarse positioning model. In one use case, the training batch is 64, that is, 64 images are loaded into the memory at one time, and back propagation is performed once after completing the forward propagation of 64 images. A stochastic gradient descent algorithm with momentum is used, the momentum is set to 0.9, the initial learning rate is set to 0.4, and the total epoch is 300.

[0061] Step 2: After completing the binarization preprocessing of the DM code coarse positioning image, a pixel-by-pixel traversal is performed. Based on the pixel value changes of the traversed pixels, several candidate corner point combinations are extracted. Each extracted candidate corner point combination includes the candidate coordinates of each positioning corner point of the DM code.

[0062] In one embodiment, pre-binarization of the DM code coarse positioning image includes calculating a grayscale histogram of pixels within a central region of the image to be processed and performing histogram equalization. The central region is defined in advance. The histogram-equalized DM code coarse positioning image is then subjected to unsharp masking and divided into a plurality of local regions. Adaptive binarization is then performed on each local region based on the pixel values ​​within the local region.

[0063] In step 3, the semantic segmentation model is used to perform semantic segmentation on the DM code coarse positioning image to extract the DM code outline. The DM code outline is then converted into a binary image, and the minimum circumscribed oblique rectangle of the DM code outline is found in the binary image. Finally, based on the vertex coordinates of the minimum circumscribed oblique rectangle of the DM code outline, several candidate corner point combinations are obtained. The meaning of the candidate corner point combinations finally extracted is the same as in step 2.

[0064] As shown in Figure 1, the DM code has a regular shape, resulting in excellent semantic segmentation results. Therefore, this step uses a pre-trained semantic segmentation model to extract the DM code outline. Therefore, a pre-training step is also required before using the semantic segmentation model. This involves constructing coarse positioning sample images, performing pixel-level annotation of the closed-loop outlines of the DM code in these coarse positioning sample images to construct a training dataset, and then training the model using the paddleseg model. The training batch size is set to 64, the total number of iterations is set to 80,000, and a stochastic gradient descent algorithm with momentum is used. The momentum is set to 0.9, and the initial learning rate is set to 0.005.

[0065] In step 2, the four corner points can be directly and accurately located during the pixel-by-pixel traversal process, thereby determining the candidate coordinates of the four corner points. In step 3, after extracting the minimum circumscribed oblique rectangle of the DM code's outline, only the four vertex coordinates of the minimum circumscribed oblique rectangle can be obtained, but it is not possible to determine which corner point of the DM code each of these four vertex coordinates corresponds to. However, the circumferential arrangement order of the four vertex coordinates of the minimum circumscribed oblique rectangle is fixed. Therefore, if the four vertex coordinates of the minimum circumscribed oblique rectangle are staggered in a clockwise order, there must be an accurate correspondence between the vertex coordinates of one candidate corner point combination and the corner point.

[0066] For example, if the four vertex coordinates of the minimum circumscribed oblique rectangle of the extracted DM code outline are (x1, y1), (x2, y2), (x3, y3), and (x4, y4) in the clockwise direction, then vertex coordinate (x1, y1) could correspond to any of corner points A, B, C, or D in Figure 1. However, once vertex coordinate (x1, y1) is fixed to corner point A, it can be determined that (x2, y2) corresponds to corner point B, (x3, y3) corresponds to corner point C, and (x4, y4) corresponds to corner point D, forming a candidate corner point combination. By staggering the vertex coordinates (x1, y1), it can be determined that (x2, y2) corresponds to corner point C, (x3, y3) corresponds to corner point D, and (x4, y4) corresponds to corner point A, forming another candidate corner point combination. Correspondingly, two more candidate corner point combinations can be obtained, thereby obtaining all four candidate corner point combinations. Among these four candidate corner point combinations, there must be one candidate corner point combination with an accurate correspondence, which will be screened and determined through subsequent methods.

[0067] It should be noted that, during actual execution, there is no specific order in which steps 2 and 3 must be executed, and they can generally be executed simultaneously.

[0068] In step 4, all candidate corner point combinations obtained are traversed in sequence. These include the candidate corner point combinations determined by pixel-by-pixel traversal in step 2 and the candidate corner point combinations determined by the semantic segmentation model in step 3. For each candidate corner point combination traversed, the DM code coarse positioning image is parsed pixel by pixel based on the candidate coordinates of each positioning corner point contained in the candidate corner point combination to obtain the corresponding candidate standard DM code image.

[0069] After obtaining a corresponding candidate standard DM code image based on a candidate corner point combination, the candidate standard DM code image is decoded according to a predetermined decoding method. If the candidate standard DM code image is successfully decoded, it indicates that the identified candidate standard DM code image is accurate, and the positioning and recognition of the DM code is completed. If the candidate standard DM code image is unsuccessful, it indicates that the candidate standard DM code image obtained at this time is inaccurate, and the next candidate corner point combination is traversed.

[0070] This method extracts candidate corner combinations using both steps 2 and 3, increasing the recognition success rate. Particularly in poor image quality situations, the pixel-by-pixel traversal method in step 2 may fail to extract candidate corner combinations, or the extracted candidate corner combinations may be highly inaccurate. If the DM code coarse positioning image is processed pixel by pixel based solely on the candidate corner combinations obtained through the pixel-by-pixel traversal, the recognition accuracy and success rate will be low. The semantic segmentation model, however, can accurately locate the DM code outline and obtain candidate corner combinations. Therefore, combining these two candidate corner combinations can improve recognition accuracy and success rates.

[0071] When traversing all candidate corner point combinations in this step, each candidate corner point combination can be traversed in a random order. However, as mentioned above, among the four candidate corner point combinations obtained in step 3, only one candidate corner point combination is accurate, and the other three candidate corner point combinations are inaccurate. If all candidate corner point combinations are mixed together and randomly traversed, the incorrect candidate corner point combination among the four candidate corner point combinations obtained in step 3 will affect the algorithm time consumption. Therefore, in another embodiment, the traversal order adopted is: first traverse the candidate corner point combinations obtained by pixel-by-pixel traversal in step 2 in sequence, and after traversing the candidate corner point combinations obtained by pixel-by-pixel traversal, traverse the candidate corner point combinations obtained by the semantic segmentation model in step 3 in sequence. Random traversal is performed when traversing the candidate corner point combinations obtained in step 2, and random traversal is performed when traversing the candidate corner point combinations obtained in step 3.

[0072] Based on this traversal order, there is a certain probability that recognition can be completed during the process of traversing the candidate corner point combinations obtained in step 2. Only when recognition cannot be achieved, the candidate corner point combinations obtained in step 3 are traversed for supplementary recognition. This can improve recognition efficiency and reduce recognition time while improving recognition accuracy and success rate. When using this traversal order, step 2 can also be executed first, and then the candidate corner point combinations obtained in step 2 are traversed. If recognition is successful, step 3 is not executed. If recognition is still unsuccessful after traversal, step 3 is executed again to obtain the candidate corner point combinations and traverse again. The specific step execution order can be customized as needed.

[0073] In one embodiment, the above step 2 extracts a plurality of candidate corner point combinations based on the pixel value changes of the traversed pixels, and includes the following steps, as shown in the flowchart of FIG3 :

[0074] [Corrected 16.05.2024 in accordance with Rule 26] Step 310 involves traversing the pixels in the DM code coarse positioning image row by row and extracting the horizontal separation points within each row of pixels. A horizontal separation point is a black pixel adjacent to a white pixel in the same row. For example, in the example of Figure 4 , assuming the partial image of the DM code coarse positioning image is shown in (a), the horizontal separation points extracted by traversing the partial image (a) row by row are represented by white grid shading as shown in (b).

[0075] A least squares fit is then performed on the horizontal separation points within the same range of row coordinates in each pixel row to obtain a vertical line. The range of each row coordinate interval is set according to actual needs. After this operation, a total of several vertical lines are obtained. Each vertical line is basically along the vertical direction. However, due to actual image deviation, it is generally difficult to ensure that the vertical lines are completely vertical.

[0076] [Corrected 16.05.2024 in accordance with Rule 26] Step 320: The DM code coarsely locates pixels in the image column by column and extracts vertical separation points within each column. Vertical separation points are black pixels adjacent to white pixels in the same column. Similar to the row-by-row traversal method, in the example of Figure 4, the vertical separation points extracted by row-by-row traversal of partial image (a) are represented by white grid shading, as shown in (c).

[0077] Similarly, a least squares fitting is performed on the longitudinal separation points of the column direction coordinates in each pixel column within the same interval to obtain a horizontal straight line, and several horizontal straight lines are obtained by fitting. Similarly, the interval range divisions of the column direction coordinates are set according to actual needs.

[0078] Likewise, there is no specific execution order for step 310 and step 320, and they can generally be executed simultaneously.

[0079] In step 330 , a plurality of L-shaped candidate positioning markers are obtained based on the obtained plurality of longitudinal straight lines and the obtained plurality of transverse straight lines, and a corresponding candidate corner point combination is extracted according to each candidate positioning marker.

[0080] (1) The vertical and horizontal straight lines obtained in step 310 and step 320 cover the entire area where the DM code is located. The number of them is often large. In order to reduce the amount of calculation and improve the detection efficiency, the vertical and horizontal straight lines obtained are first screened. Since the ultimate goal is to obtain a candidate corner point combination, it is actually necessary to detect the outer circle boundary of the edge-seeking area of ​​the DM code. The outer circle boundary can be determined based on the positioning mark, so the focus is on detecting the outer circle boundary of the positioning mark. As can be seen from Figure 1, the outer circle boundary of the positioning mark has the following characteristics: one side of the outer circle boundary of the longitudinal positioning edge 1 (the left side in Figure 1) is in the blank area, so theoretically all pixels are white. Since the pixel size is much smaller than the size of a single code element block in the DM code, the other side of the outer circle boundary of the longitudinal positioning edge 1 (the right side in Figure 1) is located in the code element block on the longitudinal positioning edge 1, so theoretically all pixels are black. The horizontal positioning edge 2 also has this feature. One side of the outer circle boundary of the horizontal positioning edge 2 (the lower side in Figure 1) is in the blank area, so theoretically all pixels are white. The other side of the outer circle boundary of the horizontal positioning edge 2 (the upper side in Figure 1) is located in the code element block on the horizontal positioning edge 2, so theoretically all pixels are black.

[0081] The pixels on both sides of the outer boundary of the two positioning edges to be found must meet the above characteristics. Therefore, based on this characteristic, other vertical and horizontal lines that do not meet the above requirements are first filtered out:

[0082] For each fitted vertical line, when the proportion of white pixels in each of the several parallel scanning columns of the DM code coarse positioning image on one side of the vertical line reaches a first threshold, and the proportion of black pixels in each of the several parallel scanning columns of the DM code coarse positioning image on the other side of the vertical line reaches a second threshold, the vertical line is determined to be a candidate vertical boundary line; otherwise, the vertical line is filtered out. The parallel scanning columns of each vertical line are parallel to the vertical line, and the number of parallel scanning columns detected is configured as needed, but does not exceed the number of pixel columns included in the horizontal interval d1 between the blank area and the edge search area, nor does it exceed the number of pixel columns covered by a single code element block width. The first and second thresholds can be customized, primarily to account for the fact that errors can cause the blank area to not be entirely white pixels, and the positioning edge to not be entirely black pixels. When a certain proportion is reached, it is considered to have basically met the requirements.

[0083] Similarly, for each horizontal line obtained by fitting, when the proportion of white pixels in each of the several parallel scanning lines of the DM code coarse positioning image on one side of the horizontal line reaches the third threshold, and the proportion of black pixels in each of the several parallel scanning lines of the DM code coarse positioning image on the other side of the horizontal line reaches the fourth threshold, the horizontal line is determined to be a candidate horizontal boundary line, otherwise the horizontal line is filtered out. Similarly, the parallel scanning lines of each horizontal line are parallel to the horizontal line, and the number of parallel scanning lines detected is configured as needed, but does not exceed the number of pixel columns included in the longitudinal interval d2 between the blank area and the edge search area, nor does it exceed the number of pixel columns covered by the width of a single code element block. The third and fourth thresholds can be customized to reduce the impact of acquisition errors.

[0084] (2) Then, several L-shaped candidate positioning marks can be obtained based on the retained several longitudinal candidate boundary lines and several transverse candidate boundary lines. After the above processing, the retained longitudinal candidate boundary lines and transverse candidate boundary lines basically meet the characteristics of the outer circle boundary. However, as mentioned above, the two positioning edges do not exist independently, but together constitute the positioning mark, and the positioning mark in the structure of the standard DM code is L-shaped, and the angle between the two positioning edges is 90°. Based on this feature, several boundary line combinations are first obtained based on the several longitudinal candidate boundary lines and several transverse candidate boundary lines. Each boundary line combination includes a longitudinal candidate boundary line and a transverse candidate boundary line, and the angle between the longitudinal candidate boundary line and the transverse candidate boundary line in the same boundary line combination is within the angle range of 90°±θ, where θ is the error angle and can be customized.

[0085] For each boundary line combination, a longitudinal boundary line segment is extracted based on the pixel values ​​of the pixel points on the longitudinal candidate boundary line in the boundary line combination of the DM code coarse positioning image, and a transverse boundary line segment is extracted based on the pixel values ​​of the pixel points on the transverse candidate boundary line in the boundary line combination of the DM code coarse positioning image, thereby obtaining an L-shaped candidate positioning marker composed of the longitudinal boundary line segment and the transverse boundary line segment. This includes:

[0086] In the DM code coarse positioning image, all black pixels located on the candidate vertical boundary line and whose pixel distance does not exceed the distance threshold are connected into line segments. The continuous line segments formed by the black pixels on the candidate vertical boundary line are used as the vertical boundary segments. In the DM code coarse positioning image, all black pixels located on the horizontal boundary line and whose pixel distance does not exceed the distance threshold are connected into line segments. The continuous line segments formed by the black pixels on the horizontal boundary line are used as the horizontal boundary segments.

[0087] The longitudinal and transverse boundary segments form an L-shaped candidate positioning marker. Due to errors, the candidate positioning markers extracted in this step are not necessarily strictly L-shaped, that is, the angle between the longitudinal and transverse boundary segments is not necessarily a perfect 90° angle. After determining the candidate positioning markers, the pixel coordinates of each endpoint of the candidate positioning marker can be used as the candidate coordinates of the corresponding positioning corner points A, B, and C. Then, based on the coordinate relationship of the rectangular structure, the candidate coordinates of positioning corner point D can be calculated accordingly, resulting in a candidate corner point combination. Depending on the actual image situation, the candidate corner point combinations extracted in this step may be 0, 1, or more.

[0088] In one embodiment, the above step 4 includes the following steps of performing pixel-by-pixel parsing on the DM code coarse positioning image based on the candidate coordinates of each positioning corner point included in the candidate corner point combination to obtain the corresponding candidate standard DM code image, as shown in the flowchart of FIG5 :

[0089] Step 510 , based on the characteristic of the standard DM code having a rectangular structure, an affine transformation is performed on the DM code coarse positioning image according to the candidate coordinates of the four positioning corner points in the candidate corner point combination to obtain a posture-calibrated image.

[0090] Step 520, after completing the binarization preprocessing of the posture calibrated image, perform pixel-by-pixel traversal, extract the L-shaped positioning mark composed of the longitudinal positioning line segment and the transverse positioning line segment based on the change of the pixel value of the traversed pixel point, and extract the image of the posture calibrated image after completing the binarization preprocessing that is located in the rectangular area composed of the positioning mark as the target segmentation image.

[0091] The method for binarization preprocessing the attitude-calibrated image is the same as the method for binarization preprocessing the DM code coarse positioning image in step 2, including: calculating the grayscale histogram of the pixels in the central area of ​​the attitude-calibrated image and performing histogram equalization, where the range of the central area is predefined. The attitude-calibrated image after histogram equalization is then subjected to unsharp masking and divided into several local areas. Each local area is then adaptively binarized based on the pixel values ​​of the pixels in each local area.

[0092] The posture calibrated image after the binarization preprocessing is traversed pixel by pixel. The method for traversing to obtain the longitudinal positioning line segment is the same as the method for traversing to obtain the longitudinal boundary line segment in step 2. The method for traversing to obtain the transverse positioning line segment is the same as the method for traversing to obtain the transverse boundary line segment in step 2. This embodiment will not be repeated.

[0093] As mentioned above, the L-shaped positioning mark can mark the area where the data area and the edge-finding area are located in the DM code, so the extracted target segmentation image is theoretically the image of the area where the data area and the edge-finding area are located.

[0094] Step 530: Determine the symbol width based on the longitudinal positioning line segment and the transverse positioning line segment.

[0095] The symbol width consists of the horizontal side length a and the vertical side length b of a single symbol block. As shown in Figure 1, for a standard DM code, the horizontal side length a of a single symbol block is equal to the width of the vertical positioning edge 1, and the vertical side length b of a single symbol block is equal to the width of the horizontal positioning edge 2. Therefore, the symbol width can be determined by detecting the widths of the vertical positioning edge 1 and the horizontal positioning edge 2.

[0096] As mentioned above, theoretically, the two positioning edges are all black pixels. Therefore, based on this feature, we can detect which pixel rows and columns are inside the two positioning edges, including:

[0097] Starting from the longitudinal positioning line segment, each pixel column adjacent to the longitudinal positioning line segment is detected in sequence along the direction toward the other end of the transverse positioning line segment. When the proportion of black pixels in the detected pixel column reaches the second threshold, it is determined that the currently detected pixel column is a valid positioning column. A valid positioning column means that the pixel column is located inside the longitudinal positioning edge 1. Then continue to detect the next pixel column along the detection direction. When the proportion of black pixels in the detected pixel column does not reach the second threshold, the detection is terminated, and the total number of columns of the previously detected longitudinal positioning line segment and all valid positioning columns is used as the first code element width. The meaning of the second threshold here is as described above. It is a threshold set to avoid detection errors.

[0098] Similarly, starting from the horizontal positioning line segment, each continuous pixel row adjacent to the horizontal positioning line segment is detected in sequence along the direction toward the other end of the vertical positioning line segment. When the proportion of black pixels in the detected pixel row reaches the fourth threshold, the currently detected pixel row is determined to be a valid positioning row. A valid positioning row means that the pixel row is located within horizontal positioning edge 2. Detection then continues along the detection direction to the next pixel row. When the proportion of black pixels in the detected pixel row does not reach the fourth threshold, the detection ends, and the total number of rows of the previously detected horizontal positioning line segment and all valid positioning rows is used as the second code element width.

[0099] In general, the symbol widths of a single symbol block can be assumed to be equal in both directions, so it's possible to detect only the first or second symbol width. However, considering that symbol widths can vary in some cases, and that in actual industrial environments, influenced by various factors such as industrial carving and photography, even if the symbol widths are equal in both directions, there may still be errors in the image. Therefore, to ensure recognition accuracy, this step detects the first and second symbol widths in both directions separately.

[0100] Step 540: Grid-divide the target segmented image according to the symbol width to obtain a plurality of grids. As described in part 1 of FIG. 1 , the data region and edge-search region of the DM code can be considered to be composed of an arrangement of symbol blocks of equal size. Therefore, after determining the symbol width of the symbol block, the determined target segmented image can be divided into a plurality of grids according to the symbol width, with each grid corresponding to a symbol block. This includes dividing the target segmented image into a plurality of columns along the row direction according to the first symbol width, and dividing the target segmented image into a plurality of rows along the column direction according to the second symbol width, thereby completing the grid division.

[0101] In step 550, all pixels in a grid where the number of black pixels is greater than the number of white pixels are set to black pixels as black symbol blocks, and all pixels in a grid where the number of white pixels is greater than the number of black pixels are set to white pixels as white symbol blocks, thereby obtaining a candidate standard DM code image. After obtaining the candidate standard DM code image, the dimensions of the candidate standard DM code image can be determined by detecting the total number of black symbol blocks and white symbol blocks on the clock marker.

[0102] The above description is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and variations directly derived or imagined by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the scope of protection of the present application.

Claims

1. A DM code positioning and recognition method combining deep learning with pixel-level detection, characterized in that: The DM code positioning and identification method includes: Extracting an image of the DM code region from the original image to be detected using a coarse positioning model to obtain a DM code coarse positioning image, wherein the coarse positioning model is trained based on a target detection network; After completing the binarization preprocessing of the DM code coarse positioning image, a pixel-by-pixel traversal is performed, and a plurality of candidate corner point combinations are extracted based on the pixel value changes of the traversed pixels, each candidate corner point combination including the candidate coordinates of each positioning corner point of the DM code; Performing semantic segmentation extraction on the DM code coarse positioning image using a semantic segmentation model to obtain the outline of the DM code, and obtaining a plurality of candidate corner point combinations based on the vertex coordinates of the minimum circumscribed oblique rectangle of the outline of the DM code; All candidate corner point combinations obtained are traversed in sequence. For each traversed candidate corner point combination, the DM code coarse positioning image is parsed pixel by pixel based on the candidate coordinates of each positioning corner point contained in the candidate corner point combination to obtain a corresponding candidate standard DM code image. When the candidate standard DM code image is decoded successfully, the positioning and recognition of the DM code is completed. When the candidate standard DM code image is decoded unsuccessfully, the next candidate corner point combination is traversed.

2. The DM code positioning and identification method according to claim 1, characterized in that: The extraction of several candidate corner point combinations based on the pixel value changes of the traversed pixels includes: Traversing the pixels in the DM code coarse positioning image row by row and extracting the horizontal separation points in each row of pixels, performing least squares fitting on the horizontal separation points in each pixel row whose row coordinates are within the same interval to obtain a vertical straight line, and fitting a total of several vertical straight lines; wherein the horizontal separation points are black pixels adjacent to white pixels in the same row; Traversing the pixels in the DM code coarse positioning image column by column and extracting the longitudinal separation points in each column of pixels, performing least squares fitting on the longitudinal separation points in each pixel column whose column coordinates are within the same interval to obtain a horizontal straight line, and fitting a total of several horizontal straight lines; wherein the longitudinal separation points are black pixels adjacent to white pixels in the same column; A plurality of L-shaped candidate positioning marks are obtained based on the obtained plurality of longitudinal straight lines and the obtained plurality of transverse straight lines, and a corresponding candidate corner point combination is extracted according to each candidate positioning mark.

3. The DM code positioning and identification method according to claim 2, characterized in that: Based on the obtained plurality of longitudinal lines and the obtained plurality of transverse lines, a plurality of L-shaped candidate positioning marks are obtained, including: For each longitudinal straight line obtained by fitting, when the proportion of white pixels in each of the plurality of parallel scanning columns on one side of the longitudinal straight line of the DM code coarse positioning image reaches a first threshold, and the proportion of black pixels in each of the plurality of parallel scanning columns on the other side of the longitudinal straight line of the DM code coarse positioning image reaches a second threshold, the longitudinal straight line is determined to be a candidate longitudinal boundary line; otherwise, the longitudinal straight line is filtered out; For each horizontal line obtained by fitting, when the proportion of white pixels in each of the plurality of parallel scanning lines on one side of the horizontal line of the DM code coarse positioning image reaches a third threshold, and the proportion of black pixels in each of the plurality of parallel scanning lines on the other side of the horizontal line of the DM code coarse positioning image reaches a fourth threshold, the horizontal line is determined to be a candidate horizontal boundary line; otherwise, the horizontal line is filtered out; A plurality of L-shaped candidate positioning marks are obtained based on the retained plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines.

4. The DM code positioning and identification method according to claim 3, characterized in that: Based on the retained plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines, a plurality of L-shaped candidate positioning marks are obtained, including: A plurality of boundary line combinations are obtained by screening the plurality of longitudinal candidate boundary lines and the plurality of transverse candidate boundary lines, each boundary line combination including a longitudinal candidate boundary line and a transverse candidate boundary line, and the angle between the longitudinal candidate boundary line and the transverse candidate boundary line in the same boundary line combination is within an angle range of 90°±θ, where θ is an error angle; For each boundary line combination, a longitudinal boundary line segment is extracted based on the pixel values ​​of the pixel points on the longitudinal candidate boundary line in the boundary line combination of the DM code coarse positioning image, and a transverse boundary line segment is extracted based on the pixel values ​​of the pixel points on the transverse candidate boundary line in the boundary line combination of the DM code coarse positioning image, thereby obtaining an L-shaped candidate positioning mark composed of the longitudinal boundary line segment and the transverse boundary line segment.

5. The DM code positioning and identification method according to claim 4, characterized in that: The extracted longitudinal boundary segments and transverse boundary segments include: Connect the black pixels on the candidate longitudinal boundary line in the DM code coarse positioning image and whose pixel distance does not exceed the distance threshold into a line segment, and use the continuous line segment formed by the black pixels on the candidate longitudinal boundary line as the longitudinal boundary segment; The black pixels in the DM code coarse positioning image that are located on the horizontal boundary line segment and whose pixel distance does not exceed the distance threshold are connected into a line segment, and the continuous line segment formed by the black pixels on the horizontal boundary line segment is used as the horizontal boundary line segment.

6. The DM code positioning and identification method according to claim 1, characterized in that: Based on each candidate corner point combination, the corresponding candidate standard DM code image includes: Performing an affine transformation on the DM code coarse positioning image according to the candidate coordinates of the four positioning corner points in the candidate corner point combination to obtain a posture-calibrated image; After completing the binarization preprocessing of the posture calibrated image, a pixel-by-pixel traversal is performed, and based on the pixel value changes of the traversed pixel points, an L-shaped positioning mark composed of a longitudinal positioning line segment and a transverse positioning line segment is extracted, and an image of the posture calibrated image after completing the binarization preprocessing and located within the rectangular area composed of the positioning mark is extracted as a target segmentation image; Determining a code element width based on the longitudinal positioning line segment and the transverse positioning line segment, and performing grid division on the target divided image according to the code element width to obtain a plurality of grids; All pixels in a grid where the number of black pixels is greater than the number of white pixels are set as black pixels, and all pixels in a grid where the number of white pixels is greater than the number of black pixels are set as white pixels to obtain a candidate standard DM code image.

7. The DM code positioning and identification method according to claim 6, characterized in that: Determining the symbol width and dividing the grid according to the symbol width includes: Starting from the longitudinal positioning line segment, sequentially detecting each pixel column adjacent to the longitudinal positioning line segment along the direction toward the other end of the transverse positioning line segment; when the proportion of black pixels in the detected pixel column reaches a second threshold, determining that the currently detected pixel column is a valid positioning column and continuing to detect the next pixel column along the detection direction; otherwise, terminating the detection and using the total number of the longitudinal positioning line segment and all valid positioning columns as the first code element width; Starting from the horizontal positioning line segment, sequentially detecting each continuous pixel row adjacent to the horizontal positioning line segment along a direction toward the other end of the vertical positioning line segment; when the proportion of black pixels in the detected pixel row reaches a fourth threshold, determining that the currently detected pixel row is a valid positioning row and continuing to detect the next pixel row along the detection direction; otherwise, terminating the detection and using the total number of the horizontal positioning line segment and all valid positioning rows as the second code element width; The target divided image is divided into a number of columns along the row direction according to the first code element width, and the target divided image is divided into a number of rows along the column direction according to the second code element width, thereby completing the grid division.

8. The DM code positioning and identification method according to claim 6, characterized in that: For any one of the images to be processed, the DM code coarse positioning image and the attitude calibrated image, performing binarization preprocessing on the image to be processed includes: The grayscale histogram of the pixel points in the central area of ​​the image to be processed is calculated and histogram equalization is performed; the image to be processed after the histogram equalization is subjected to USM sharpening and then divided into a number of local areas, and adaptive binarization processing is performed on each local area according to the pixel values ​​of the pixel points in each local area.

9. The DM code positioning and identification method according to claim 1, characterized in that: The candidate corner point combinations obtained based on the coordinates of the minimum oblique rectangle circumscribing the outline of the DM code include: The coordinates of the four vertices of the minimum oblique rectangle circumscribing the outline of the DM code are staggered and arranged in a clockwise order to obtain four candidate corner point combinations.

10. The DM code positioning and identification method according to claim 1, characterized in that: All candidate corner point combinations obtained by traversing in sequence include: The traversal is performed in the order of first traversing each candidate corner point combination obtained by pixel-by-pixel traversal, and then traversing each candidate corner point combination obtained by the semantic segmentation model.

Images