CenterNet-based dm code key point detection model construction method and detection method

Abstract

The application discloses a CenterNet-based DM code key point detection model construction method and detection method, first, a DM code is photographed in multiple scales and multiple angles to obtain an image set, DM code type information and position information of four corner points of the image are labeled, then, a CenterNet target detection network is improved, a detection head is designed in a form suitable for DM code key point detection, a loss function of DM code geometric constraint is designed to further optimize DM code key point learning, and in the data enhancement in the training process, a perspective transformation is used to simulate camera view angle change, a detection model is trained to directly obtain 4 key point position information of the DM code in the image, so that the DM code can be accurately positioned. The application can more accurately position the DM code key point, improve the fitting degree of the detected polygon and the DM code, reduce the calculation amount of the post-processing part caused by non-accurate positioning, and reduce the processing time.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a CenterNet-based DM code key point detection model construction method and detection method. BACKGROUND

[0002] With the development of the times and the continuous progress of science and technology, the Internet of Things technology has been widely applied in intelligent manufacturing enterprises. In intelligent manufacturing, a two-dimensional code plays an important role in information interface. A DM code (Data Matrix code) has the characteristics of large coding capacity, high density, high information security, etc. Compared with other two-dimensional bar codes, the DM code can contain the most data information under the same size and density. Therefore, the DM code is widely used in industrial manufacturing.

[0003] Current DM code detection mainly includes two steps, positioning and decoding. Precise and fast positioning of the DM code is a key link of DM code detection, which greatly affects the detection rate. The traditional method for detecting the DM code is mainly through image processing, which extracts the code information from the image through domain method or threshold segmentation. In recent years, with the rapid development of deep learning technology, a large number of target detection algorithms have emerged, which have greatly improved the speed and accuracy compared with the traditional method, such as SSD, YOLO, etc. The current DM code detection algorithm through horizontal two-dimensional frame has improved the speed, but the detection result is only the horizontal envelope of the DM code, which cannot achieve precise positioning. The rotating target detection algorithm is an important branch in the field of target detection in recent years. It is aimed at the detection of objects with orientation angle information. The main idea is to make the boundary box outline the target according to the target orientation, maximize the area difference between the target real area and the boundary box area, so as to accurately separate the rotating target from the background and solve the problem of rotating target detection caused by target density, target proportion, complex background, etc. Although the above method can obtain very good detection results, it has the problems of low detection rate and accuracy, and cannot solve the problem of perspective transformation. The current DM code detection mainly has the following problems: the DM code detection cannot achieve precise positioning, and the detection result contains a lot of redundant information. The DM code detection does not solve the problem of image perspective transformation. SUMMARY

[0004] In view of the problems in the prior art, the purpose of the present application is to provide a CenterNet-based DM code key point detection model construction method and detection method, so as to solve the problems of the prior art that the DM code detection cannot achieve precise positioning and cannot solve the problem of image perspective transformation.

[0005] In order to solve the above technical problems, the present application adopts the following technical solutions:

[0006] A CenterNet-based DM code key point detection model construction method, comprising the following steps:

[0007] Step 1: Obtain an image set: multi-scale and multi-angle shooting is performed on a DM code scene to obtain an image set;

[0008] Step 2: Label the image set: DM code labeling is performed on the image set, and the labeling result is type information and position information of four corner points of the DM code, wherein the four corner points are key points of the DM code;

[0009] Step 3: Train the detection model: input the image set, and perform data enhancement on the image set to obtain DM code type information, center point offset and key point relative to the center point offset, and then obtain key point coordinate output through decoding to train a DM code key point detection model based on the CenterNet network;

[0010] The multi-task detection head output of the CenterNet network comprises:

[0011] 128x128xC, C is the number of type information;

[0012] 128x128x2, 2 is the horizontal and vertical coordinates of the center point offset of the DM code;

[0013] And 128x128x(4x3), 4 is the four corners of the DM code, and 3 is the distance between the key point and the center point, the sin value and the cos value of the included angle between the line connecting the key point and the center point and the horizontal direction;

[0014] The data enhancement on the image set comprises random contrast transformation, horizontal flip, rotation and perspective projection change on the image set;

[0015] The loss function of the above model training comprises a basic loss function and a strengthening loss function; the basic loss function comprises a DM code classification loss function, a DM code center point offset coordinate regression loss function and a DM code key point offset coordinate regression loss function; the strengthening loss function comprises a DM code key point heat map loss function, a standard DM code projection loss function and a DM code polygon IoU loss function.

[0016] The present application also comprises the following technical features:

[0017] Optionally, the multi-task detection head output of the CenterNet network is obtained through the following steps:

[0018] ResNet50 is selected as the feature extraction network, and the 3-class feature maps output by the network are 64*64*512, 32*32*1024 and 16*16*2048 respectively; multi-scale feature fusion is performed to obtain a fused high-resolution feature map of 128*128*64; 1*1 convolution, standardization and ReLU activation function are used on the fused high-resolution feature map to obtain 3 outputs of the multi-task detection head: 128*128*C, C is the number of type information, 128*128*2, 2 is the horizontal and vertical coordinates of the center point offset of the DM code; 128*128*(4*3), 4 is the 4 corner points of the DM code, and 3 is the distance between the key point and the center point, the sin value and the cos value of the angle between the key point and the center point.

[0019] Optionally, the DM code classification loss function has the following formula:

[0020]

[0021] wherein, L c is the DM code classification loss, c is the class, N is the number of positive samples, C is the total number of classes, k is the class index, i and j are the horizontal and vertical coordinates under the feature map 128*128, and α and β are hyperparameters for adjusting the loss weights of positive and negative samples. For each real labeled DM code, a group of response values are generated in the heat feature map using a Gaussian kernel function represents the predicted response value.

[0022] Optionally, the DM code center point offset coordinate regression loss function has the following formula:

[0023]

[0024] wherein, L co is the DM code center point offset coordinate regression loss, co is the center offset, N is the number of positive samples, represents whether there is a DM code target at i,j of the output feature map, obj represents the target, and p ct represents the real image coordinates of the DM code center point in the original image, ct represents the center, represents the real image coordinates of the DM code center point in the feature map M co ; represents the coordinate value at i,j in the feature map M co ;

[0025] Optionally, the DM code key point offset coordinate regression loss function has the following formula:

[0026]

[0027] wherein, L ko is the DM code keypoint offset coordinate regression loss, ko is the keypoint offset, N represents the number of positive samples, represents whether there is a DM code target at i, j of the output feature map, obj represents the target object, p kp represents the real image coordinates of the keypoint in the original image, represents the real image coordinates of the DM code keypoint in the feature map M ko ; represents the coordinate value at i, j in the feature map M ko , and S is a down-sampling factor.

[0028] Optionally, the DM code keypoint heat map loss function has the following formula:

[0029]

[0030] wherein, L kp is the DM code keypoint heat map loss, kp is the keypoint, N is the number of positive samples, C is the total number of categories, k is the category index, i and j are the horizontal and vertical coordinates in the feature map M kp , and α and β are hyperparameters respectively used to adjust the loss weights of positive and negative samples. For each real labeled DM code keypoint, a group of response values are generated in the heat feature map M kp using a Gaussian kernel function represents the predicted response value.

[0031] Optionally, the standard DM code projection loss function has the following formula:

[0032]

[0033] wherein, L proj represents the standard DM code projection loss, proj represents the projection, N represents the number of positive samples, represents whether there is a DM code target at i, j of the output feature map, and represents the predicted value and the real value of the standard DM code center point at i, j of the output feature map, ct represents the center, kp represents the keypoint, pstd represents the predicted standard, and std represents the real standard, and represents the predicted value and the true value of the standard DM code keypoint at the output feature map i, j, dist(·) is a distance function of coordinates, λ stdct and λ stdkp are weight values, stdct represents standard center, and stdkp represents standard keypoint.

[0034] Optionally, the DM code polygon IoU loss function has the following formula:

[0035]

[0036] wherein L iou represents the DM code polygon IoU loss, iou represents intersection over union, N is the number of positive samples, represents whether there is a DM code target at i, j of the output feature map, and respectively represent the polygons formed by the DM code keypoint predicted value and the true value at M ko feature map i, j, poly represents polygon, and IoU represents the operation strategy of the intersection over union loss.

[0037] Optionally, the multi-task loss function is defined by the basic loss function and the enhanced loss function as follows:

[0038] L=λ c L c +λ co L co +λ ko L ko +λ kp L kp +λ proj L proj +λ iou L iou (7)

[0039] wherein λ is a weight for balancing the losses of each part, λ c = 1, L c represents the DM code classification loss, c represents class, λ co = 1, L co represents the DM code center point offset coordinate regression loss, co represents center offset, λ ko = 0.1, L ko represents the DM code keypoint offset coordinate regression loss, ko represents keypoint offset, λ kp = 0.1, L kpThe loss represents the heatmap loss of DM code keypoints, where kp represents the keypoint and λ represents the DM code keypoint. proj =0.1, L proj λ represents the standard DM code projection loss, proj represents the projection, and λ represents the projection. iou =1,L iou This represents the IoU loss of the DM code polygon, where iou represents the intersection-over-union ratio.

[0040] A key point detection method for DM codes based on CenterNet includes the following steps:

[0041] Load the DM code key point detection model based on the CenterNet network, resize the image to be detected to 512×512 and input it into the network to obtain the DM code type information, DM code center point offset information and DM code key point offset information;

[0042] For samples with correct classification results, decode them to extract the predicted coordinates of the center point and key points; then convert the coordinate information onto the original image to draw the prediction box and obtain the precise location of the DM code.

[0043] Compared with the prior art, the present invention has the following technical effects:

[0044] Compared with current methods, the method of this invention can extract accurate target information more quickly, avoids redundant information in horizontal bounding box target detection, and uses the CenterNet structure to make network operations faster. It is more suitable for accurate and rapid detection of DM codes in complex backgrounds and has high practical value. Attached Figure Description

[0045] Figure 1 DM code key point detection network structure diagram;

[0046] Figure 2 Example image of DM code;

[0047] Figure 3 A schematic diagram of heatmap loss for key point detection in DM codes;

[0048] Figure 4 A schematic diagram of the projection loss of the standard DM code;

[0049] Figure 5 A schematic diagram of the polygon IoU loss for DM codes;

[0050] Figure 6 Example graph for data augmentation;

[0051] Figure 7 A visualization of the DM code detection results. DETAILED DESCRIPTION

[0052] The following gives specific embodiments of the present application, it should be noted that the present application is not limited to the following specific embodiments, any equivalent transformation made on the basis of the technical scheme of the present application falls within the protection scope of the present application.

[0053] Embodiment:

[0054] The embodiment provides a CenterNet-based DM code key point detection model construction method, as shown in Figure 1 The centernet network is used as the basis and further improved, so that the network outputs four key point coordinates of the DM code; the method comprises the following steps:

[0055] Step 1, obtaining an image set: multi-scale and multi-angle shooting is performed on a DM code scene to obtain an image set; specifically, in step 1, a common camera and an industrial camera are used to perform multi-scale and multi-angle shooting on DM code images in a non-fixed and fixed scene, and the shooting samples are as shown in Figure 2 , which contains multiple scenes such as printed codes (such as Figure 2 The upper two images in the figure), engraved codes (such as Figure 2 The lower two images in the figure), etc.

[0056] Step 2, labeling the image set: DM code labeling is performed on the image set, and the labeling result is type information and position information of four corner points of the DM code, wherein the four corner points are key points of the DM code; specifically, in the embodiment, the image set is divided into a training set and a test set in proportion, the proportion is 9.5:1, the number of images in the training set is greater than 2500, and it is not appropriate to be too small, the training set images are labeled by using software labelme, and the labeling information is type information and position information of four corner points of the DM code in the image;

[0057] Step 3, training the detection model: taking the image set as input and performing data enhancement on the image set to obtain DM code type information, center point offset and key point relative to the center point offset, and then decoding to obtain key point coordinate output, and training a DM code key point detection model based on the CenterNet network; specifically, the detection model is based on the centernet network, and the detection head output is improved for DM code key point detection;

[0058] The multi-task detection head output of the centernet network comprises:

[0059] 128x128xC, C is the number of type information;

[0060] 128x128x2, 2 is the horizontal and vertical coordinates of the center point offset of the DM code;

[0061] and 128x128x(4x3), 4 is the 4 corners of the DM code, 3 is the distance between the key point and the center point, the sin value and the cos value of the included angle between the line connecting the key point and the center point and the horizontal direction;

[0062] The data augmentation of the image set includes random contrast transformation, horizontal flip, rotation and perspective projection change of the image set;

[0063] The loss function of the above model training includes a basic loss function and an enhanced loss function; the basic loss function includes a DM code classification loss function, a DM code center point offset coordinate regression loss function and a DM code key point offset coordinate regression loss function; the enhanced loss function includes a DM code key point heat map loss function, a standard DM code projection loss function and a DM code polygon IoU loss function;

[0064] The multi-task detection head output of the centernet network is obtained through the following steps:

[0065] ResNet50 is selected as the feature extraction network, and the 3 types of feature maps output by the network are 64x64x512, 32x32x1024 and 16x16x2048 respectively; multi-scale feature fusion is performed to obtain a fused high-resolution feature map of 128x128x64; 1x1 convolution, standardization and ReLU activation function are used on the fused high-resolution feature map to obtain 3 outputs of the multi-task detection head: 128x128xC, C is the number of type information, this algorithm only studies the positioning of DM code, so only DM code is considered, C=1; 128x128x2, 2 is the horizontal and vertical coordinates of the center point offset of the DM code; 128x128x(4x3), 4 is the 4 corners of the DM code, 3 is the distance between the key point and the center point, the sin value and the cos value of the included angle between the line connecting the key point and the center point and the horizontal direction; the third type of 12 outputs introduces a polar coordinate method, which is more conducive to the correlation between coordinate information at any angle than the Cartesian coordinate method, and is conducive to obtaining more accurate results.

[0066] The DM code classification loss function, the DM code occupies a small area in the image, and has the typical positive and negative sample imbalance characteristics; in order to solve the problem of positive and negative sample imbalance, the improved Focal Loss is used for DM code classification loss, and the formula is as follows:

[0067]

[0068] wherein, L cFor DM code classification loss, c is the class, N is the number of positive samples, C is the total number of classes, k is the class index, i and j are the horizontal and vertical coordinates under the feature map 128*128, and a and b are hyperparameters for adjusting the loss weights of positive and negative samples, which are set to 2 and 4 in this paper; for each real labeled DM code, a set of response values is generated in the heat map M c represents the predicted response value.

[0069] DM code center point offset coordinate regression loss function, L1 regression loss is adopted, and the specific formula is as follows:

[0070]

[0071] L center = 1 N å i å j å k å c å p å obj å ct å ct' å ct" å ct"' å ct"" å ct"'" å ct""" co L center = 1 N å i å j å k å c å p å obj å ct å ct' å ct" å ct"' å ct"" å ct"'" å ct""" represents whether there is a DM code target at i, j of the output feature map, obj represents the target object, p ct represents the real image coordinates of the DM code center point in the original image, ct represents the center, represents the real image coordinates of the DM code center point in the feature map M co represents the coordinate value at i, j in the feature map M co , and S is a down-sampling factor, which is 4 in this paper.

[0072] DM code key point offset coordinate regression loss function, L1 regression loss is adopted, and the specific formula is as follows:

[0073]

[0074] L keypoint = 1 N å i å j å k å c å p å obj å ct å ct' å ct" å ct"' å ct"" å ct"'" å ct""" ko L keypoint = 1 N å i å j å k å c å p å obj å ct å ct' å ct" å ct"' å ct"" å ct"'" å ct""" represents whether there is a DM code target at i, j of the output feature map, obj represents the target object, p kp represents the real image coordinates of the key point in the original image, represents the real image coordinates of the DM code key point in the feature map M ko represents the coordinate value at i, j in the feature map M ko , and S is a down-sampling factor, which is 4 in this paper.

[0075] ​​​DM code keypoint heat map loss function, a set of key points in the figure can correspond to a set of response heat maps, when the key point position is predicted accurately, the corresponding heat map response value will also be located at the accurate position, therefore, this paper uses heat map to further constrain the key point position; as Figure 3 shown, the DM code key point offset decoded by the network output is recovered to the DM code key point coordinates, and a set of response values is generated at the key point position using the Gaussian kernel function and a set of constraints are formed with the DM code generated by the real label, and the improved Focal Loss is also used, and the specific formula is as follows:

[0076]

[0077] where L kp is the DM code keypoint heat map loss, kp is the keypoint, N is the number of positive samples, C is the total number of categories, k is the category index, i and j are the horizontal and vertical coordinates of the feature map M kp , and a and b are hyperparameters for adjusting the loss weights of positive and negative samples, respectively, which are set to 2 and 4 in this paper; for each real labeled DM code keypoint, a set of response values is generated in the heat feature map M kp represents the predicted response value.

[0078] Standard DM code projection loss function, since the number of grids in the width and height directions of the DM code is the same, the DM code presents as a square in the physical space; each DM code in the image can be perspective transformed to obtain the corresponding standard square DM code image, therefore, this paper uses the perspective transformation projection to further constrain the DM code key point prediction. The real perspective transformation matrix is generated by the real label, and the perspective transformation is denoted as standard DM code perspective transformation, and the obtained center point and key point are standard DM code center point and key point; as Figure 4 shown, the DM code center point and key point offset decoded by the network output are recovered to the predicted DM code center point and key point coordinates, and the predicted standard DM code center point and key point are obtained through the corresponding standard DM code perspective transformation, and a set of constraints are formed with the real standard DM code center point and key point, and the L1 regression loss is used, and the specific formula is as follows:

[0079]

[0080] where L proj represents the standard DM code projection loss, proj represents the projection, N represents the number of positive samples, represents whether there is a DM code target at i, j of the output feature map, and ​predicated and ground truth value of standard DM code center point at output feature map i, j, ct represents center point center, kp represents keypoint, pstd represents predicated standard, std represents standard, and predicated and ground truth value of standard DM code keypoint at output feature map i, j, dist(·) is a distance function of coordinates, λ stdct and λ stdkp are weight values, which are set to 0.5 and 0.25 in this paper, stdct represents standard center, stdkp represents standard keypoint.

[0081] DM code polygon IoU loss function, the IoU loss can reflect the correlation between keypoint coordinates and the matching degree of the polygon surrounded by keypoint, which is conducive to faster convergence of the network, since the DM code is an irregular quadrilateral, this paper uses the polygon IoU loss to further constrain the DM code keypoint position; as shown in Figure 5 the IoU of the polygon composed of the DM code keypoint coordinates recovered from the DM code keypoint offset decoded by the network output and the polygon composed of the real DM code keypoint constitutes a set of constraints, using L1 regression loss, the specific formula is as follows:

[0082]

[0083] wherein, L iou represents DM code polygon IoU loss, iou represents intersection over union, N is the number of positive samples, represents whether there is a DM code target at i, j of the output feature map, and respectively represent the polygon surrounded by the DM code keypoint predicated value and the ground truth value at M ko feature map i, j, poly represents polygon, IoU represents the operation strategy of intersection over union loss, this paper adopts polygon IoU loss.

[0084] Through the basic loss function and the strengthening loss function, the multi-task loss function is defined as follows:

[0085] L = λ c L c + λ co L co + λ ko L ko + λ kp L kp + λ projL proj +λ iou L iou (7)

[0086] wherein, lambda is the weight of balancing each part loss, lambda c =1, L c represents DM code classification loss, c represents class, lambda co =1, L co represents DM code center point offset coordinate regression loss, co represents center offset, lambda ko =0.1, L ko represents DM code key point offset coordinate regression loss, ko represents key point offset, lambda kp =0.1, L kp represents DM code key point heat map loss, kp represents key point, lambda proj =0.1, L proj represents standard DM code projection loss, proj represents projection, lambda iou =1, L iou represents DM code polygon IoU loss, iou represents intersection over union.

[0087] Before training, first data augmentation is carried out, that is, random contrast transformation, horizontal flip, rotation and perspective projection change are carried out on the image set; since the actual shooting scene is complex and changeable, the limited scene in the data set cannot simulate all the collection environments, in order to expand the sample number of the data set and prevent network overfitting, data augmentation is carried out in the training process, as shown in Figure 6 , including random contrast transformation, horizontal flip and rotation; at the same time, according to the camera imaging principle, the change of camera space position, angle and internal parameter corresponds to the change of projection matrix, therefore, the perspective transformation of the image is adopted in the paper to simulate the projection change, which is used to simulate the variable image collection angle; in order to prevent image deformation, the original image is scaled by equal proportion, and the redundant part is filled with gray pixels.

[0088] The application also provides a DM code key point detection method based on CenterNet, which can directly obtain the position information of four key points of the DM code in the image, so as to accurately position the DM code; the method comprises the following steps:

[0089] A DM code key point detection model based on CenterNet network is loaded, the image to be detected is sized to 512*512 and input into the network, the type information of the DM code, the center point offset information of the DM code and the key point offset information of the DM code are obtained;

[0090] The sample with correct classification result is decoded to take out the center point prediction coordinate M co and the key point prediction coordinate M ko ; and the coordinate information is converted to the original picture for prediction box drawing to obtain accurate positioning of the DM code, and the detection result is as shown in Figure 7 .

Claims

1. A method for constructing a DM code keypoint detection model based on CenterNet, characterized in that, Includes the following steps: Step 1, Obtain the image set: Take pictures of the DM code scene at multiple scales and angles to obtain the image set; Step 2, Annotate the image set: Annotate the image set with DM codes. The annotation results are the type information of the DM code and the position information of the four corner points, where the four corner points are the key points of the DM code; Step 3, train the detection model: take the image set as input and perform data augmentation on the image set to obtain DM code type information, center point offset and key point offset relative to the center point. After decoding, the key point coordinates are output, and a DM code key point detection model based on the CenterNet network is trained. The multi-task detection header output of the CenterNet network includes: C represents the number of type information items; 2 represents the x and y coordinates of the center point offset of the DM code; as well as , 4 are the four corner points of the DM code, 3 are the distance between the key point and the center point, and the sin and cos values ​​of the angle between the line connecting the key point and the center point and the horizontal direction; Data augmentation of an image set includes random contrast transformation, horizontal flipping, rotation, and perspective projection changes. The loss functions used in training the above models include a basic loss function and a reinforcement loss function. The basic loss functions include the DM code classification loss function, the DM code center point offset coordinate regression loss function, and the DM code key point offset coordinate regression loss function. The reinforcement loss functions include the DM code key point heatmap loss function, the standard DM code projection loss function, and the DM code polygon IoU loss function. The DM code classification loss function is as follows: （1） in, For DM code classification loss, For category, The number of positive samples. The total number of categories, For category indexing, The x and y coordinates are the feature map dimensions under a 128*128 pixel size. and To adjust the loss weights for positive and negative samples, a set of response values ​​is generated in the heatmap for each ground truth labeled DM code using a Gaussian kernel function. , Indicates the predicted response value; The regression loss function for the center point offset coordinates of the DM code is as follows: （2） in, The regression loss is the offset coordinate of the DM code center point. The center offset is the center point. The number of positive samples. This indicates that in the output feature map Is there a DM code target at this location? Represents the target object. This represents the actual image coordinates of the center point of the DM code in the original image. Indicates center, In the feature map The actual image coordinates of the center point of the DM code; Representation of feature map middle coordinates at that location The downsampling factor; The regression loss function for the keypoint offset coordinates of the DM code is as follows: （3） in, For DM code keypoint offset coordinate regression loss, This refers to the keypoint offset. Indicates the number of positive samples. This indicates that in the output feature map Is there a DM code target at this location? Represents the target object. This represents the true image coordinates of key points in the original image. In the feature map The actual image coordinates of key points in the DM code; Representation of feature map middle coordinates at that location This is the downsampling factor.

2. The method for constructing a DM code keypoint detection model based on CenterNet as described in claim 1, characterized in that, The multi-task detection header output of the CenterNet network is obtained through the following steps: ResNet50 was chosen as the feature extraction network, and the three types of feature maps output by the network are as follows: , and Perform multi-scale feature fusion to obtain Fusion of high-resolution feature maps; Use for fusing high-resolution feature maps Convolution, normalization, and ReLU activation functions yield the three outputs of the multi-task detection head: C represents the number of type information. 2 represents the x and y coordinates of the center point offset of the DM code; , 4 represents the four corner points of the DM code, and 3 represents the sin and cos values ​​of the distance between the key point and the center point, and the angle between the line connecting the key point and the center point and the horizontal direction.

3. The method for constructing a DM code keypoint detection model based on CenterNet as described in claim 2, characterized in that, The loss function for the DM code key point heatmap is as follows: （4） in, For the loss of DM code key point heatmap, For keypoints The number of positive samples. The total number of categories, For category indexing, For feature map The horizontal and vertical coordinates below, and For each ground truth labeled DM code keypoint, a hyperparameter used to adjust the loss weights for positive and negative samples is defined in the heatmap. A set of response values ​​is generated using the Gaussian kernel function. , This indicates the predicted response value.

4. The method for constructing a DM code keypoint detection model based on CenterNet as described in claim 3, characterized in that, The standard DM code projection loss function is given by the following formula: （5） in, This indicates the projection loss of the standard DM code. Indicates projection. Indicates the number of positive samples. This indicates that in the output feature map Is there a DM code target at this location? and This indicates that in the output feature map The predicted and actual values ​​of the center point of the standard DM code at that location. This represents the center point. This represents a keypoint. This indicates the predicted standard. This represents the true standard. and This indicates that in the output feature map Predicted and actual values ​​of key points in the standard DM code. Find the distance function for the coordinates. and These are the weight values, Indicates standard center, This refers to the standard keypoint.

5. The method for constructing a DM code keypoint detection model based on CenterNet as described in claim 4, characterized in that, The DM code polygon IoU loss function is formulated as follows: （6） in, This represents the polygon IoU loss of the DM code. This indicates the intersection over union ratio. The number of positive samples. This indicates that in the output feature map Is there a DM code target at this location? and They represent in Feature map The polygon formed by the predicted and actual values ​​of key points in the DM code. Represents a polygon. This represents the computational strategy for intersection-union ratio loss.

6. The method for constructing a DM code keypoint detection model based on CenterNet as described in claim 5, characterized in that, Based on the basic loss function and the enhanced loss function, the multi-task loss function is defined as follows: （7） in, To balance the weights of the losses in each part, take , This represents the DM code classification loss, where c represents the class. , This represents the regression loss due to the offset coordinates of the DM code center point. This indicates the center offset. , This represents the regression loss for the offset coordinates of key points in the DM code. This represents the keypoint offset. , This indicates the loss in the DM code key point heatmap. This represents a keypoint. , This indicates the projection loss of the standard DM code. Indicates projection. , This represents the polygon IoU loss of the DM code. This indicates the intersection over union ratio.

7. A method for detecting key points in DM codes based on CenterNet, characterized in that, Includes the following steps: Load the DM code key point detection model based on the CenterNet network as described in claim 1, and resize the image to be detected to... The data is then input into the network to obtain the DM code type information, DM code center point offset information, and DM code key point offset information. For samples with correct classification results, decode them to extract the predicted coordinates of the center point and key points; then convert the coordinate information onto the original image to draw the prediction box and obtain the precise location of the DM code.

Images