Express code identification method in complex environment based on improved RT-DETR

Abstract

The invention belongs to the technical field of express code detection, and particularly relates to an improved RT-DETR-based express code identification method in a complex environment, A0 and A1 are spliced along a channel dimension by introducing a CGFM module, and information of two feature maps can be combined; according to the fusion mode, the model can consider features from different sources or different levels at the same time, an attention mechanism is combined, and the ability to understand complex scenes and the ability to recognize small targets are enhanced; the false detection condition of the baseline network can be effectively avoided; the condition of incomplete target identification generated by bar code detection is avoided, meanwhile, the identification model can be combined with a neural network module of context information fusion and an attention mechanism, the ability of understanding a feature map is enhanced, and the position and the size of a prediction frame are effectively optimized; the detection precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of express code detection, and in particular relates to an express code recognition method in a complex environment based on improved RT-DETR. Background Art

[0002] With the rapid development of the domestic e-commerce economy, the express delivery industry, which supports it, has maintained rapid growth, and automation and intelligence have gradually become the focus of the express logistics industry. In terms of traditional express code detection, companies still use traditional image processing methods to detect express codes: for example, the barcode is detected and located by using the L edge, and the "L" shape is obtained by detecting the straight edge through Hough or Radon transform to rotate and correct the barcode, and then the barcode on the image is analyzed and screened. However, for express barcodes in complex environments, different detection methods need to be designed to deal with different image features using traditional image processing methods, which also leads to poor versatility of the detection methods. In addition, during the express barcode detection process, the barcode reading time consumes the longest time.

[0003] With the rapid development of deep learning, the target detection algorithm model has made a qualitative leap. The main feature of this method is that it can automatically learn the features of the image without the need for manual feature design. It has a stronger detection capability for multi-scale and multi-target barcodes on express delivery bills, which significantly improves the detection accuracy and efficiency. However, in the actual use process, there are still some shortcomings such as large number of parameters and low recognition accuracy. Therefore, it is necessary to further explore and study more efficient and more accurate express barcode recognition methods to deal with the problem of express code detection in complex environments. Summary of the invention

[0004] The purpose of the present invention is to provide a method for recognizing express codes in a complex environment based on improved RT-DETR. The recognition model in the recognition method introduces the CGFM module, splices A0 and A1 along the channel dimension, combines the information of the two feature maps, and quickly recognizes the express delivery barcode through visual comparison.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: A method for identifying express codes in a complex environment based on improved RT-DETR, comprising the following steps: Step 1: Collect images of express delivery bills with express codes in complex environments to construct an original express delivery bill image dataset. After annotating the images, divide them into a training set and a validation set in a ratio of 5:1. Step 2, build the FCG-RT-DETR model and replace the backbone in the RT-DETR network with the backbone of FasterNet; Step 3, integrating the content-guided fusion module into the encoder module; Step 4: Introduce a loss function that integrates the Inner idea and replace the original GIoU loss function with a more efficient Inner-GIoU loss function; Step 5, use the training set divided in step 1 to train the FCG-RTDETR model to obtain the weight file; Step 6: Input the validation set divided in step 1 and the weight file in step 3 into the FCG-RT-DETR detection model for verification.

[0006] Furthermore, the step 1 includes the following operations: By shooting barcodes of various express delivery labels in different scenarios, we constructed an original data set of express delivery codes, which contains 2934 images in total, including 2408 training sets and 526 validation sets. The barcodes in the express delivery label images were marked using marking software, and the marked images were divided into training and validation sets in a 5:1 ratio.

[0007] Furthermore, the step 2 includes the following operations: By replacing the RT-DETR backbone with the FasterNet backbone feature extraction network, the 640×640 express delivery label image is used as input data and passes through Embedding, FasterNetBlock, Merging, FasterNet Block×2, Merging, FasterNet Block×8, Merging, FasterNet Block×2 in sequence; the FasterNet Block is composed of partial convolution, point convolution, BN and Gaussian error linear unit activation function; Embedding and Merging are composed of downsampling convolution and BN; the first FasterNet Block×2, FasterNet Block×8 and the last FasterNet Block×2 in the Backbone are output to the efficient hybrid encoder.

[0008] Furthermore, the step 2 includes the following operations: Input features A0 and A1 enter the content-guided feature fusion module (CGFM) module, and the features of the two input layers A0 and A1 are fused (concatenated) to obtain A con , where: A0 and A1 have C channels, A con The number of channels is 2C; For input Aon After entering the SE attention mechanism, it is divided into two identical feature subsets B0 and B1, both of which have C channels; For input feature B0, multiply A0 with it (element-wise mul) to get feature C0, and for input feature B1, multiply A1 with it (element-wise mul) to get feature C1; For input feature C0, add A1 to it (element-wise add) to get feature D0, and for input feature C1, multiply X0 to it (element-wise add) to get feature D1; The input features D0 and D1 are concatenated to obtain the final output.

[0009] Furthermore, step 4 includes the following operations: Replace the original GIoU loss function with the Inner-GIoU loss function; its calculation formula is: L Inner-GIoU =L GIoU +IoU-IoU inner Where: L GIoU is the original GIou loss in the baseline network, IoU is the intersection over union loss, IoU inner is the Inner-IoU loss; inner=(w gt *h gt )*(γ) 2 +(w*h)*(γ) 2 -inter (6) in, is the center point of the GIoU real box and the Inner-IoU real box, (x c ,y c ) represents the center point of the GIoU prediction box; W gt and h gt They represent the width and height of the real box, w and h represent the width and height of the predicted box, and γ represents the scale factor.

[0010] The beneficial effects of the present invention are as follows: the recognition method of express codes in a complex environment based on the improved RT-DETR provided by the present invention can combine the information of the two feature maps by introducing the CGFM module and splicing A0 and A1 along the channel dimension; this fusion method enables the model to simultaneously consider features from different sources or different levels, and also combines the attention mechanism to enhance the ability to understand complex scenes and recognize small targets; it can effectively avoid false detection of the baseline network; it can avoid the occurrence of incomplete target recognition of barcodes, and at the same time, the recognition model of the present invention can combine the neural network module of context information fusion and attention mechanism to enhance the ability to understand feature maps; the Inner-GIoU loss function that incorporates the Inner idea effectively optimizes the prediction box position and size by introducing auxiliary bounding boxes and strategies for distinguishing different IoU samples; it enhances the target detection capability and improves the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is the overall framework diagram of the recognition method of the express code in a complex environment based on the improved RT-DETR; Figure 2 It is the overall structure diagram of the FCG-RT-DETR model in the present invention; Figure 3 It is the structural diagram of the CGFM module in the present invention; Figure 4 3. A comparison chart of the detection results of the model of the present invention. DETAILED DESCRIPTION

[0012] Specific embodiment 1: The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that: in the present invention, if there is no special explanation, all the implementation methods and preferred implementation methods mentioned in this article can be combined with each other to form a new technical solution. In the present invention, if there is no special explanation, all the technical features and preferred features mentioned in this article can be combined with each other to form a new technical solution. Unless otherwise specified, the professional and scientific terms used in this article have the same meaning as those familiar to those skilled in the art.

[0013] Aiming at the situation that the barcode detection and recognition accuracy of express delivery bills is low and the efficiency is low in complex environments, the present invention designs and provides a method for recognizing express delivery codes in complex environments based on improved RT-DETR, as shown in the attached manual. Figure 1As shown in the figure, the method first constructs the original express delivery image dataset, which is divided into a training set and a validation set after manually annotating the barcodes; then the RT-DETR model is improved to obtain the FCG-RT-DETR model; as shown in the attached manual Figure 2 As shown in the figure, the improvement methods include using FasterNet as the backbone network to realize the lightweight of the model; introducing the context guide fusion module to strengthen the feature fusion ability of the model; integrating the model loss function with the inner idea to improve the detection accuracy of the model; then inputting the divided training set into the constructed FCG-RT-DETR model for training to obtain the weight file of the detection model, and inputting the verification set and the weight file into the detection model to complete the target detection of the barcode on the express delivery bill in a complex environment; in the specific operation, by shooting the barcodes of various express delivery bills in different scenes, the original data set of the express code is constructed, of which there are 2934 images in total, including 2408 training sets and 526 verification sets; The barcodes in the express delivery bill images were marked using marking software, and the marked images were divided into training sets and validation sets at a ratio of 5:1. The RT-DETR backbone was replaced with the FasterNet backbone feature extraction network, and the 640×640 express delivery bill images were used as input data and sequentially passed through Embedding, FasterNetBlock, Merging, FasterNet Block×2, Merging, FasterNet Block×8, Merging, and FasterNet Block×2. The FasterNet Block consisted of partial convolution, point convolution, BN, and Gaussian error linear unit activation functions. Embedding and Merging consisted of downsampling convolution and BN. The first FasterNet Block×2, FasterNet Block×8, and the last FasterNet Block×2 in the Backbone were output to the efficient hybrid encoder. Input features A0 and A1 enter the content-guided feature fusion module (CGFM) module, and the features of the two input layers A0 and A1 are fused (concatenated) to obtain A con , where: A0 and A1 have C channels, A con The number of channels is 2C; For input A on After entering the SE attention mechanism, it is divided into two identical feature subsets B0 and B1, both of which have C channels; For input feature B0, multiply A0 with it (element-wise mul) to get feature C0, and for input feature B1, multiply A1 with it (element-wise mul) to get feature C1; For input feature C0, add A1 to it (element-wise add) to get feature D0, and for input feature C1, multiply X0 to it (element-wise add) to get feature D1; The input features D0 and D1 are concatenated to obtain the final output. Replace the original GIoU loss function with the Inner-GIoU loss function; its calculation formula is: L Inner-GIoU =L GIoU +IoU-IoU inner Where: L GIoU is the original GIou loss in the baseline network, IoU is the intersection over union loss, IoU inner is the Inner-IoU loss. inner=(w gt *h gt )*(γ) 2 +(w*h)*(γ) 2 -inter (13) in, is the center point of the GIoU real box and the Inner-IoU real box, (x c ,y c ) represents the center point of the GIoU prediction box. gt and h gt They represent the width and height of the real box, w and h represent the width and height of the predicted box, and γ represents the scale factor.

[0014] The present invention provides a method for recognizing courier codes in a complex environment based on improved RT-DETR. Through visual comparison, it is possible to more intuitively see the recognition effect of the courier barcode before and after the model is improved, as shown in the attached manual. Figure 3As shown; in which, the baseline network in rows (1) and (4) produces false detection of the barcode. The recognition model of the present invention introduces the CGFM module to splice A0 and A1 along the channel dimension, which can combine the information of the two feature maps; this fusion method helps the model to simultaneously consider features from different sources or different levels; at the same time, it also combines the attention mechanism to enhance the ability to understand complex scenes and recognize small targets; and thus can effectively avoid the false detection of the baseline network in row (1); when the baseline network in rows (2) and (3) produces incomplete target recognition of the barcode, the recognition model of the present invention enhances the ability to understand the feature map by introducing the CGFM combined with the neural network module of the context information fusion and the attention mechanism; at the same time, the Inner-GIoU loss function of the Inner idea is incorporated, and by introducing the auxiliary bounding box and the strategy of distinguishing different IoU samples, it can effectively optimize the prediction box position and size; enhance the target detection capability and improve the detection accuracy; and effectively avoid the occurrence of incomplete recognition of the predicted target box; Compared with other deep learning model detection methods, the FCG-RT-DETR detection model has a higher recognition accuracy and a lighter detection model, as shown in Table 1 below; Among them, mAP(0.5) represents the average accuracy of target detection when the intersection-over-union ratio is 0.5, Racall represents the recall rate, and precision represents the accuracy rate. Parameters(M) represents the number of network model parameters, FLOPs(G) represents the amount of network model calculations; ModelSize(MB) represents the size of model memory usage. The higher the values ​​of mAP(0.5), Racall, and precision, the higher the accuracy of target detection; the lower the values ​​of ModelSize(MB), Parameters(M), and FLOPs(G), the lighter the network model of target detection; through comparison in the table, the indicators of the present invention are all in an advantageous position, indicating that the present invention has obvious advantages in express delivery bill barcode recognition accuracy and lightweight in complex environments, and is more conducive to application to detection equipment with low computing power requirements while ensuring accuracy, thereby reducing costs. The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for identifying express codes in complex environments based on improved RT-DETR, characterized in that: The following steps are involved: Step 1: Collect images of express delivery bills with express codes in complex environments to construct an original express delivery bill image dataset. After annotating the images, divide them into a training set and a validation set in a ratio of 5:

1. Step 2, build the FCG-RT-DETR model and replace the backbone in the RT-DETR network with the backbone of FasterNet; Step 3, integrating the content-guided fusion module into the encoder module; Step 4: Introduce a loss function that integrates the Inner idea and replace the original GIoU loss function with a more efficient Inner-GIoU loss function; Step 5, use the training set divided in step 1 to train the FCG-RTDETR model to obtain the weight file; Step 6: Input the validation set divided in step 1 and the weight file in step 3 into the FCG-RT-DETR detection model for verification.

2. According to claim 1, a method for identifying express codes in a complex environment based on improved RT-DETR, characterized in that: The step 1 includes the following operations: By shooting barcodes of various express delivery labels in different scenarios, we constructed an original data set of express delivery codes, which contains 2934 images in total, including 2408 training sets and 526 validation sets. The barcodes in the express delivery label images were marked using marking software, and the marked images were divided into training and validation sets in a 5:1 ratio.

3. According to claim 2, a method for identifying express codes in a complex environment based on improved RT-DETR, characterized in that: The step 2 includes the following operations: By replacing the RT-DETR backbone with the FasterNet backbone feature extraction network, the 640×640 express delivery label image is used as input data and passes through Embedding, FasterNetBlock, Merging, FasterNetBlock×2, Merging, FasterNet Block×8, Merging, FasterNet Block×2 in sequence; among which FasterNetBlock is composed of partial convolution, point convolution, BN and Gaussian error linear unit activation function; Embedding and Merging are composed of downsampling convolution and BN; among which the first FasterNet Block×2, FasterNet Block×8 and the last FasterNet Block×2 in the Backbone are output to the efficient hybrid encoder.

4. According to claim 3, a method for identifying express codes in a complex environment based on improved RT-DETR is characterized in that: The step 2 includes the following operations: Input features A0 and A1 enter the content-guided feature fusion module (CGFM) module, and the features of the two input layers A0 and A1 are fused (concatenated) to obtain A con , where: A0 and A1 have C channels, A con The number of channels is 2C; For input A on After entering the SE attention mechanism, it is divided into two identical feature subsets B0 and B1, both of which have C channels; For input feature B0, multiply A0 with it (element-wise mul) to get feature C0, and for input feature B1, multiply A1 with it (element-wise mul) to get feature C1; For input feature C0, add A1 to it (element-wise add) to get feature D0, and for input feature C1, multiply X0 to it (element-wise add) to get feature D1; The input features D0 and D1 are concatenated to obtain the final output.

5. According to claim 1, a method for identifying express codes in a complex environment based on improved RT-DETR, characterized in that: The step 4 includes the following operations: Replace the original GIoU loss function with the Inner-GIoU loss function; its calculation formula is: L Inner-GIoU =L GIoU +IoU-IoU inner Where: L GIoU is the original GIou loss in the baseline network, IoU is the intersection over union loss, and IoU inner is the Inner-IoU loss; inner=(w gt *h gt )*(c) 2 +(w*h)*(c) 2 -inter (6) in, is the center point of the GIoU real box and the Inner-IoU real box, (x c ,y c ) represents the center point of the GIoU prediction box; W gt and h gt They represent the width and height of the real box, w and h represent the width and height of the predicted box, and γ represents the scale factor.

Images