Box type detection and bar code recognition method based on improved YOLOv8 model

Abstract

The invention discloses a box type detection and bar code recognition method based on an improved YOLOv8 model. The method comprises the following steps: 1) constructing an improved YOLOv8 model, carrying out box type detection and bar code recognition, and introducing target detection of a CA attention mechanism to the YOLOv8 model; target detection of WIoU is introduced into the YOLOv8 model; target detection of a GSConv + Slim Neck framework is introduced into the YOLOv8 model; step 2), based on Omniver digital twinning data generation, experiment data collection is carried out, a self-built data set is constructed, and the self-built data set comprises three types of cartons, material boxes and bar codes; and step 3), training the improved model on the basis of the self-built data set, and carrying out box type detection and bar code recognition. According to the invention, the original identification equipment can be replaced, the cost is saved, and the error rate of the box type detection equipment can be greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual recognition, and in particular relates to a box type detection and barcode recognition method based on an improved YOLOv8 model. Background Art

[0002] The operations in the logistics center are roughly divided into warehousing, outbound and replenishment operations, which mainly involve two types of boxes: material boxes and cartons, and also involve multiple code scanning and identification links. Code scanning involves the review of handheld code guns, side scanning of material boxes and top scanning of cartons, but there are currently problems such as insufficient accuracy in box classification, low review efficiency and high cost of code scanning and identification.

[0003] During the operation of the logistics center, it was found that its conveying system was affected by a large number of box detection and barcode recognition devices, and had problems such as low detection accuracy and high equipment purchase cost. It was urgently necessary to make overall improvements to its conveying system to improve detection accuracy and reduce the purchase cost of identification equipment.

[0004] In the existing technology, the box type detection and barcode recognition equipment in the logistics center is widely distributed, which dominates the normal operation of the conveying system. However, due to the limitation of equipment accuracy, the error rate of box type recognition using photoelectric sensors can only reach 3‰, which cannot meet the 1‰ error rate required by actual operations.

[0005] The traditional photoelectric recognition of box types has a high error rate and cannot meet the actual operational needs; secondly, although the method of identifying barcodes through visual technology is efficient, the cost of purchasing equipment is also extremely expensive; finally, box type recognition and barcode recognition are highly correlated in the actual operation process, but the recognition equipment for the two is not effectively integrated, and further control is still needed to achieve coordinated operation of the two in the actual operation process. Summary of the invention

[0006] In order to overcome the above problems existing in the prior art, the purpose of the present invention is to provide a box type detection and barcode recognition method based on an improved YOLOv8 model. By adopting advanced visual recognition solutions, it can not only replace the original recognition equipment and save costs, but also greatly reduce the error rate of box type detection equipment to meet the operational needs of enterprises, and can also achieve a high degree of integration of box type detection and barcode recognition processes, thereby further improving operational efficiency.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A box type detection and barcode recognition method based on an improved YOLOv8 model comprises the following steps;

[0009] Step 1): Build an improved YOLOv8 model to perform box detection and barcode recognition, and introduce the target detection of CA attention mechanism into the YOLOv8 model to improve the model's perception and processing capabilities of key information;

[0010] Introducing WIoU object detection into the YOLOv8 model to improve the generalization performance of the model;

[0011] Introducing GSConv+Slim Neck architecture for object detection into the YOLOv8 model;

[0012] Step 2): Based on the data generation of the Omniverse digital twin, conduct experiments to collect data and build a self-built data set. The self-built data set contains three categories: cartons, material boxes, and barcodes. Data exploration is carried out with significant feasibility and advantages.

[0013] Step 3): Train the improved model based on the self-built data set to perform box type detection and barcode recognition.

[0014] Step 1) improved the three modules of the model, step 2 was to collect data for the experiment, and step 3) conducted experiments on box detection and barcode recognition. The accuracy and speed of the improved model were improved.

[0015] In the step 1), the CA attention mechanism, WioU loss function and GSConv+Slim Neck architecture are added to the YOLOv8 model structure for improvement; the CA attention mechanism improves the accuracy and efficiency of target detection, and by introducing the WIoU loss function as the bounding box regression loss function, the detection accuracy and generalization performance of the model are improved, and the GSConv+Slim Neck architecture is introduced to reduce the model complexity and reasoning time to improve the reasoning efficiency.

[0016] In the backbone network of the original YOLOv8 model, the CA attention mechanism is added between the C2f and SPPF modules to improve detection accuracy, the GSConv+Slim Neck architecture is added between the C2f and CBS modules to improve reasoning efficiency, and the WioU loss function is added in the back-propagation process to improve the detection accuracy and generalization of the model.

[0017] The YOLOv8 original model structure is:

[0018] The basic structure of YOLOv8 is divided into the input layer, backbone network, feature enhancement module and detection head;

[0019] (1) Input

[0020] The input layer of YOLOv8 is responsible for receiving the raw box image and converting it into a tensor format that the deep learning model can understand. Usually, the size of the input layer matches the image size of the training data to ensure that the model can effectively process the input data.

[0021] (2) Backbone

[0022] The backbone network is responsible for extracting features from the input image;

[0023] (3) Feature Enhancement Module (Neck)

[0024] The role of the Neck layer is to pool and fuse the input feature map; it helps the model better understand the input image and improve the detection performance;

[0025] (4) Detection Head

[0026] The detection head is responsible for performing target detection on the output of the backbone network to generate bounding boxes and category probabilities of the target. In YOLOv8, the detection head may use a multi-scale prediction strategy to generate predictions in different layers of the network at the same time to capture targets of different sizes and scales.

[0027] The backbone network of the YOLOv8 model includes 5 CBS modules, 4 C2f modules and 1 SPPF module;

[0028] The role of the backbone network is to extract features from the input image; the Neck layer of the YOLOv8 model imitates the organizational structure in PANet, first upsampling the output feature map of the SPPF module (Upsample means the upsampling process), then downsampling, and fusing the corresponding feature map in the upsampling process with the feature map in the downsampling process, which can greatly enhance the network's feature extraction and feature fusion capabilities. The generated fused feature map is then sent to the Detect Head layer to perform the detection task, generating information such as the target's bounding box and category probability.

[0029] The CA attention mechanism added in step 1) is divided into three steps from input to output

[0030] Step 1: Global pooling is often used in channel attention to globally encode spatial information. The global pooling is decomposed. For a box-shaped input feature map x (x is a picture with a length of H and a width of W), each channel is first encoded along the horizontal and vertical coordinate directions using pooling kernels of size (H, 1) and (1, W);

[0031] The output expression of the cth channel with a height of h is shown in formula (1). Similarly, the output expression of the cth channel with a width of w is shown in formula (2). The input feature map with a size of C*H*W is pooled in the X direction and Y direction respectively to generate box-type feature maps with sizes of C*H*1 and C*1*W respectively.

[0032]

[0033] Step 2: After transforming the generated C*1*W box-shaped feature map, perform a cascade (Concat) operation, use a shared 1×1 convolution kernel to reduce the dimension, and reduce its dimension to the original C / r (r represents the downsampling ratio). Then, the batched box-shaped feature map F1 is sent to the Sigmoid activation function to obtain a feature map f of the shape of C / r*1*(W+H), as shown in formula (3);

[0034] f=δ(F1([z h ,z w ])) (3)

[0035] Step 3: Split the box-shaped feature map f into two separate tensors f along the spatial dimension h ∈R C / r×H×1 、f w ∈R C / r×1×W , and then use 1×1 convolution kernel to perform dimension increase operation, and then combine the Sigmoid activation function to get the attention vector, g h ∈R C×H×1 , g w ∈R C×1×W ; Its mathematical expressions are shown in formula (4) and formula (5); Finally, by multiplying the original feature map and performing weighted calculation as the attention weight, the feature map with attention weights in the width and height directions will be obtained. The final output of the CA module is shown in formula (6);

[0036] g h =σ(F h (f h )) (4)

[0037] g w =σ(F w (f w )) (5)

[0038]

[0039] The WioU loss function in step 1) is an improvement on WIoU v3, and uses outliers to describe the quality of the predicted box. The expression is as follows:

[0040]

[0041] Among them, the smaller β is, the higher the quality of the prediction box is. A smaller gradient gain is assigned to it, so that the bounding box regression focuses on the prediction box of normal quality. The larger β is, the lower the quality of the prediction box is. A smaller gradient gain is assigned to it to reduce the adverse effects on low-quality examples. β is used to construct a non-monotonic focusing coefficient r and apply it to WIoUv1. The mathematical expression is as follows:

[0042]

[0043] Among them, when β = δ, r = 1, when the outlier degree of the predicted box satisfies β = C (C is a constant), the gradient gain is the highest, because It is dynamic, and the quality division criteria of the prediction box are also dynamic, which enables WIoU v3 to make the gradient gain allocation strategy that best suits the current situation at every moment.

[0044] The GSConv+Slim Neck architecture is an object detector architecture that combines the GSConv and Slim Neck structures.

[0045] The step 2) is specifically as follows:

[0046] Use actual site photos, photos collected from the Internet, and Omniversecode in the Omniverse software to simulate various environments and generate the required data photos as data sets for experiments;

[0047] In step 3), the improved YOLOv8 model is used to perform the tasks of carton, material box and barcode recognition, and accurate box and barcode recognition is achieved in subsequent experiments.

[0048] The specific identification process is:

[0049] The improved model marks the type of carton and the barcode, with the carton marked as 1, the material box marked as 2, and the barcode marked as 3. When an object is detected, it will first identify whether it is a carton or a material box. After the box type is detected and classified, the barcode on the box will be synchronously detected. First, the barcode is accurately located through the improved YOLOv8 model, and then the Zbar library responsible for barcode recognition will automatically identify the barcode information for the barcode area, thereby completing the box type detection and barcode recognition process.

[0050] Beneficial effects of the present invention:

[0051] This paper proposes an optimization scheme for multiple existing problems in the logistics center. By introducing deep learning technology, the process of the logistics center is improved from a technical level. Subsequently, an improved YOLOv8 model is proposed based on the proposed optimization scheme. The model performance of YOLOv8 is improved by introducing the coordinate attention mechanism (CA), WIoU loss function and GSConv+Slim Neck architecture. The composition of the data set used for model training is introduced, in which a digital twin-based method is used to generate some data sets using the Omniverse platform. After introducing the experimental environment and the selection of evaluation indicators, the experimental results of the model are analyzed and displayed, proving the effectiveness of the improved model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is the overall network structure of YOLOv8.

[0053] Figure 2 This is the structural diagram of the CA attention mechanism.

[0054] Figure 3 It is the feature map in the X direction.

[0055] Figure 4 It is the feature map in the Y direction.

[0056] Figure 5 It is the feature map after cascading.

[0057] Figure 6 This is the YOLOv8 network structure diagram after the introduction of CA.

[0058] Figure 7 This is the SC calculation process.

[0059] Figure 8 This is the DSC calculation process.

[0060] Fig. 9 It is the GSConv calculation process.

[0061] Fig.10 For each module structure.

[0062] Fig.11 This is the YOLOv8 network structure diagram after introducing CA and GSConv.

[0063] Fig.12 This is a partial bin data set.

[0064] Fig.13 This is a partial carton dataset.

[0065] Fig.14 This is a partial barcode dataset.

[0066] Fig.15The synthetic bin dataset.

[0067] Fig.16 Comparison chart of the improved YOLOv8 model and other models that introduce the Slim neck architecture.

[0068] Fig.17 is the convergence curve based on the Yolov8 model.

[0069] Fig.18 This is the convergence curve of the Yolov8 model after the introduction of CA+WIOU.

[0070] Fig.19 This is the convergence curve of the Yolov8 model that introduces the CA+WIOU+GSConv+Slim Neck architecture.

[0071] Fig. 20 Composite cardboard image for Omniverse.

[0072] Fig.21 This is a real cardboard picture of Omniverse.

[0073] Fig. 22 This is the Omniverse crafting box image.

[0074] Fig.23 This is a real material box picture of Omniverse.

[0075] Fig.24 A picture of a barcode is shown. DETAILED DESCRIPTION

[0076] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0077] The present invention provides a box detection and barcode recognition method based on an improved YOLOv8 model. On the basis of the YOLOv8 model, the CA attention mechanism is added, the WIoU target detection is introduced, and the GSConv+Slim Neck architecture is introduced to improve the accuracy and speed of target detection, and a data set training model is generated based on the digital twin.

[0078] The network structure of the YOLOv8 model is as follows Figure 1 As shown, the specific steps are as follows:

[0079] Step 1: Introducing the CA attention mechanism for target detection

[0080] (1) Experimental principle

[0081] In order to achieve accurate and efficient recognition of boxes and cartons, and precise positioning of barcodes. In order to improve system performance, the introduction of the attention mechanism becomes a key strategy. The attention mechanism is an important mechanism in deep learning models. Its purpose is to enable the model to focus more on specific parts of the input data to improve the model's perception and processing capabilities of key information. The proposal of the attention mechanism is inspired by the human visual system. When processing information, people tend to selectively focus on certain specific areas instead of generalizing the entire scene.

[0082] In the process of identifying boxes and cartons, the attention mechanism helps the system better understand and distinguish targets, and improves the ability to distinguish different objects. The application of the attention mechanism also enables the system to focus on the detection object more intelligently, thereby improving the positioning accuracy and detection efficiency of the barcode. Especially in the task of barcode positioning, the use of the attention mechanism enables the system to focus more on the potential barcode location, reducing errors and improving accuracy. Such precise positioning not only enables the system to capture barcode information more quickly, but also has a positive effect on the subsequent recognition process.

[0083] (2) Selection of CA Attention Mechanism

[0084] The CA attention mechanism can be divided into three categories according to the different dimensions of its action: channel attention mechanism, spatial attention mechanism, and hybrid attention mechanism. The channel CA attention mechanism focuses on operating on the channel dimension of the feature map. By learning the weight of each channel, the network can focus more on the response to specific features, thereby increasing the attention to important information. Channel attention performs well when processing feature maps with multiple channels. The spatial attention mechanism focuses on the spatial dimension of the feature map. By learning the weight of each spatial position, the network can more accurately locate and focus on important areas in the image. This attention mechanism is very useful when dealing with tasks that require attention to different areas of the image, such as object detection and image segmentation. The hybrid attention mechanism combines channel and spatial attention, fully considering the information of the feature map in channels and space. This comprehensive attention mechanism can improve the perception ability of the model at different levels, enabling it to more comprehensively capture the key information of the input data, thereby improving the performance of the model.

[0085] Among the objects studied in this paper, the research focuses on the accurate and efficient recognition of boxes and cartons and the precise positioning of barcodes. In this scenario, the spatial structure and position information of the object are crucial to whether it can be accurately recognized, and the research scenario of this paper will have long-range dependency problems. For example, when locating the barcodes on boxes and cartons, the barcodes only occupy a small area of ​​the image, and the model needs to span a long distance to capture the features of the target in order to accurately detect the target. Therefore, the CA attention mechanism is chosen to improve the YOLOv8 model.

[0086] (3) Implementation process of CA attention mechanism;

[0087] like Figure 2 As shown in the figure, it is the structure diagram of the CA module, which is divided into three steps from input to output:

[0088] Step 1: Global pooling is often used in channel attention to globally encode spatial information. In order to avoid compressing all spatial information into the channel, global pooling is not used here. In order to facilitate the attention module to capture long-range spatial dependencies with precise location information, the global pooling is decomposed. For the input x, first use pooling kernels of size (H, 1) and (1, W) to encode each channel along the horizontal coordinate direction and the vertical coordinate direction. Therefore, the output expression of the cth channel with a height of h is shown in formula (1). Similarly, the output expression of the cth channel with a width of w is shown in formula (2). The input feature map of size C*H*W is pooled in the X direction and Y direction respectively to generate feature maps of size C*H*1 and C*1*W respectively. As Figure 3 and Figure 4 shown.

[0089]

[0090] Step 2: After transforming the generated C*1*W feature map, perform a cascade (Concat) operation, use a shared 1×1 convolution kernel to reduce the dimension, reduce its dimension to the original C / r (r represents the downsampling ratio), and then send the batched feature map F1 to the Sigmoid activation function to obtain a feature map f of the form C / r*1*(W+H). The expression is shown in formula (3). The feature map after the cascade operation is as follows Figure 5 shown.

[0091] f=δ(F1([z h ,z w ])) (3)

[0092] Step 3: Along the spatial dimension, f is split into two separate tensors, and then the 1×1 convolution kernel is used to perform dimensionality increase, and then the attention vector is obtained by combining the Sigmoid activation function. Its mathematical expressions are shown in Equations (4) and (5). Finally, the original feature map is multiplied by weighted calculation as the attention weight, and the final feature map with attention weights in the width and height directions is obtained. The final output of the CA module is shown in Equation (6).

[0093] g h =σ(F h (f h )) (4)

[0094] g w =σ(F w (f w )) (5)

[0095]

[0096] The YOLOv8 model structure after introducing the CA attention mechanism is as follows Figure 6 shown.

[0097] When designing the channel attention module, we ensure that sufficient spatial position information is retained while enhancing the information interaction between channels. Although the traditional global pooling method can effectively encode global information, it often sacrifices spatial details. In order to overcome this limitation, an innovative strategy is adopted, namely decomposing global pooling.

[0098] The specific approach is to decompose the global pooling operation into two one-dimensional pooling steps along the horizontal and vertical directions, encoding each channel separately. The information of each channel can retain a certain degree of spatial position sensitivity while capturing long-range spatial dependencies.

[0099] In order to further refine this information and generate effective attention weights, the feature maps obtained by pooling in different directions are transformed and cascaded. This step not only integrates the spatial information in the horizontal and vertical directions, but also achieves feature dimensionality reduction through a shared 1×1 convolution kernel, improving the computational efficiency of the model. The processed feature map is activated using the Sigmoid activation function to obtain a feature map containing rich attention information.

[0100] This attention information is effectively applied to the original feature map, and the obtained feature map is split into two parts along the spatial dimension, and the dimensionality is increased by a 1×1 convolution kernel respectively, and the attention vector is generated again using the Sigmoid activation function. The two attention vectors represent the attention weights in the width and height directions respectively. By multiplying the two attention vectors with the original feature map, a feature map with fine attention weights in both the width and height directions is obtained. This process not only enhances the model's attention to important features, but also improves the richness and accuracy of feature expression.

[0101] Step 2: Introduce WIoU target detection

[0102] The YOLOv8 model uses CIoU as the loss function to calculate the bounding box regression loss. Its main idea is to improve the accuracy of the bounding box measurement. In fact, the IoU before this was improved based on this idea. However, if there are some or a large amount of abnormal data in the training data set itself, no matter how much the accuracy of the bounding box measurement is improved, it will still have an adverse effect on the training of the model. Therefore, Focal loss was proposed. Its goal is to maintain the detection performance of the model when processing low-quality data, and a focusing mechanism is used to adjust the weight of the loss function to better handle data of different qualities. However, the focusing mechanism of Focal loss is static, which means that its focusing mechanism remains unchanged throughout the training process. In this case, it is possible to consider introducing a dynamic focusing mechanism to adjust the weight of the loss function according to the learning situation of the model during the training process, and then WIoU was proposed.

[0103] WIoU includes 3 versions. WIoU v1 adjusts the IoU weight by adding an attention mechanism, v2 uses a monotonic focusing mechanism to improve stability, and v3 introduces a dynamic non-monotonic focusing mechanism to achieve better performance.

[0104] Since low-quality examples are inevitably included in the training data, geometric metrics (such as distance and aspect ratio) will intensify the penalty for low-quality examples, resulting in decreased generalization performance of the model. An excellent loss function should weaken the penalty of geometric metrics when the bounding boxes overlap well.

[0105] Based on this, WIoU v1 is proposed, and its mathematical expressions are shown in Equations (7) and (8).

[0106] L WIoUv1 =R W I oU L IoU (7)

[0107]

[0108] Among them, R is the penalty, L is the loss, * represents the separation operation, x, y and x gt ,y gt are the center coordinates of the predicted box and the real box, respectively, g , H g is the width and height of the real frame. WIoU ∈[1,e) will significantly enlarge the L of the normal quality prediction box IoU ∈[0,1] will significantly reduce the R of high-quality prediction boxes WIoU In order to prevent R WIoU Produce gradients that hinder convergence, W g ,H g Detach from the computation graph.

[0109] WIoU v2 imitates the focusing mechanism of Focal loss and constructs L WIoUv1 The monotonic focusing coefficient (also called gradient gain), the mathematical expression (9) is as follows:

[0110]

[0111] because Will follow L IoU The decrease of decreases, which slows down the convergence, so we introduce (L IoU The mean) is used as the normalization factor, and the mathematical expression (10) is as follows:

[0112]

[0113] Among them, the normalization factor is dynamically updated to make the gradient gain The overall level is maintained at a high level, solving the problem of slow convergence in the later stages of training.

[0114] WIoU v3 has made further improvements, using outliers to describe the quality of the predicted box. The mathematical expression is as follows:

[0115]

[0116] Among them, the smaller β is, the higher the quality of the prediction box is, and a smaller gradient gain is assigned to it, so that the bounding box regression focuses on the prediction box of normal quality. The larger β is, the lower the quality of the prediction box is, and a smaller gradient gain is assigned to it to reduce the adverse effects on low-quality examples. β is used to construct the non-monotonic focusing coefficient r and apply it to WIoUv1. The mathematical expression is as follows:

[0117]

[0118] Among them, when β = δ, r = 1. When the outlier degree of the predicted box satisfies β = C (C is a constant), the gradient gain is the highest. It is dynamic, and the quality division criteria of the prediction box are also dynamic, which enables WIoU v3 to make the gradient gain allocation strategy that best suits the current situation at every moment.

[0119] As an optimization loss function in the field of object detection, WIoU has undergone a gradual evolution from v1 to v3, aiming to improve the detection accuracy and stability of the model in different situations through more sophisticated weight adjustment and focusing mechanisms. In WIoU v1, by introducing the attention mechanism, this version can dynamically adjust the weight of IoU (intersection over union), so that the model can pay more attention to those areas that are critical to detection accuracy. This design helps to alleviate the negative impact of low-quality samples in the dataset (such as blur, occlusion, etc.) on model training. WIoU v2 adopts a monotonic focusing mechanism to ensure that as the overlap between the predicted box and the true box increases, the sensitivity of the loss function to the prediction error gradually decreases. This smooth monotonic decreasing feature helps the model maintain stability during training and reduces problems caused by excessive punishment of low-quality samples by geometric metrics (such as border distance, aspect ratio, etc.). The latest WIoU v3 version, based on the advantages of the previous two versions, innovatively introduces a dynamic non-monotonic focusing mechanism. This mechanism allows the sensitivity to prediction errors to be appropriately increased at a specific stage when the predicted box is close to the true box but not yet completely overlapped, thereby encouraging the model to make more refined adjustments. In this way, the v3 version not only maintains the stability of the model, but also further improves its ability to capture subtle differences in bounding boxes, achieving better detection performance.

[0120] Therefore, WIoU v3 is used as the bounding box regression loss function of the improved model. For the homemade dataset in this article, there may be uneven annotation quality, which leads to uneven quality of image data, especially when the barcode and the background are easily confused. The penalty for low-quality image data is adjusted and the original model is improved to improve the generalization performance of the model.

[0121] Step 3: Introduce target detection with GSConv+Slim Neck architecture

[0122] Given that the proposed improved YOLOv8 model needs to be tested on-site in a pharmaceutical logistics center to verify the effectiveness and practical applicability of the improved model, it is necessary to focus on the complexity and reasoning efficiency of the model.

[0123] The CA attention mechanism is introduced to improve the accuracy and efficiency of target detection. The WIoU loss function is introduced as the bounding box regression loss function to improve the detection accuracy and generalization performance of the model. However, it does not consider how to reduce the model complexity and reasoning time to improve reasoning efficiency.

[0124] Therefore, we choose to further introduce the GSConv (Grouped Shuffle Convolution) + Slim Neck architecture.

[0125] GSConv is a lightweight convolution method that combines depthwise separable convolution (DSC) and channel shuffle operations to reduce the number of model parameters and computational complexity while maintaining or improving feature extraction capabilities. In this way, GSConv can maximize the number of channels of feature maps while minimizing the width and height of feature maps, thereby achieving an effective balance between computational cost and performance. Slim Neck is a design paradigm that applies GSConv to the neck of the detector to reduce computational complexity and inference time while maintaining high accuracy. The GSConv+Slim Neck architecture is an object detector architecture that combines the GSConv and Slim Neck structures.

[0126] By integrating the attention mechanism into the GSConv+Slim Neck architecture, the performance of the object detector can be further improved. This method can not only improve the model's positioning accuracy of the target, but also improve the performance and efficiency of the object detection model while keeping the model lightweight.

[0127] (1) GSConv

[0128] Figure 7 What is shown is the calculation process of ordinary convolution (SC), Figure 8 The calculation process of the depthwise separable convolution (DSC) is shown. As can be seen from the figure, DSC greatly reduces the number of parameters required by the model by decomposing the convolution operation into two steps: depthwise convolution and pointwise convolution. In the depthwise convolution stage, applying the convolution kernel to each input channel separately significantly reduces the redundancy of parameters. However, this leads to the separation of the channel information of the input image during the calculation process, making the feature extraction and fusion capabilities of DSC much lower than SC. In order to make the output of DSC as close to SC as possible, a mixed convolution of SC, DSC and shuffle is performed and named GSConv.

[0129] like Fig. 9 The figure shows the calculation process of GSConv. Shuffle is a hybrid strategy that can completely mix the information from SC into the output of DSC by uniformly exchanging local feature information on different channels without any additional functions.

[0130] GSConv is used to solve the problem of speed of prediction calculation in convolutional neural network (CNN). In the backbone network of CNN, the input image almost always undergoes a similar transformation process: spatial information is gradually transferred to the channel. Each spatial (width and height) compression and channel expansion of the feature map will lead to partial loss of semantic information. SC maximizes the retention of implicit connections between each channel, while DSC completely cuts off these connections. GSConv maintains these connections as much as possible and has lower time complexity. Time complexity is defined by floating point operations (FLOPs), so the time complexity of SC, DSC and GSConv is shown in equations (13), (14), and (15):

[0131] Time sc ~O(W·H·K1·K2·C1·C2)(13)

[0132] Time DSC ~O(W·H·K1·K2·1·C2)(14)

[0133]

[0134] Where W is the width of the output feature map, H is the height, K1·K2 is the size of the convolution kernel, C1 is the number of channels of the input feature map, and C2 is the number of channels of the output feature map. O represents the output or consumption of the process.

[0135] (2) Slim Neck

[0136] The basic principles of Slim Neck are: the introduction of GSConv, modular elements, and flexibility. GSConv has been introduced in the previous article. Its existence is to speed up the prediction calculation of images. The modular elements refer to the fact that after GSConv, the researchers continued to introduce the GS bottleneck and cross stage partial network modules (VoV-GSCSP). These modules are designed to further improve performance, such as Fig.10 shown.

[0137] Flexibility means that if you flexibly use GSConv, GS bottleneck, VoV GSCSP1, VoV GSCSP2, and VoV GSCSP3 modules, you can use these modules to easily build the Slim Neck layer.

[0138] Considering the actual reasoning test of the model, GSConv can speed up the model reasoning speed, so the introduction of GSConv is necessary. The introduction of other modules is used to further improve the model accuracy, but the increase of modules will affect the model reasoning speed. Therefore, only GSConv is added on the basis of the introduction of the CA module, and no other module elements are introduced.

[0139] The basic structure of YOLOv8 is mainly divided into the input layer, backbone network, feature enhancement module and detection head. The following is a brief introduction to these components:

[0140] (1) Input

[0141] The input layer of YOLOv8 is responsible for receiving the raw image and converting it into a tensor format that the deep learning model can understand. Typically, the size of the input layer matches the image dimensions of the training data to ensure that the model can effectively process the input data.

[0142] (2) Backbone

[0143] The backbone network is the core component of YOLOv8 and is responsible for extracting features from the input image.

[0144] (3) Feature Enhancement Module (Neck)

[0145] The role of the Neck layer is to pool and fuse the input feature map to help the model better understand the input image and improve detection performance.

[0146] (4) Detection Head

[0147] The detection head is responsible for performing object detection on the output of the backbone network to generate bounding boxes and category probabilities for the object. In YOLOv8, the detection head may use a multi-scale prediction strategy to generate predictions in different layers of the network at the same time to capture objects of different sizes and scales.

[0148] The backbone network of the YOLOv8 model includes 5 CBS modules, 4 C2f modules and 1 SPPF module. The role of the backbone network is to extract features from the input image. The Neck layer of the YOLOv8 model imitates the organizational structure in PANet. The output feature map of the SPPF module is first upsampled (Upsample means the upsampling process), and then downsampled. The corresponding feature map in the upsampling process is fused with the feature map in the downsampling process, which can greatly enhance the network's feature extraction and feature fusion capabilities. The generated fused feature map is then sent to the Detect Head layer to perform the detection task and generate information such as the target's bounding box and category probability.

[0149] The improved YOLOv8 model framework is as follows Fig.11 As shown in the figure, the CA attention mechanism improves the accuracy and efficiency of target detection. By introducing the WIoU loss function as the bounding box regression loss function, the detection accuracy and generalization performance of the model are improved. The GSConv+Slim Neck architecture is introduced to reduce the model complexity and reasoning time to improve reasoning efficiency.

[0150] In the deep convolution stage, applying convolution kernels to each input channel separately significantly reduces parameter redundancy. However, this results in the separation of channel information of the input image during the calculation process, making the feature extraction and fusion capabilities of DSC much lower than SC. In order to make the output of DSC as close to SC as possible, researchers introduced a new method - a hybrid convolution of SC, DSC and shuffle. The use of GSConv is to solve the speed problem of prediction calculation in convolutional neural networks (CNN).

[0151] GSConv is a lightweight convolution method that combines depthwise separable convolution (DSC) and channel shuffle operations to reduce the number of model parameters and computational complexity while maintaining or improving feature extraction capabilities. In this way, GSConv can maximize the number of channels of feature maps while minimizing the width and height of feature maps, thereby achieving an effective balance between computational cost and performance. Slim Neck is a design paradigm that applies GSConv to the neck of the detector to reduce computational complexity and inference time while maintaining high accuracy. The GSConv+Slim Neck architecture is an object detector architecture that combines the GSConv and Slim Neck structures.

[0152] By integrating the attention mechanism into the GSConv+Slim Neck architecture, the performance of the object detector can be further improved. This method can not only improve the model's positioning accuracy of the target, but also improve the performance and efficiency of the object detection model while keeping the model lightweight.

[0153] Step 4: Experimental preparation and data collection

[0154] (1) Data source

[0155] In order to train the model for the classification and review business solved by this research, it is necessary to establish a certain number of data sets and annotate these data sets. The data sets used in this paper include actual site photos, photos collected from the Internet, and photos generated by digital twins. There are more than 2,000 photos involved, including photos of material boxes, cartons, and barcodes. Figure 12 to Figure 14 .

[0156] (2) Data generation based on Omniverse digital twin

[0157] Since the data set of field shooting is limited and it is impossible to collect data in consideration of lighting, environment, object posture and other conditions in the actual scene, the study explored a new way of data generation - data generation based on digital twins. Using digital twin technology for data exploration has significant feasibility and advantages. Digital twin technology can simulate a variety of complex real-world scenes, including different environmental conditions, lighting changes, object postures, etc., thus providing a highly controllable data generation process. This controllability enables researchers to generate the large amount of data required for specific scenarios and problems without spending a lot of time and resources on field collection.

[0158] Based on Nvidia's Omniverse software for data generation, this work aims to use digital twin technology to generate virtual scenes and data of the real world. By using the Omniverse code in the Omniverse software, various environments can be simulated and the required data can be generated. This data can be used to train deep learning models, help improve the generalization ability of the model, and achieve better performance in tasks such as visual perception and target detection.

[0159] like Fig.15 Shown is a portion of the bin dataset synthesized using the Omniverse code.

[0160] (3) Dataset annotation and division

[0161] The present invention divides the data set into a training set, a validation set, and a test set in a ratio of 8:1:1. In order to be able to train the target detection model of this paper well, the collected data set needs to be labeled. Among them, the label of the carton is 0, the label of the material box is 1, and the label of the barcode is 2.

[0162] (4) Experimental environment configuration

[0163] In terms of environmental configuration, this article selects the GPU model as RTX 3060, the operating system as Ubuntu 20.04, and the programming language as Python3.8. The specific environment configuration is shown in Table 1.

[0164] Table 1 Link configuration parameters

[0165]

[0166] (5) Selection of evaluation indicators

[0167] In order to verify the performance of the improved YOLOv8 algorithm in this paper, the precision, recall, model parameter quantity, floating point operations (FLOPs), mean average precision (mAP), latency and frame rate (FPS) are selected as important indicators for experimental evaluation.

[0168] 1) Precision and recall

[0169] Precision refers to the proportion of samples that are actually positive among the samples predicted by the classification model to be positive. It is one of the important indicators for evaluating the prediction accuracy of the classification model. The formula for calculating precision is as follows (16).

[0170]

[0171] Among them, TP (True Positive) indicates the number of positive samples correctly predicted by the model as positive samples, and FP (False Positive) indicates the number of negative samples incorrectly predicted by the model as positive samples. The precision rate ranges from 0 to 1, and the higher the value, the higher the accuracy of the model in predicting samples as positive categories.

[0172] Precision is used together with recall, and the two together constitute the performance evaluation index of the classification model. Recall, also known as sensitivity or true positive rate, refers to the ratio of the number of samples correctly predicted as positive by the classification model to the number of actual positive samples. It is also one of the important indicators for evaluating the ability of the classification model to discover positive samples. The calculation formula of recall is as follows (17).

[0173]

[0174] Among them, TP (True Positive) indicates the number of positive samples correctly predicted by the model as positive samples, and FN (False Negative) indicates the number of positive samples incorrectly predicted by the model as negative samples. The recall rate ranges from 0 to 1, and the higher the value, the stronger the model's ability to discover positive samples.

[0175] 2) mAP (mean Average Precision)

[0176] mAP (mean Average Precision) is an indicator used to evaluate model performance in object detection tasks. Its mathematical expression is as shown in (18).

[0177]

[0178] Where AP is the average precision of each category, mAP@0.5 and mAP@[0.5:0.95] are two different indicators used to evaluate model performance in object detection tasks.

[0179] mAP@0.5 is calculated by setting the confidence threshold of the detection results to 0.5 when calculating mAP. For each category, the detection results are sorted from high to low according to the confidence, and the AP value of each category is calculated. When calculating Precision and Recall, only the part of the detection results with a confidence greater than or equal to 0.5 is considered. mAP@0.5 is the average AP value of all categories.

[0180] mAP@[0.5:0.95] is when calculating mAP, the confidence threshold of the detection result is gradually increased from 0.5 to 0.95, and the mAP values ​​under multiple different thresholds are calculated and averaged. For each category, Precision and Recall are calculated at different confidence thresholds, and the AP value under each threshold is calculated. Then the average of these AP values ​​is taken, which is the mAP value of the category in the entire confidence range. Finally, mAP@[0.5:0.95] is the average of the mAP values ​​of all categories in the confidence threshold range from 0.5 to 0.95.

[0181] Step 5: Experimental results and analysis

[0182] Experiment 1: In order to reflect the superiority of the improved YOLOv8 model, this study selected the current popular deep learning-based target detection algorithm for comparison, and used the RTX 3060 graphics card to train the model on the COCO dataset. The experimental parameters are the general parameters for COCO dataset training.

[0183] Experiment 2: In order to verify the effectiveness of the improved model in this paper in completing the logistics center target detection task, the model was trained based on the self-built data set. The learning rate was set to 0.001, the batch size was 8, and the number of iterations was 1000.

[0184] (1) Analysis of Experiment 1 Results

[0185] Table 2 Environmental configuration parameters

[0186]

[0187]

[0188] The YOLOv8 model used in this paper, the YOLOv8 model with CA and WIoU, and the YOLOv8 model with GSConvSlim Neck are compared, and the indicators of some current mainstream target detection models are compared. Table 2 shows the comparison results between the models. The last two rows are the model training results of the first improvement (introducing CA attention mechanism and WIoU loss function) and the second improvement (further introducing GSConv+Slim architecture) of this paper.

[0189] By comparing with the original YOLOv8 model, we can see that after the first improvement, the mAP0.5 value of the model increased from 60.2% to 66.0%, and the mAP0.95 value increased from 39.0% to 42.0%, with an increase of 5.8% and 3.0% respectively. After the second improvement, the mAP0.5 value of the model increased from 60.2% to 68.1%, and the mAP0.95 value increased from 39.0% to 44.2%, with an increase of 7.9% and 5.2% respectively. This shows that in the process of improving the YOLOv8 model in this paper, each improvement will improve the accuracy of the model, proving that the improvement method in this paper is effective.

[0190] Although the latency increased from 6.3ms to 6.8ms and the inference speed decreased slightly after the first introduction of the CA attention mechanism and the WIoU loss function, this is because the network structure was added when the attention mechanism was introduced to improve the accuracy of the model. The increase in network structure modules slowed down the inference speed. However, the slowdown in inference speed was not large, and the accuracy increased a lot in comparison, indicating that the first improvement was very effective. After the GSConv+Slim architecture was further introduced for the second time, since the purpose of introducing this architecture was to reduce model complexity and inference time, the inference speed of the model was slightly improved compared to the first improvement on the basis of improving accuracy, proving that the improvements in this article are very effective.

[0191] Compared with other mainstream deep learning-based object detection algorithms, it can be found that the mAP value of the improved model proposed in this paper has increased significantly compared with the original model. Although it is slightly inferior to the mAP increase of the YOLOv3-tiny and YOLOv4-tiny models that introduced the Slim Neck architecture, it may be due to the low mAP value of the original model. After introducing the GSConv+Slim architecture on the basis of CA and WIoU, the mAP value of the model is higher than that of other models, which also proves the effectiveness and superiority of the improved model in this paper.

[0192] like Fig.16 As shown in the figure, the accuracy and inference speed of the improved model in this paper are compared with other models after the Slim Neck architecture is introduced. It can be clearly seen that the YOLOv8 model after the GSConv+Slim Neck architecture is greatly improved compared with the original model, and the comparison results with other models also reflect the superiority of the improved model in this paper.

[0193] (2) Analysis of Experiment 2 Results

[0194] In order to verify the effectiveness of the improved model in this paper in completing the target detection task of the logistics center, model training was carried out based on the self-built data set.

[0195] like Figures 17 to 19 As shown in the figure, in order to verify the convergence performance of the improved model, the convergence of the original YOLOv8 model and the improved YOLOv8 model proposed in this paper is compared. The figure contains the loss curves of bounding box loss, confidence loss, and classification loss, and also draws the convergence curves of the four indicators of accuracy, recall, mAP@0.5, and mAP@[0.5:0.95] for comparison.

[0196] By comparison, it can be found that the YOLOv8 model converges faster after only introducing the CA attention mechanism and WIoU loss function, and has less fluctuation at the end of convergence than the original YOLOv8 model. After further introducing the GSConv+Slim Neck architecture, the convergence curves of bounding box loss, confidence loss, and classification loss are faster and smoother, and the performance is also better in the four indicators of accuracy, recall, mAP@0.5, and mAP@[0.5:0.95]. This shows that each improvement proposed in this article is effective, and the final model training effect is the best.

[0197] The previous analysis has shown that the improved YOLOv8 model is superior to the YOLOv8 model in performance. This section shows the results of using the improved YOLOv8 model to detect images to prove the improved model's ability to actually detect targets. The detection results are shown in Table 3.

[0198] Table 3 Test results statistics

[0199]

[0200]

[0201] By introducing deep learning methods, the process of the logistics center is improved from a technical level. First, an improved YOLOv8 model is designed. The model performance of YOLOv8 is improved by introducing CA attention mechanism, WIoU loss function and GSConv+Slim Neck architecture, and two experiments are designed. Experiment 1 shows that the detection accuracy of the proposed improved model is greatly improved by comparing with other mainstream models, and the values ​​of mAP0.5 and mAP0.95 reach 68.1% and 44.2% respectively, which is higher in accuracy among the compared models, and the inference speed is not greatly increased. Experiment 2 uses a digital twin-based approach to collect self-built data sets, and generates some data sets using the Omniverse platform. Comparing the convergence diagrams of each improvement in Experiment 2, it can be seen that each improvement is effective, and the final test results show the effectiveness of the practical application of the improved model.

[0202] The previous analysis has shown that the improved YOLOv8 model is superior to the YOLOv8 model in performance. This section shows the results of using the improved YOLOv8 model to detect images to prove the improved model's ability to actually detect targets. The detection results are shown in Table 4.

[0203] Table 4. Statistics of test results in English translation.

[0204]

[0205] like Fig. 20 , Fig.21 As shown in the figure, there are Omniverse synthetic cartons and real cartons, and 0 represents a carton. From the detection results in the figure, we can see that the detection accuracy is 89.1%, 90.8%, 90.1%, 90.1%, and 91.8% respectively, and all cartons are accurately detected.

[0206] like Fig. 22 , Fig.23 As shown in the figure, there are pictures of Omniverse synthetic bins and real bins, 1 represents a bin. From the detection results in the figure, we can see that the detection accuracy is 92.8%, 92.0%, and 91.0% respectively, and all bins are accurately detected.

[0207] like Fig.24 The barcode image is shown. The barcode type is CODE-128. The multiple barcode information in the image are all accurately identified.

[0208] Improve module code

[0209] CA attention mechanism module

[0210]

[0211]

Claims

1. A box detection and barcode recognition method based on an improved YOLOv8 model, characterized in that: The steps include: Step 1): Build an improved YOLOv8 model to perform box detection and barcode recognition, and introduce the target detection of CA attention mechanism into the YOLOv8 model; Introducing WIoU object detection into the YOLOv8 model; Introducing GSConv+Slim Neck architecture for object detection into the YOLOv8 model; Step 2): Based on the data generation of Omniverse digital twins, conduct experiments to collect data and build a self-built data set. The self-built data set contains three categories: cartons, material boxes, and barcodes; Step 3): Train the improved model based on the self-built data set to perform box type detection and barcode recognition.

2. According to the box type detection and barcode recognition method based on the improved YOLOv8 model according to claim 1, it is characterized in that: In the step 1), a CA attention mechanism is added between the C2f and SPPF modules in the backbone network of the original YOLOv8 model to improve the detection accuracy, a GSConv+Slim Neck architecture is added between the C2f and CBS modules to improve the reasoning efficiency, and a WioU loss function is added in the back propagation process to improve the detection accuracy and generalization of the model.

3. According to claim 2, a box type detection and barcode recognition method based on an improved YOLOv8 model is characterized in that: The YOLOv8 original model structure is: The basic structure of YOLOv8 is divided into the input layer, backbone network, feature enhancement module and detection head; (1) Input The input layer of YOLOv8 is responsible for receiving the raw box image and converting it into a tensor format that the deep learning model can understand. Usually, the size of the input layer matches the image size of the training data to ensure that the model can effectively process the input data. (2) Backbone The backbone network is responsible for extracting features from the input image; (3) Feature Enhancement Module (Neck) The role of the Neck layer is to pool and fuse the input feature map; it helps the model better understand the input image and improve the detection performance; (4) Detection Head The detection head is responsible for performing target detection on the output of the backbone network to generate bounding boxes and category probabilities of the target. In YOLOv8, the detection head may use a multi-scale prediction strategy to generate predictions in different layers of the network at the same time to capture targets of different sizes and scales. The backbone network of the YOLOv8 model includes 5 CBS modules, 4 C2f modules and 1 SPPF module; The role of the backbone network is to extract features from the input image; the Neck layer of the YOLOv8 model imitates the organizational structure in PANet, first upsampling the output feature map of the SPPF module (Upsample means the upsampling process), then downsampling, and fusing the corresponding feature map in the upsampling process with the feature map in the downsampling process, which can greatly enhance the network's feature extraction and feature fusion capabilities. The generated fused feature map is then sent to the Detect Head layer to perform the detection task, generating information such as the target's bounding box and category probability.

4. According to claim 3, a box type detection and barcode recognition method based on an improved YOLOv8 model is characterized in that: The CA attention mechanism added in step 1) is divided into three steps from input to output Step 1: Global pooling is often used in channel attention to globally encode spatial information. The global pooling is decomposed. For a box-shaped input feature map x (x is a picture with a length of H and a width of W), each channel is first encoded along the horizontal and vertical coordinate directions using pooling kernels of size (H, 1) and (1, W); The output expression of the cth channel with a height of h is shown in formula (1). Similarly, the output expression of the cth channel with a width of w is shown in formula (2). The input feature map with a size of C*H*W is pooled in the X direction and Y direction respectively to generate box-type feature maps with sizes of C*H*1 and C*1*W respectively. Step 2: After transforming the generated C*1*W box-shaped feature map, perform a cascade (Concat) operation, use a shared 1×1 convolution kernel to reduce the dimension, and reduce its dimension to the original C / r (r represents the downsampling ratio). Then, the batched box-shaped feature map F1 is sent to the Sigmoid activation function to obtain a feature map f of the shape of C / r*1*(W+H), as shown in formula (3); f=δ(F1([z h ,z w ])) (3) Step 3: Split the box-shaped feature map f into two separate tensors f along the spatial dimension h ∈R C / r×H×1 、f w ∈R C / r×1×W , and then use 1×1 convolution kernel to perform dimension increase operation, and then combine the Sigmoid activation function to get the attention vector, g h ∈R C×H×1 , g w ∈R C×1×W ; Its mathematical expressions are shown in formula (4) and formula (5); Finally, by multiplying the original feature map and performing weighted calculation as the attention weight, the feature map with attention weights in the width and height directions will be obtained. The final output of the CA module is shown in formula (6); g h =σ(F h (f h )) (4) g w =σ(F w (f w )) (5) 5. According to claim 3, a box type detection and barcode recognition method based on an improved YOLOv8 model is characterized in that: The WioU loss function in step 1) is an improvement on WIoU v3, and the outlier degree is used to describe the quality of the predicted box. The expression is as follows: Among them, the smaller β is, the higher the quality of the prediction box is. A smaller gradient gain is assigned to it, so that the bounding box regression focuses on the prediction box of normal quality. The larger β is, the lower the quality of the prediction box is. A smaller gradient gain is assigned to it to reduce the adverse effects on low-quality examples. β is used to construct a non-monotonic focusing coefficient r and apply it to WIoU v1. The mathematical expression is as follows: Among them, when β = δ, r = 1, when the outlier degree of the predicted box satisfies β = C (C is a constant), the gradient gain is the highest, because It is dynamic, and the quality division criteria of the prediction box are also dynamic, which enables WIoU v3 to make the gradient gain allocation strategy that best suits the current situation at every moment.

6. According to the box type detection and barcode recognition method based on the improved YOLOv8 model of claim 3, it is characterized in that: The GSConv+Slim Neck architecture is an object detector architecture that combines the GSConv and Slim Neck structures.

7. The box type detection and barcode recognition method based on the improved YOLOv8 model according to claim 1 is characterized in that: The step 2) is specifically as follows: We use actual on-site photos, photos collected from the Internet, and the Omniverse code in the Omniverse software to simulate various environments and generate the required data photos as data sets for experiments.

8. The box type detection and barcode recognition method based on the improved YOLOv8 model according to claim 1 is characterized in that: In step 3), the improved YOLOv8 model is used to perform the tasks of carton, material box and barcode recognition, and accurate box and barcode recognition is achieved in subsequent experiments.

9. The box type detection and barcode recognition method based on the improved YOLOv8 model according to claim 8, characterized in that: The specific identification process is: The improved model marks the type of carton and the barcode, with the carton marked as 1, the material box marked as 2, and the barcode marked as 3. When an object is detected, it will first identify whether it is a carton or a material box. After the box type is detected and classified, the barcode on the box will be synchronously detected. The barcode is first accurately located using the improved YOLOv8 model, and then the Zbar library responsible for barcode recognition will automatically identify the barcode information for the barcode area, thereby completing the box type detection and barcode recognition process.

Images