Article sorting method and device and model training method and device based on SENet and YOLOv5s

Abstract

The invention discloses an article sorting method and device and a model training method and device based on SENet and YOLOv5s, and relates to the field of article sorting. An article sorting mark image to be recognized is recognized through a YOLOv5s target detection model based on a SENet attention mechanism, and the model is constructed by adding a SENet channel attention mechanism module behind a residual block in a CSP module of YOLOv5s and / or adding the SENet channel attention mechanism module behind a last splicing layer in a PAN structure of YOLOv5s; when the model is trained, data enhancement is performed on a training sample through geometric enhancement, color enhancement and noise injection, model optimization is performed by using an AdamW optimizer, a learning rate is set based on a cosine annealing method, overfitting is prevented by using a discarding method and L2 regularization, and pruning and quantification are performed on the model; the method can be suitable for scenes with complex backgrounds, dense targets and large scale changes, and can improve the detection precision and the accuracy while having relatively low calculation overhead and relatively high calculation efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of item sorting, and in particular relates to an item sorting method and device based on SENet and YOLOv5s, and a model training method and device. Background Art

[0002] In the fields of logistics, supply chain, and industrial production, sorting goods and materials is a common means of production management. Goods and materials are usually equipped with sorting marks (such as QR code labels, barcode labels, direction signs, area marks, etc.). According to these sorting marks, the items can be classified, tracked, located, and distributed to ensure that the goods and materials flow efficiently to the correct destination according to established rules. With the rapid development of artificial intelligence and computer vision technology, the key to improving sorting efficiency and reducing labor costs is to use target detection technology to quickly and accurately identify sorting marks on goods and materials. Target detection not only requires high-precision identification of target items, but also real-time response capabilities and low computing overhead. Therefore, how to balance accuracy, computing efficiency, and real-time performance has become the core challenge of target detection technology.

[0003] However, taking logistics scenarios as an example, warehouse environments may face multiple challenges, including changes in lighting conditions, such as strong light, shadows, or low-light scenes; complex visual environments caused by stacked goods and cluttered backgrounds; and the small size of some sorting marks; therefore, scenes with complex backgrounds, dense targets, and large scale changes make it difficult for traditional detection methods to effectively identify them.

[0004] As one of the most representative target detection models, the YOLO (You Only Look Once) series is widely used in embedded systems, unmanned driving, and robotics due to its excellent real-time performance and low computational consumption. As a lightweight model in the YOLO series, YOLOv5s is committed to achieving better target detection performance through structural optimization and efficient reasoning algorithms. However, in scenes with complex backgrounds, dense targets, and large scale changes, using only YOLOv5s for target detection may have the defects of low detection accuracy and weak ability to process complex features. In order to improve the detection accuracy of the model and its adaptability to complex scenes, researchers have proposed a variety of solutions combining YOLOv5s with other solutions in recent years.

[0005] In order to further improve the efficiency of the model, researchers also tried to combine efficient backbone networks such as EfficientNet with YOLOv5. EfficientNet itself has high performance and low computational complexity, so its combination with YOLOv5 can provide more efficient target detection capabilities than traditional YOLO. EfficientNet is used as the backbone network (Backbone) to replace the original backbone network of YOLOv5 (such as CSPDarknet). Through multi-level network structure optimization, EfficientNet can significantly improve the feature extraction ability of the model without increasing too much computation. This solution combines high efficiency and accuracy, and can improve target detection accuracy while ensuring low computation. EfficientNet's own network optimization method makes the calculation more efficient and suitable for deployment on embedded and mobile devices. However, the combination of EfficientNet and YOLOv5 may increase the complexity of the model, resulting in slower reasoning speed, especially in the real-time processing of high-resolution images, which may encounter bottlenecks.

[0006] In addition, there are some improvement schemes that try to improve the performance of YOLOv5s through different channel recalibration mechanisms. For example, channel attention mechanisms such as CBAM (Convolutional Block Attention Module) and BAM (Bottleneck Attention Module) have also been applied to target detection tasks. Adding CBAM or BAM after each convolutional block of YOLOv5s will simultaneously pay attention to the channel and spatial dimensions. Integrating these attention mechanisms into key parts of the YOLOv5s network, such as the backbone network and the neck network (Neck), improves the ability to recognize targets. The channel attention mechanism can effectively increase the focus on key features, thereby improving detection accuracy. Compared with the traditional YOLOv5s structure, it can achieve better performance in scenes with multiple targets and complex backgrounds. However, although the computational overhead is small, it still increases the complexity and computation of the model, which may affect the inference speed. Additional training data and computing resources are required to adjust the parameters of the attention mechanism, which increases the difficulty of training. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes an item sorting method and device, and a model training method and device based on SENet and YOLOv5s, which can be applicable to scenes with complex backgrounds, dense targets, and large scale changes. It has low computational overhead and high computational efficiency, and can improve detection accuracy and improve accuracy.

[0008] In a first aspect, the present invention provides an item sorting method based on SENet and YOLOv5s, comprising:

[0009] Obtaining a sorting mark image of an item to be identified;

[0010] Input the to-be-recognized item sorting mark image into the trained YOLOv5s target detection model based on SENet attention mechanism for recognition, and obtain the predicted category of the to-be-recognized item sorting mark image;

[0011] Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0012] In an optional embodiment, the SENet channel attention mechanism module is added to at least one CSP module in the backbone network of the YOLOv5s network model, including:

[0013] The SENet channel attention mechanism module is added after the residual block in the CSP module; wherein, the SENet channel attention mechanism module is added after the residual block of the last CSP module in the backbone network, or the SENet channel attention mechanism module is added after the residual blocks of the last two CSP modules in the backbone network.

[0014] In an optional embodiment, the adding of a SENet channel attention mechanism module into the PAN structure in the neck network of the YOLOv5s network model comprises:

[0015] Add the SENet channel attention mechanism module after the last concatenation layer in the PAN structure, or add the SENet channel attention mechanism module after the last two concatenation layers in the PAN layer structure.

[0016] In a second aspect, the present invention provides a YOLOv5s target detection model training method based on SENet attention mechanism, comprising:

[0017] Build the YOLOv5s target detection model and loss function based on SENet attention mechanism to be trained;

[0018] Acquire a training sample data set, wherein the training samples in the training sample data set include item sorting mark images and label information of the item sorting mark images;

[0019] Preprocessing the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement;

[0020] Input the training sample data set into the YOLOv5s target detection model to be trained based on the SENet attention mechanism for forward propagation to obtain a training result;

[0021] Inputting the training result and the label information into the loss function to obtain a loss result;

[0022] According to the loss result, back propagation is performed using an optimizer to adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges; and

[0023] The model obtained when the loss function converges is used as a YOLOv5s target detection model based on the SENet attention mechanism;

[0024] Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0025] In an optional embodiment, before or after the step of using the model obtained when the loss function converges as the YOLOv5s target detection model based on the SENet attention mechanism, the step further includes:

[0026] The YOLOv5s target detection model based on the SENet attention mechanism is pruned and quantized to obtain a pruned and quantized YOLOv5s target detection model based on the SENet attention mechanism.

[0027] In an optional embodiment, the item sorting mark images are item sorting mark images under various lighting conditions, various background complexities and various target types, and the data enhancement includes at least one of geometric enhancement, color enhancement and noise injection.

[0028] In an optional embodiment, the construction of a YOLOv5s target detection model and a loss function to be trained based on the SENet attention mechanism includes:

[0029] The network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism are set, and a loss function based on classification loss, positioning loss and confidence loss is constructed; wherein the network parameters include optimizer parameters and learning rate, the optimizer is set to the AdamW optimizer, and the learning rate is set based on the cosine annealing method.

[0030] In an optional embodiment, adjusting the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism includes:

[0031] A dropout layer is added after the SENet channel attention mechanism module, and / or the loss function is adjusted using the L2 regularization method to prevent overfitting.

[0032] In a third aspect, the present invention provides an item sorting device based on SENet and YOLOv5s, comprising:

[0033] An image acquisition module, used to acquire the sorting mark image of the item to be identified;

[0034] A target recognition module is used to input the item sorting mark image to be identified into a trained YOLOv5s target detection model based on SENet attention mechanism for identification, so as to obtain the predicted category of the item sorting mark image to be identified;

[0035] Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0036] In a fourth aspect, the present invention provides a YOLOv5s target detection model training device based on SENet attention mechanism, comprising:

[0037] Model building module, used to build the YOLOv5s target detection model and loss function based on SENet attention mechanism to be trained;

[0038] A training data acquisition module, used to acquire a training sample data set, wherein the training samples in the training sample data set include an item sorting mark image and label information of the item sorting mark image;

[0039] A training data preprocessing module, used to preprocess the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement;

[0040] A forward propagation module is used to input the training sample data set into the YOLOv5s target detection model based on the SENet attention mechanism to be trained for forward propagation to obtain a training result;

[0041] A loss calculation module, used for inputting the training result and the label information into the loss function to obtain a loss result;

[0042] A back propagation module, used to perform back propagation using an optimizer according to the loss result, and adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges;

[0043] A model determination module, used to use the model obtained when the loss function converges as a YOLOv5s target detection model based on the SENet attention mechanism;

[0044] Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0045] The beneficial effect brought by the technical solution provided by the embodiment of the present invention is: in the present invention, the sorting mark image of the item to be identified is identified by the YOLOv5s target detection model based on the SENet attention mechanism, and the model is constructed by adding the SENet channel attention mechanism module to the CSP module of YOLOv5s, and / or adding the SENet channel attention mechanism module to the PAN structure of YOLOv5s, so that it can be suitable for scenes with complex backgrounds, dense targets, and large scale changes, and can improve detection accuracy and accuracy while having low computational overhead and high computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a flow chart of an item sorting method based on SENet and YOLOv5s provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the network structure principle of the SENet channel attention mechanism module provided in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the algorithm principle of the SENet channel attention mechanism module provided for an embodiment of the present invention, which is intended to embody the compression and excitation principles;

[0049] Figure 4 A schematic diagram of the network structure of the YOLOv5s target detection model based on the SENet attention mechanism provided in an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of the network structure of a CSP module with a SENet channel attention mechanism module added according to an embodiment of the present invention;

[0051] Figure 6 A flowchart of a YOLOv5s target detection model training method based on the SENet attention mechanism provided in an embodiment of the present invention;

[0052] Figure 7 A schematic diagram of the system principle of an item sorting device based on SENet and YOLOv5s provided in an embodiment of the present invention;

[0053] Figure 8 A schematic diagram of the system principle of a YOLOv5s target detection model training device based on the SENet attention mechanism provided in an embodiment of the present invention;

[0054] Fig. 9 A schematic diagram of the system principle of an electronic device provided by an embodiment of the present invention.

[0055] In the figure: 110 - image acquisition module; 120 - target recognition module; 210 - model building module; 220 - training data acquisition module; 230 - training data preprocessing module; 240 - forward propagation module; 250 - loss calculation module; 260 - back propagation module; 270 - model determination module; L1 - global information input layer; L2 - adaptive recalibration layer; L3 - weight layer; 400 - electronic device; 401 - communication interface; 402 - processor; 403 - memory; 404 - bus. DETAILED DESCRIPTION

[0056] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0057] See also Figure 1 The item sorting method based on SENet and YOLOv5s of this embodiment includes the following steps S110 to S120.

[0058] Step S110, obtaining the sorting mark image of the item to be identified.

[0059] This embodiment is explained by taking the logistics scenario as an example. The item sorting mark refers to a label pasted on the outer packaging of the goods (for example, a QR code, a barcode, etc.), or a mark printed on the outer packaging of the goods (for example, a direction mark, an area mark). The method of this embodiment is applied to the NVIDIA Jetson AGX Xavier embedded device, which has real-time processing capabilities. The trained YOLOv5s target detection model based on the SENet attention mechanism is deployed to the logistics automatic sorting system, and the automated operation is completed in combination with the conveyor belt and the robotic arm. The high frame rate camera captures the image of the item sorting mark to be identified on the conveyor belt with a resolution of 1280x720.

[0060] Step S120: input the to-be-recognized item sorting mark image into the trained YOLOv5s target detection model based on the SENet attention mechanism for recognition, and obtain the predicted category of the to-be-recognized item sorting mark image.

[0061] In specific implementation, after the image is input into the trained model for recognition, the model outputs the category (label / mark), confidence and location information of each target. According to the label information, the robot arm is guided to complete the grasping and sorting. According to the sorting mark, the direction of cargo placement or the conveying path is adjusted.

[0062] Among them, the YOLOv5s target detection model based on SENet attention mechanism is obtained by adding SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or adding SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model. The attention mechanism can well improve the accuracy of the network when extracting target features. In the process of extracting target features in the traditional network, it does not consider the influence of different feature channels on the detection target. However, the idea of ​​the attention mechanism will learn by processing a large amount of data, and the weight value corresponding to the degree of dependence on each channel will be obtained in this process. After adding the SENet channel attention mechanism module to YOLOv5s, the feature map originally output by the convolution layer will be sent to the SENet channel attention mechanism module for channel weighted operation, and then the enhanced feature map will be obtained. In this way, YOLOv5s can better capture the details of the target, especially when the target and background are complex and the target scale is large.

[0063] The SENet (Squeeze-and-Excitation Networks, channel attention mechanism) attention module is a convolutional neural network architecture based on the attention mechanism. It introduces the SE (Squeeze-and-Excitation) module to learn the weights of each channel of the input features, allowing the network to focus on more important information, thereby improving the network's representation ability and performance. SENet was first proposed in the article "Squeeze-and-Excitation Networks" released by Hu Jie's team at Momenta, a domestic autonomous driving company in 2017. The channel attention mechanism correlates the feature channels in the feature map extracted by the network, and represents the importance of different feature channels based on weights. The structure of the SENet channel attention mechanism module is as follows: Figure 2 shown.

[0064] The SENet channel attention mechanism is implemented through two parts: compression and excitation. The specific process of its implementation is as follows: Figure 3 As shown, where X represents input, X ～ Denotes the output, W, H, and C denote the width, height, and number of channels of the network, respectively. Compression compresses the information of each channel through global average pooling (GAP) to generate a global descriptor for each channel; motivation learns the importance of each channel through a lightweight fully connected network and generates a channel weight vector, which is then multiplied by the original feature map to re-weight the channel.

[0065] The two parts of compression and excitation form a computational unit, which can be built on the transformation Ftr mapping input X∈R (H ' ×W'×C') To the feature map U∈R (H×W×C) , consider Ftr as a convolution operator and use V = [V1, V2, ...., V C ] represents the learned filter kernel set, where Vc refers to the parameters of the cth filter, and the output U = [U1, U2, ..., U c ], as shown in formula (2.1).

[0066]

[0067] Among them, * indicates the convolution operation. is a two-dimensional kernel representation of V c A single channel acts on the corresponding channel of X, X = [x 1 ,x 2 ,…,x c′ ],Uc ∈R (H×W) .

[0068] Formula 2.1 adds the SENet channel attention mechanism module to the key parts of YOLOv5s such as CSPDarknet or PANet, and sends the feature map originally output by the convolutional layer to the SENet channel attention mechanism module for channel weighted operation.

[0069] Compression and excitation are described in detail below, as described in (1) to (2).

[0070] (1) Compression layer, i.e. Figure 3 The global information embedding layer L1 in the compression layer extracts the global information of each channel through global average pooling.

[0071] The local receptive field is used to process each learning filter in the model, but for each single source of output change U, it is impossible to use the contextual information outside the area. At this time, the channel statistics generated based on global average pooling can be used to process the global spatial information and compress it in the channel descriptor to solve the problem. The specific implementation process is shown in the following formula (2.2).

[0072]

[0073] Among them, Z C represents the cth element of Z, Z∈R C , is obtained by reducing the spatial dimension of U to H×W; H represents the height of the feature map, and W represents the width of the feature map; U c ,(i,j) represents the pixel value of the cth channel in the feature map.

[0074] (2) Excite n layers, that is Figure 3 The adaptive recalibration layer L2 in the excitation layer generates a channel weight vector (i.e. Figure 3 The weight layer L3 in the image is used to weight the feature map and enhance the expressiveness of key features.

[0075] The purpose of the excitation layer is to fully capture channel dependencies. The function of the excitation layer must meet two criteria. First, it must be flexible; second, it must learn a non-mutually exclusive relationship because it is hoped to ensure that multiple channels are allowed to be emphasized (rather than enforcing a one-time activation). In order to meet these criteria, a simple gating mechanism with sigmoid activation is chosen. The specific implementation process is shown in the following formula (2.3).

[0076] S=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)), (2.3)

[0077] Among them, δ represents the result of the activation function ReLU; W1 represents the weight matrix of the first layer, W1∈R (C / r×C) ; W2 represents the weight matrix of the second layer, W2∈R (C / r×C) .

[0078] The final output of the SENet channel attention mechanism module is obtained by rescaling U using the activation function S, as shown in Equation (2.4).

[0079] X ～ c =F scale (U c ,S c )=S c U c , (2.4)

[0080] Where X ～ c =[X ～ 1,X ～ 2,……X ～ c ] and F scale (U c ,S c ) refers to the scalar S C And the feature map U c ∈R H×W The channel multiplication between .

[0081] The design and joining method of SENet are introduced in detail below.

[0082] The key point of integrating SENet into YOLOv5s is how to integrate it with the original YOLOv5 network structure. Usually, we will choose to add SENet to the CSPDarknet or PANet part of YOLOv5s. Doing so can enhance the channel-level expression of the feature map and improve the accuracy of the target model (i.e., the YOLOv5s target detection model based on the SENet attention mechanism in this embodiment). It should be noted that, without changing the original structure of YOLOv5 (such as the backbone network and the neck network), ensure that the addition of the SENet channel attention mechanism module does not increase the computational burden too much. Specifically, the SENet channel attention mechanism module is added after the residual block in the CSP module; wherein, the SENet channel attention mechanism module is added after the residual block of the last CSP module in the backbone network, or the SENet channel attention mechanism module is added after the residual blocks of the last two CSP modules in the backbone network. Add the SENet channel attention mechanism module after the last splicing layer in the PAN structure, or add the SENet channel attention mechanism module after the last two splicing layers in the PAN layer structure. Further, the scaling factor of the SENet channel attention mechanism module is 14 to 19. Preferably, the scaling factor of the SENet channel attention mechanism module of this embodiment is 16 (ie, a scaling ratio of 1 / 16 is used).

[0083] See also Figure 4 and Figure 5 As shown, in this embodiment, in the last two CSP modules in the backbone network ( Figure 4 The residual block of the CSP (SENet) backbone network in Figure 5 After adding the SENet channel attention mechanism module ( Figure 5 In the SENet in the neck network, a SENet channel attention mechanism module ( Figure 4 SENet of the mid-neck network, four in total).

[0084] It should be noted that the SENet channel attention mechanism module is added only after the residual block of CSPD. This is because the residual block of CSPD is used to retain key features under the premise of lightweight design. After the output of the residual block is weighted by SENet, the importance of the feature channel can be further strengthened, which helps to improve the overall feature expression ability. It should also be noted that the present embodiment adds the SENet channel attention mechanism module after the multi-scale feature map is spliced ​​or fused (Concat operation). This is because in the pyramid feature fusion stage, the target model needs to process feature maps of different scales (such as features of large and small targets), and the SENet channel attention mechanism module can adaptively adjust the channel weights to ensure the effective aggregation of multi-scale information. This embodiment only processes the feature map output by the residual block, rather than adding the SENet channel attention mechanism module to all feature layers of CSPD one by one, and does not apply additional SENet channel attention mechanism modules to other parts of the backbone network of YOLOv5s.

[0085] In addition, it is also necessary to control the number of SENet channel attention mechanism modules added to the entire target model, and only add them to important modules to avoid redundant calculations. For example, in the backbone network, the SENet channel attention mechanism module is only added after 1 to 2 deep residual blocks to balance performance and computational efficiency; and the residual blocks in the middle and high levels are preferred, such as the middle and rear parts of the backbone network (close to the feature output part). For the neck network, the SENet channel attention mechanism module is added after 1 to 2 key splicing layers or upsampling layers, and the higher layers (i.e., the layers close to the final prediction, such as the deep splicing layers of PANet) are preferred, because the multi-scale features are integrated here, which plays an important role in target classification and positioning.

[0086] A possible implementation of adding the SENet channel attention mechanism module is further described below through the following steps (1) to (3).

[0087] In step (1), the global information is extracted after the concatenation operation through the compression layer. Combined with formula (2.2), the global information of the concatenated feature map is compressed through global average pooling. The global average value of each channel is calculated to generate the global descriptor of the channel.

[0088] Step (2) generates weights through the excitation layer. Combined with formula (2.3), two fully connected layers (FC) and activation functions are used to model the channel descriptors and generate channel weight vectors. The width of the first layer is usually reduced by a scaling factor (such as r = 16r = 16r = 16) to control the number of parameters.

[0089] Step (3) weights the concatenated features. Combined with formula (2.4), the generated weight vector is multiplied channel by channel with the concatenated feature map to complete the channel weighting.

[0090] The advantages of YOLOv5s with the SENet channel attention mechanism module are as follows (1) to (3).

[0091] (1) Improve the representation ability of the target model. The SENet channel attention mechanism module can effectively capture the dependencies between channels, allowing the network to better focus on features that are helpful for target detection while ignoring irrelevant or noisy features. Especially in complex backgrounds or multi-target scenes, the SENet channel attention mechanism module can help the network focus on key areas and improve accuracy.

[0092] (2) Enhance the distinguishability of features. By adaptively adjusting the weights of feature channels, the SENet channel attention mechanism module improves the network’s ability to distinguish target features, especially when the target scale and background interference are large, and can better extract useful information.

[0093] (3) Improved accuracy. In some standard target detection tasks (such as the COCO dataset), the SENet channel attention mechanism module improves the mAP (meanAverage Precision) of the target model, that is, improves the detection accuracy and recall rate.

[0094] The core task of target detection is to extract effective features from images and classify, locate and identify them through the network. The SENet channel attention mechanism module strengthens the network's ability to focus on key features through the channel attention mechanism. In the feature extraction stage, it can automatically learn which channels (different channels of the feature map) are more important to the final target detection results, thereby enhancing the ability to identify targets in complex scenes. This embodiment compresses global information through compression operations and adjusts the weight of each channel through excitation operations, so that the target model can adaptively weight the key features of the target, thereby enhancing the expressive power of the features.

[0095] The detection accuracy of the sorting mark in this embodiment reaches 94.3%, and the label detection accuracy is 92.7%. Due to the enhanced effect of the SENet channel attention mechanism module, the false alarm rate is reduced by 12%. The inference speed tested on AGX Xavier is 33FPS, which meets the real-time requirements.

[0096] See also Figure 6 The YOLOv5s target detection model training method based on the SENet attention mechanism provided in this embodiment includes the following steps S210 to S270.

[0097] Step S210, constructing a YOLOv5s target detection model and loss function based on the SENet attention mechanism to be trained. The YOLOv5s target detection model based on the SENet attention mechanism to be trained is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0098] After adding the SENet channel attention mechanism module, it is necessary to ensure that the channel weighting does not over-adjust the balance structure of the original network. Therefore, the stability of network training should be ensured through appropriate weight initialization and learning rate settings. The network parameter settings in this step include optimizer parameters and learning rate, and the optimizer is set to AdamW optimizer. The initial learning rate is set to 0.01, and the learning rate is set based on the cosine annealing method. The learning rate is dynamically reduced to prevent overfitting. In this step, loss functions based on classification loss (CrossEntropy), positioning loss (GIoU) ​​and confidence loss are also constructed.

[0099] Step S220: acquiring a training sample data set, wherein the training samples in the training sample data set include item sorting mark images and label information of the item sorting mark images.

[0100] Here, about 1,000 item sorting mark images were collected from logistics warehouse scenes. The item sorting mark images are item sorting mark images of various lighting conditions, various background complexities, and various target types. Specifically, cargo images under different lighting conditions, such as normal lighting, strong light, weak light, and shadow scenes; cargo images under different background complexity conditions, including cargo images under solid color backgrounds, cargo stacking backgrounds, and complex warehouse environments; cargo images of different target types, including cargo images covering a variety of label styles (QR codes, barcodes) and marks (such as direction arrows).

[0101] Step S230, preprocessing the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement.

[0102] Here, for the target detection model, especially the target model with the SENet channel attention mechanism module, it may overfit some specific types of targets (such as smaller targets, low-contrast targets, etc.), so the model needs to have strong generalization ability. This embodiment needs to ensure its stable performance through more diverse training data and model verification, and the diversity of data (including different backgrounds, illumination, scale, etc.) is very important. This embodiment simulates different scenes and changes through data enhancement to enhance the robustness of the target model. The data enhancement of this embodiment includes geometric enhancement, color enhancement and noise injection. Among them, geometric enhancement can be random cropping, rotation, translation, and geometric enhancement of the image can enhance the robustness to changes in target direction and position; color enhancement can be brightness, contrast, and hue adjustment, and color enhancement can adapt to various lighting conditions; noise injection refers to adding images of simulated image noise to enhance the anti-interference ability of the target model. The geometrically enhanced images, color-enhanced images, and noise-injected images are added to the original training sample data set. This embodiment increases the diversity of the training set through a data enhancement strategy, so that the network can generalize better. It should be emphasized that the data enhancement in this embodiment is to generate new training samples and add them to the original training sample data set (the number of samples increases). This can expand the data scale to prevent overfitting, solve the category imbalance problem, improve the robustness of the target model, and facilitate learning more diverse features from a larger data set scale.

[0103] Step S240, input the training sample data set into the YOLOv5s target detection model to be trained based on the SENet attention mechanism for forward propagation to obtain the training result.

[0104] Step S250, input the training results and label information into the loss function to obtain the loss result.

[0105] Step S260, according to the loss result, back propagation is performed using the optimizer to adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges.

[0106] Step S270, using the model obtained when the loss function converges as the YOLOv5s target detection model based on the SENet attention mechanism.

[0107] One of the advantages of YOLOv5s is its lightweight structure, fast reasoning speed, and is suitable for running on resource-constrained devices. When adding the SENet channel attention mechanism module, it is necessary to be careful not to increase the number of parameters and the amount of calculation of the target model too much. Too many parameters may lead to a decrease in reasoning speed, especially when the performance on edge devices (such as embedded devices, mobile devices, etc.) no longer meets the real-time requirements. The model can be compressed after adding the SENet channel attention mechanism module by pruning, quantization, etc., to reduce the additional computational burden introduced and maintain the efficiency of the original structure. Therefore, before or after step S270 of this embodiment, the following steps are also included: pruning and quantizing the YOLOv5s target detection model based on the SENet attention mechanism to obtain a pruned and quantized YOLOv5s target detection model based on the SENet attention mechanism.

[0108] In target detection, over-reliance on the SENet channel attention mechanism may lead to overfitting of the target model, because it may cause excessive attention to local features and ignore global information by enhancing the feature expression of specific channels. For this reason, it is necessary to combine regularization methods (such as discarding method, L2 regularization) during training to prevent overfitting and avoid over-reliance on the channel attention mechanism. Therefore, the adjustment of the network parameters in this embodiment includes adding a discarding layer after the SENet channel attention mechanism module, and adjusting the loss function using the L2 regularization method to prevent overfitting. The network parameter adjustment strategy of this embodiment during training can reduce the amount of target model parameters, optimize the calculation graph, reduce redundant calculations, and other methods to reduce the computational complexity and memory consumption of the target model.

[0109] In addition, compared with the standard Adam optimizer, the AdamW optimizer adds an L2 regularization term when updating weights. Therefore, regularization is also reflected in the AdamW optimizer, that is, by adding a penalty term to the loss function to limit the size of the weight, prevent overfitting, and improve the generalization ability of the target model.

[0110] The mAP of the trained model is 4.5% higher than that of using YOLOv5s alone, and the improvement is particularly significant in the small object detection task; the training time increases by about 15% compared to using YOLOv5s alone, but the convergence speed of the target model remains stable.

[0111] The strategies adopted during the target model training process are further described below.

[0112] 1. Pruning and quantization to maintain the efficiency of the original structure.

[0113] (1) Pruning. The goal of pruning is to reduce the computational workload and storage overhead of the target model by removing redundant neural network parameters. After adding the SENet channel attention mechanism module, although the feature expression capability is improved, it will also lead to an increase in the number of parameters, so the target model needs to be pruned and optimized. For example, in the YOLOv5s network, some channels in the convolutional layer can be pruned. The weight values ​​of these channels may be relatively small and have low contributions. Through pruning operations, the computational burden can be reduced without significantly affecting the accuracy.

[0114] You can use TensorFlow Lite (TFLite for short, a lightweight machine learning framework) or PyTorch's pruning API (a pruning tool for neural network models) for pruning, by gradually reducing some unimportant channels or convolution kernels during training, or using a gradient-based pruning algorithm to evaluate the contribution of each channel and prune the low-contribution part. After pruning, you can further fine-tune to restore accuracy.

[0115] (2) Quantization. Quantization is the process of converting the floating-point weights of the target model to low-precision (such as 8-bit integers) without significant loss of accuracy. This method greatly reduces memory usage and speeds up the inference process. For YOLOv5s with the SENet channel attention mechanism module added, quantization-aware training (QAT) can be used to quantize the target model to ensure a balance between accuracy and speed during inference.

[0116] You can use the quantization tools in TensorFlow Lite or PyTorch to integer quantize the target model. Quantization not only reduces the storage space of the target model, but also uses hardware accelerators (such as quantized CPUs or mobile accelerators) to increase the inference speed.

[0117] 2. Dropout and L2 regularization methods are used to avoid over-reliance on the channel attention mechanism.

[0118] (1) Dropout: This is a common regularization technique that randomly drops the output of some neurons during training to reduce the network's over-reliance on certain nodes and prevent the target model from overfitting. After combining the SENet channel attention mechanism module, the dropout method can be applied after the SENet channel attention mechanism module to reduce the risk of the attention mechanism over-emphasizing certain features. Specifically, during training, the activation values ​​of some neurons are randomly dropped to force the network to learn more generalized feature representations.

[0119] In specific implementation, a Dropout layer is added after the SENet layer, and an appropriate Dropout layer ratio is set (usually between 0.3 and 0.5). This can effectively prevent the SENet channel attention mechanism module from over-adjusting the channel weights, thereby reducing overfitting.

[0120] (2) L2 regularization: L2 regularization penalizes the complexity of the target model by adding the square sum of the parameters to the loss function, preventing the target model from overfitting the training data. When the SENet channel attention mechanism module is added, L2 regularization can prevent the weight value of the target model from being too large, thereby avoiding overfitting.

[0121] During training, you can add an L2 regularization term to the loss function to control the size of the network weights. For example, in PyTorch, you can implement L2 regularization through the weight_decay parameter.

[0122] 3. Ensure that weight initialization and learning rate settings are reasonable.

[0123] (1) Weight initialization: When adding the SENet channel attention mechanism module, the parameter distribution of the network is changed. Therefore, appropriate weight initialization is very important. Common initialization methods such as He initialization (applicable to ReLU activation function) can effectively prevent gradient disappearance or gradient explosion.

[0124] Use PyTorch or TensorFlow's built-in initialization method to initialize the weights of the SENet layer to ensure the stability of network training.

[0125] (2) Learning rate setting: After adding the SENet channel attention mechanism module, the target model structure becomes more complex, so the learning rate setting needs to be more cautious. A learning rate scheduler (such as Cosine Annealing or ReduceLROnPlateau) can be used to dynamically adjust the learning rate during training to avoid excessively large learning rates in the early stages of training that lead to unstable target models, or too small learning rates in the later stages that lead to slow training.

[0126] During the training process, when building the target model, the cosine annealing method is used to set the learning rate; using dynamic learning rate adjustment tools such as Warm-up, you can first use a smaller learning rate for warm-up, gradually increase the learning rate, and finally gradually reduce it.

[0127] 4. Control the inference latency of the target model.

[0128] (1) Select the appropriate number of SENet layers: It is not necessary to apply SENet to every layer of the network. Generally, in object detection tasks, applying SENet layers to the backbone network is more effective. Because the backbone network is the core area for extracting features, applying the SENet channel attention mechanism module can improve the feature expression ability, while other parts (such as the detection head) usually do not require too much attention mechanism.

[0129] In YOLOv5s, we only need to add SENet channel attention mechanism modules to some convolutional layers of the backbone network instead of adding SENet channel attention mechanism modules to every convolutional layer. We select the optimal number of layers through experimental evaluation.

[0130] (2) Optimizing inference speed: By reducing the complexity of the network (such as reducing the number of SENet layers or using it only in important parts) and combining the aforementioned pruning and quantization, we ensure that after adding the SENet channel attention mechanism module, the network's inference delay can still be kept within the real-time requirements.

[0131] You can use benchmarking tools such as TensorRT or ONNX Runtime to test the impact of different optimization methods on inference latency and ultimately choose the appropriate structure.

[0132] 5. Data diversity and target model generalization ability.

[0133] (1) Image Augmentation: In order to improve the generalization ability of the target model, various data augmentation techniques can be used during the training process, such as random cropping, rotation, scaling, color jitter, etc., to simulate different scene changes. Especially after adding the SENet channel attention mechanism module, the target model may overfit to some small targets or low-contrast targets. Enhancing the diversity of data helps improve the robustness of the model.

[0134] Common data enhancement strategies such as MixUp or CutMix are added to the training data pipeline of YOLOv5s to synthesize training samples in different scenarios.

[0135] (2) Hard Example Mining: In order to prevent the target model from overfitting to difficult-to-detect targets (such as small targets or low-contrast targets), hard example mining technology can be used to focus on training difficult-to-detect targets during the training process. For example, the loss weights for these difficult targets can be increased to allow the target model to better learn their features.

[0136] The focal loss function (Focal Loss) or online hard example mining (OHEM) is used to increase the proportion of difficult and easy samples during training, thereby improving the generalization ability of the model.

[0137] The above strategies can maintain the efficiency of the original structure and avoid over-reliance on the channel attention mechanism.

[0138] In summary, this embodiment introduces the SENet channel attention mechanism at an important position of YOLOv5s, thereby enhancing the ability to extract key features of the target model, improving the ability to distinguish the importance between channels, and significantly improving the detection accuracy in complex scenes; by designing a reasonable parameter scaling ratio and module embedding position, it ensures that the target model performance is improved while maintaining the reasoning speed; through a variety of data enhancement strategies, the generalization ability of the target model is improved to adapt to the diverse needs of logistics scenarios; through pruning and quantization, the model size is further compressed to improve the reasoning efficiency on embedded devices; through the discarding method and L2 regularization, the complexity of the target model is controlled to avoid the overfitting problem after adding the attention mechanism. This embodiment can be applied to scenes with complex backgrounds, dense targets, and large scale changes. It has a small computational overhead and high computational efficiency while improving detection accuracy and accuracy.

[0139] See also Figure 7 The article sorting device based on SENet and YOLOv5s provided in this embodiment includes an image acquisition module 110 and a target recognition module 120. The image acquisition module 110 is used to acquire an article sorting mark image to be identified. The target recognition module 120 is used to input the article sorting mark image to be identified into a trained YOLOv5s target detection model based on the SENet attention mechanism for identification, and obtain the predicted category of the article sorting mark image to be identified. Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0140] Optionally, a SENet channel attention mechanism module is added after the residual block in the CSP module; wherein, the SENet channel attention mechanism module is added after the residual block of the last CSP module in the backbone network, or the SENet channel attention mechanism module is added after the residual blocks of the last two CSP modules in the backbone network.

[0141] Optionally, a SENet channel attention mechanism module is added after the last concatenated layer in the PAN structure, or a SENet channel attention mechanism module is added after the last two concatenated layers in the PAN layer structure.

[0142] See also Figure 8 The YOLOv5s target detection model training device based on SENet attention mechanism provided in this embodiment includes a model construction module 210, a training data acquisition module 220, a training data preprocessing module 230, a forward propagation module 240, a loss calculation module 250, a back propagation module 260 and a model determination module 270. The model construction module 210 is used to construct a YOLOv5s target detection model based on SENet attention mechanism to be trained and a loss function. The training data acquisition module 220 is used to obtain a training sample data set, wherein the training samples in the training sample data set include an item sorting mark image and label information of the item sorting mark image. The training data preprocessing module 230 is used to preprocess the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement. The forward propagation module 240 is used to input the training sample data set into the YOLOv5s target detection model based on SENet attention mechanism to be trained for forward propagation to obtain a training result. The loss calculation module 250 is used to input the training result and the label information into the loss function to obtain a loss result. The back propagation module 260 is used to perform back propagation using an optimizer according to the loss result, and adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges. The model determination module 270 is used to use the model obtained when the loss function converges as the YOLOv5s target detection model based on the SENet attention mechanism. Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

[0143] Optionally, a pruning and quantization module is also connected before or after the model determination module 270 to prune and quantize the YOLOv5s target detection model based on the SENet attention mechanism to obtain the pruned and quantized YOLOv5s target detection model based on the SENet attention mechanism.

[0144] Optionally, the item sorting mark images are item sorting mark images under various lighting conditions, various background complexities and various target types, and the data enhancement includes at least one of geometric enhancement, color enhancement and noise injection.

[0145] Optionally, the model building module 210 is also used to set the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism, and to construct a loss function based on classification loss, positioning loss and confidence loss; wherein the network parameters include optimizer parameters and learning rate, the optimizer is set to the AdamW optimizer, and the learning rate is set based on the cosine annealing method.

[0146] Optionally, the back propagation module 260 is also used to add a dropout layer after the SENet channel attention mechanism module, and / or adjust the loss function using L2 regularization to prevent overfitting.

[0147] The device provided in the embodiment of the present application is adopted. Since the device adopts the same inventive concept as the above-mentioned method provided in the embodiment of the present application, on the premise that the method can solve the technical problem, the device can also solve the technical problem, which will not be repeated here.

[0148] Reference Fig. 9 An embodiment of the present invention further provides an electronic device 400, including a communication interface 401, a processor 402, a memory 403 and a bus 404, wherein the processor 402, the communication interface 401 and the memory 403 are connected via the bus 404; the memory 403 is used to store a computer program that supports the processor 402 to execute the item sorting method based on SENet and YOLOv5s, or to execute a computer program that executes the YOLOv5s target detection model training method based on the SENet attention mechanism, and the processor 402 is configured to execute the program stored in the memory 403.

[0149] Optionally, an embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor 402, wherein the program code enables the processor 402 to execute an item sorting method based on SENet and YOLOv5s as in the above embodiment, or a YOLOv5s target detection model training method based on the SENet attention mechanism.

[0150] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. An object sorting method based on SENet and YOLOv5s, characterized in that: include: Obtaining a sorting mark image of an item to be identified; Input the to-be-recognized item sorting mark image into the trained YOLOv5s target detection model based on SENet attention mechanism for recognition, and obtain the predicted category of the to-be-recognized item sorting mark image; Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

2. The method for sorting items based on SENet and YOLOv5s according to claim 1, characterized in that: The step of adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model includes: The SENet channel attention mechanism module is added after the residual block in the CSP module; wherein, the SENet channel attention mechanism module is added after the residual block of the last CSP module in the backbone network, or the SENet channel attention mechanism module is added after the residual blocks of the last two CSP modules in the backbone network.

3. The object sorting method based on SENet and YOLOv5s according to claim 1, characterized in that: The SENet channel attention mechanism module is added to the PAN structure in the neck network of the YOLOv5s network model, including: Add the SENet channel attention mechanism module after the last concatenation layer in the PAN structure, or add the SENet channel attention mechanism module after the last two concatenation layers in the PAN layer structure.

4. A YOLOv5s target detection model training method based on SENet attention mechanism, characterized in that: include: Build the YOLOv5s target detection model and loss function based on SENet attention mechanism to be trained; Acquire a training sample data set, wherein the training samples in the training sample data set include item sorting mark images and label information of the item sorting mark images; Preprocessing the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement; Input the training sample data set into the YOLOv5s target detection model to be trained based on the SENet attention mechanism for forward propagation to obtain a training result; Inputting the training result and the label information into the loss function to obtain a loss result; According to the loss result, back propagation is performed using an optimizer to adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges; and The model obtained when the loss function converges is used as a YOLOv5s target detection model based on the SENet attention mechanism; Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

5. The YOLOv5s target detection model training method based on SENet attention mechanism according to claim 4, characterized in that: Before or after the step of using the model obtained when the loss function converges as the YOLOv5s target detection model based on the SENet attention mechanism, the method further includes: The YOLOv5s target detection model based on the SENet attention mechanism is pruned and quantized to obtain a pruned and quantized YOLOv5s target detection model based on the SENet attention mechanism.

6. The YOLOv5s target detection model training method based on SENet attention mechanism according to claim 4, characterized in that: The item sorting mark images are item sorting mark images under various lighting conditions, various background complexities and various target types, and the data enhancement includes at least one of geometric enhancement, color enhancement and noise injection.

7. The YOLOv5s target detection model training method based on SENet attention mechanism according to claim 4, characterized in that: The construction of the YOLOv5s target detection model and loss function based on the SENet attention mechanism to be trained includes: The network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism are set, and a loss function based on classification loss, positioning loss and confidence loss is constructed; wherein the network parameters include optimizer parameters and learning rate, the optimizer is set to the AdamW optimizer, and the learning rate is set based on the cosine annealing method.

8. The YOLOv5s target detection model training method based on SENet attention mechanism according to claim 4, characterized in that: The adjusting the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism includes: A dropout layer is added after the SENet channel attention mechanism module, and / or the loss function is adjusted using the L2 regularization method to prevent overfitting.

9. An item sorting device based on SENet and YOLOv5s, characterized in that: include: An image acquisition module, used to acquire the sorting mark image of the item to be identified; A target recognition module is used to input the item sorting mark image to be identified into a trained YOLOv5s target detection model based on SENet attention mechanism for identification, so as to obtain the predicted category of the item sorting mark image to be identified; Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

10. A YOLOv5s target detection model training device based on SENet attention mechanism, characterized in that: include: Model building module, used to build the YOLOv5s target detection model and loss function based on SENet attention mechanism to be trained; A training data acquisition module, used to acquire a training sample data set, wherein the training samples in the training sample data set include an item sorting mark image and label information of the item sorting mark image; A training data preprocessing module, used to preprocess the training sample data set to obtain a preprocessed training sample data set, wherein the preprocessing includes data enhancement; A forward propagation module is used to input the training sample data set into the YOLOv5s target detection model based on the SENet attention mechanism to be trained for forward propagation to obtain a training result; A loss calculation module, used for inputting the training result and the label information into the loss function to obtain a loss result; A back propagation module, used to perform back propagation using an optimizer according to the loss result, and adjust the network parameters of the YOLOv5s target detection model to be trained based on the SENet attention mechanism until the loss function converges; A model determination module, used to use the model obtained when the loss function converges as a YOLOv5s target detection model based on the SENet attention mechanism; Among them, the YOLOv5s target detection model based on the SENet attention mechanism is obtained by adding a SENet channel attention mechanism module to at least one CSP module in the backbone network of the YOLOv5s network model, and / or by adding a SENet channel attention mechanism module to the PAN structure in the neck network of the YOLOv5s network model.

Images