Target object shielding and stacking identification detection method based on YOLOv8 network

Abstract

The invention provides a target object shielding and stacking identification and detection method based on a YOLOv8 network. The method comprises the following steps: S11, constructing a data set; s12, improving a YOLOv8 network, wherein the improvement comprises the steps of replacing a Conv unit of a Backbone module with an AKConv unit, adding an ECA attention mechanism in front of an SPFF layer, replacing a C2f unit of a Neck module with a VoV-GSCSP unit, replacing the Conv unit with a GSConv unit, replacing a detection head of a Head module with a DBB re-parameter unit, and using a GIoU loss function; s13, the data set is input into the improved YOLOv8 network for training, and the weight of the improved YOLOv8 network after training is completed is obtained; and S14, inputting the pictures of the stacked objects into the improved YOLOv8 network for detection and evaluation. The detection precision and the feature extraction capability of the model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision target detection, and in particular to a target detection method for stacked occluded objects based on optimized and improved YOLOv8. Background Art

[0002] With the gradual development of computer vision, target detection algorithms have become a hot topic in the field of computer vision. Target detection algorithms are the basis of other advanced visual tasks, such as instance segmentation, target detection, and target tracking. In recent years, the rapid development of deep learning has promoted the vigorous development of target detection algorithms. As the average detection accuracy of target detection algorithms increases, problems also arise. In practical fields such as industry, such as robotics and autonomous driving, target detection algorithms need to be executed in a timely manner under limited resource conditions. The target detection accuracy is high, and the number of parameters and computational complexity of the entire algorithm will be higher. Therefore, it is necessary to lightweight the target detection algorithm. Lightweight processing refers to reducing the number of parameters and computational complexity of the algorithm through model pruning, structural optimization, and other methods in the convolutional neural network part, while retaining its average detection accuracy as much as possible. Therefore, it is very necessary to lightweight the target detection algorithm.

[0003] In recent years, image recognition technology based on deep learning has made significant progress, especially the application of target detection algorithms such as the YOLO series has gradually matured. These algorithms can accurately identify and locate targets in complex scenes through efficient neural network structures and large-scale data training. YOLOv8 is a target detection algorithm in the YOLO series of algorithms, which is famous for its simple and efficient design and detection speed. However, the generalization ability of YOLOv8 in some complex scenes needs to be improved, and there are still certain deficiencies in the detection of stacked occluded targets. Summary of the invention

[0004] The purpose of the present invention is to provide a target object occlusion stacking recognition and detection method based on the YOLOv8 network, aiming to solve the problem of low detection accuracy when the detected target object is occluded and stacked.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a target object occlusion stacking recognition and detection method based on YOLOv8 network, the method comprising: S11, take pictures of stacked objects, annotate the objects in the pictures, and build a data set; S12. Improve the YOLOv8 network to construct a Further_YOLOv8_F network, wherein the improvements include replacing the Conv unit of the Backbone module with an AKConv unit, adding an ECA attention mechanism before the SPFF layer, replacing the C2f unit of the Neck module with a VoV-GSCSP unit, and replacing the Conv unit with a GSConv unit, using the DBB re-parameter unit as the detection head of the Head module, and using the GIoU loss function; S13, input the data set into the Further_YOLOv8_F network for training, and obtain the trained Further_YOLOv8_F network weights; S14. Input the stacked object image into the Further_YOLOv8_F network for detection and evaluate the detection results.

[0006] Furthermore, in step S11, the target objects in the image are labeled, specifically including: separately labeling the objects with random occlusion stacking position relationship and the objects without occlusion stacking position relationship.

[0007] Furthermore, in the Backbone module, do the following: The AKConv unit obtains the offset of the convolution kernel of the corresponding data set through convolution operation, and adjusts the sampling position of convolution kernels of different sizes according to the offset; The ECA attention mechanism uses 1D convolution to perform channel attention calculations, determine the coverage of local cross-channel interactions, and extract features from the dataset.

[0008] Further, in step S12, the Backbone module includes a Conv unit, an AKConv unit, a first C2f unit, an AKConv unit, a second C2f unit, an AKConv unit, a third C2f unit, an AKConv unit, a fourth C2f unit, an ECA attention mechanism unit and an SPPF unit connected in sequence, and the second C2f unit, the third C2f unit and the SPPF unit are respectively connected to the Neck module.

[0009] Furthermore, perform the following operations in the Neck module: The VoV-GSCSP unit uses a one-shot aggregation method to fuse features from different stages of the feature map; The GSConv unit compresses the features in spatial dimension and expands the channel dimension, and evenly exchanges local feature information on different channels, so that the information of the standard 2D convolutional layer of the 2D batch normalization layer and the activation layer can be mixed into the output of the deep convolutional layer of the 2D batch normalization layer and the activation layer.

[0010] Further, in step S13, the Neck module includes an Upsample module, a Concat module, a VoV-GSCSP module, an Upsample module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module and a VoV-GSCSP module which are connected in sequence.

[0011] Furthermore, perform the following operations in the Head module: The DBB heavy parameter unit enriches the feature representation capability by introducing convolution branches of different sizes and structures; The GIoU loss function is On this basis, the minimum enclosing rectangle of the detection box and the real box is introduced to obtain the proportion of the detection box and the real box in the closure area. The expression of the GIoU loss function is:

[0012]

[0013]

[0014] in, is the ratio of the common part C between the real frame A and the detection frame B to the union, is the real frame, is the detection frame, C is the common part of the real frame and the detection frame, is the IoU loss function, is the GIoU loss function.

[0015] Furthermore, in the Head module, the Head module includes a first DBB unit, a second DBB unit, and a third DBB unit, and the Neck module is connected to the first DBB unit, the second DBB unit, and the third DBB unit, respectively.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a target object occlusion stacking recognition and detection method based on the YOLOv8 network. First, the backbone network is optimized, the AKConv convolution layer is replaced, and the ECA attention mechanism is added before the SPPF to maintain the network performance while significantly reducing the model complexity. The coverage of local cross-channel interaction is determined by adaptively selecting the convolution kernel size. The GSConv module introduced in Neck improves the reasoning calculation speed in the convolutional neural network, and the VoV-GSCSP module further improves the utilization efficiency of the extracted features and the network performance. When dealing with more complex structures, the DBB detection head keeps the macro architecture unchanged and extracts features faster. The use of the GIoU loss function is more suitable for dealing with crowded occasions where there are overlapping and non-overlapping targets, especially in the field of stacked material target detection. In summary, the improvement of the network structure of the present invention further enhances the network feature extraction capability and speed, and Further_YOLOv8_F can improve the accuracy and evaluation score of stacked occluded object recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work. Figure 1 A flowchart of a target object occlusion stacking recognition and detection method based on a YOLOv8 network provided in an embodiment of the present invention; Figure 2 A diagram of the Further_YOLOv8_F network structure provided by an embodiment of the present invention; Figure 3 The accuracy curve, recall curve, mAP50 curve and mAP50-90 curve of the YOLOv8 network; Figure 4 The accuracy curve, recall curve, mAP50 curve and mAP50-90 curve of the Further_YOLOv8_F network provided in the embodiment of the present invention; Figure 5 This is a visualization of the first experiment of the YOLOv8 network; Figure 6 A visualization diagram of the first experiment of the Further_YOLOv8_F network provided in an embodiment of the present invention; Figure 7 This is the visualization effect diagram of the second experiment of the YOLOv8 network; Figure 8A visualization diagram of the second experiment of the Further_YOLOv8_F network provided in an embodiment of the present invention; Fig. 9 This is a comparison table of the accuracy, recall, average precision 50% and average precision 50-95% of the YOLOV8 network and the Further_YOLOv8_F network. DETAILED DESCRIPTION

[0018] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0019] Reference Figure 1 and Figure 2 This embodiment provides a target object occlusion stacking recognition and detection method based on a YOLOv8 network, the method comprising: S11. Take pictures of stacked objects, annotate the target objects in the pictures, and construct a data set. In this embodiment, in order to enrich the data set and improve the generalization ability of the network model to detect objects, the data set is expanded by performing data enhancement processing such as translation, rotation, flipping, cropping, changing brightness and Gaussian noise on the image. The preprocessed data set is divided into a data set, a validation set, and a test set. For example, 274 images are constructed as a data set and preprocessed, and the preprocessed data set is divided into a data set train, a validation set val, and a test set test in a ratio of 88:7:5.

[0020] In step S11, the target objects in the image are labeled, specifically including: labeling the objects with random occlusion stacking position relationship and objects without occlusion stacking position relationship respectively. In this embodiment, a labeling tool is used to label the objects with random occlusion stacking position relationship and objects without occlusion stacking position relationship. For example, the Roboflow labeling tool is used to label six objects with random occlusion stacking position relationship and objects without occlusion stacking position relationship. The target object labeling labels are divided into two cubes, two cuboids and two cylinders.

[0021] S12. Improve the YOLOv8 network to construct the Further_YOLOv8_F network. The improvements include replacing the Conv unit of the Backbone module with the AKConv unit, adding the ECA attention mechanism before the SPFF layer, replacing the C2f unit of the Neck module with the VoV-GSCSP unit, and replacing the Conv unit with the GSConv unit, using the DBB re-parameter unit as the detection head of the Head module, and using the GIoU loss function.

[0022] In this embodiment, the YOLOv8 network is divided into four parts: input, backbone network, neck network and head network. The input end mainly includes mosaic data enhancement, adaptive anchor frame calculation and adaptive grayscale filling. The backbone network mainly includes Conv, C2f and SPPF (spatial pyramid pooling fusion) structures, and the neck network adopts the path aggregation network (PAN) structure, which can enhance the network's feature fusion ability for objects of different scales. The head network decouples the classification and detection processes, mainly including loss calculation and target detection box screening. The loss calculation process mainly includes positive and negative sample allocation strategies and Loss calculation, for example: Task Aligned Assigner allocation strategy, that is, selecting positive samples based on the weighted results of classification and regression scores; loss calculation covers two branches, classification and regression. The classification branch uses binary cross entropy loss (BCE Loss) for training; the regression branch combines distribution focal loss (DF Loss) and complete intersection over union loss (CIOU Loss) to improve the accuracy of the model for bounding box detection.

[0023] In your Backbone module do the following: The AKConv unit obtains the offset of the convolution kernel of the corresponding data set through convolution operation, and adjusts the sampling position of convolution kernels of different sizes according to the offset.

[0024] In this embodiment, the replaced AKConv module allows the convolution kernel to have any number of parameters, which allows it to adjust its size and shape according to actual needs and adapt to target changes more effectively; for convolution kernels of different sizes, AKConv adjusts the sampling position of the irregular convolution kernel by obtaining the offset to improve the accuracy of feature extraction; AKConv supports linear increase or decrease in the number of convolution parameters, which helps to optimize performance in a hardware environment and is particularly suitable for the use of lightweight models.

[0025] The ECA attention mechanism uses 1D convolution to perform channel attention calculations, determine the coverage of local cross-channel interactions, and extract features from the dataset.

[0026] In this embodiment, the added ECA attention mechanism avoids the dimensionality reduction operation in the channel attention module. By taking local cross-channel interactions and using 1D convolution to achieve efficient channel attention calculation, the complexity of the model is reduced while maintaining performance. The coverage of local cross-channel interactions is determined by adaptively selecting the convolution kernel size.

[0027] The ECA attention mechanism is an improvement based on the SE attention mechanism. SE attention is divided into three steps. One is Squeeze, which is used to compress the feature maps on different channels through global average pooling. The other is Excitation, which assigns weights to the previous compression result. The number of weights is the same as the input compression result. Finally, the weight number is multiplied by the corresponding channel through Scale. The different colors in the figure represent different weights. Although the model results are good, the parameter calculation is inevitably improved. The SE attention method has side effects, which is manifested in the loss of information during dimensionality reduction, and the calculation of the weight information of all channels leads to low efficiency. Therefore, the ECA attention mechanism removes the fully connected layer after using global domain average pooling and uses 1*1 convolution instead, which realizes cross-channel information flow interaction and prevents dimensionality reduction. The model after using the ECA attention mechanism has significantly improved performance, and its computational complexity is far less than the SE attention method, making it effectively a low-complexity and efficient attention model that can be implanted in any stage.

[0028] In step S12, the Backbone module includes a Conv unit, an AKConv unit, a first C2f unit, an AKConv unit, a second C2f unit, an AKConv unit, a third C2f unit, an AKConv unit, a fourth C2f unit, an ECA attention mechanism unit and an SPPF unit connected in sequence, and the second C2f unit, the third C2f unit and the SPPF unit are respectively connected to the Neck module.

[0029] In the Neck module do the following: The VoV-GSCSP unit uses a one-shot aggregation method to fuse features from different stages of the feature map.

[0030] The GSConv unit compresses the features in spatial dimension and expands the channel dimension, and evenly exchanges local feature information on different channels, so that the information of the standard 2D convolutional layer of the 2D batch normalization layer and the activation layer can be mixed into the output of the deep convolutional layer of the 2D batch normalization layer and the activation layer.

[0031] In this embodiment, the Neck of YOLOv8 is optimized to Slim-Neck, that is, the C2f module is replaced by the VoV-GSCSP module on the basis of the neck, and the Conv module is replaced by the GSConv module. Among them, the GSConv module is a lightweight convolution that reduces computational complexity. Its computational cost is about half of that of the standard convolution. The GSConv module solves the speed problem of detection calculation in the convolutional neural network. The VoV-GSCSP module uses a cross-stage partial network module designed by a one-time aggregation method to perform effective information fusion between feature maps of different stages.

[0032] In order to speed up the final detection, the input image needs to go through a similar transformation process in the backbone network, that is, gradually transferring spatial information to channels. Each compression of the spatial dimension (width and height) of the feature and the expansion of the channel dimension will lead to the loss of some semantic information. Channel-intensive convolution maximizes the retention of hidden connections between each channel, while channel-sparse convolution completely cuts off these connections. GSConv has a low time complexity while preserving these connections as much as possible. The features generated by channel-intensive convolution are permeated into every part of the features generated by DSC depthwise separable convolution using the shuffle operation. The shuffle operation is a uniform mixing, which enables the information of channel-intensive convolution to be fully mixed into the output of depthwise separable convolution by uniformly exchanging local feature information on different channels. Based on GSConv, the grouped spatial convolution bottleneck structure is continued to be introduced, and a single aggregation strategy is adopted to design an efficient cross-stage partial network module, namely VoV-GSCSP, to reduce computational complexity and inference time while maintaining accuracy.

[0033] In step S13, the Neck module includes an Upsample module, a Concat module, a VoV-GSCSP module, an Upsample module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module and a VoV-GSCSP module which are connected in sequence.

[0034] In the Head module do the following: The DBB re-parameterization unit enriches the feature representation capability by introducing convolution branches of different sizes and structures. In this embodiment, in order to improve usability and versatility, the basic convolutional neural network (ConvNet) component K*Kconv is upgraded, namely the Diverse Branch Block (DBB). DBB can be equivalently converted to a conventional convolutional layer for deployment, so that the microstructure during training can be complicated without affecting the macro architecture (e.g., ResNet) or the structure during reasoning. The basic principle of the DBB re-parameterization module replaced by the detection head Head to enrich the feature representation capability of the network is: diversified branch structure, DBB combines branches of different scales and complexities, such as convolution kernels of different sizes and average pooling, to increase the feature expression capability of a single convolution; training and reasoning separation, in the training stage, DBB adopts a branch structure, and in the reasoning stage these branches can be equivalently converted to a single convolution layer to maintain efficient reasoning; the macro architecture remains unchanged, DBB allows it to be inserted into the existing network as a substitute for the conventional convolution layer without changing the overall network architecture.

[0035] The GIoU loss function is On the basis of , the minimum enclosing rectangle of the detection box and the real box is introduced to obtain the proportion of the detection box and the real box in the closure area. The expression of the GIoU loss function is:

[0036]

[0037]

[0038] in, is the ratio of the common part C between the real frame A and the detection frame B to the union, is the real frame, is the detection frame, C is the common part of the real frame and the detection frame, is the IoU loss function, is the GIoU loss function.

[0039] In this embodiment, the GIoU loss function used considers non-overlapping areas on the basis of IoU to more comprehensively evaluate the bounding box. IoU is used to evaluate the relationship between the result of model reasoning and the target. IoU quantifies the accuracy of algorithm detection by calculating the proportion of common components between the two. GIoU is introduced to solve the problem that IoU cannot reflect how two boxes intersect. It obtains the proportion of the detection box and the real box in the closure area by introducing the minimum enclosing rectangle of the detection box and the real box. GIoU adds attention to non-common areas on the basis of IoU, which can better reflect the intersection of the two boxes in the closure area.

[0040] In the Head module, the Head module includes a first DBB unit, a second DBB unit, and a third DBB unit, and the Neck module is connected to the first DBB unit, the second DBB unit, and the third DBB unit, respectively.

[0041] S13, input the data set into the Further_YOLOv8_F network for training, and obtain the trained Further_YOLOv8_F network weights. In this embodiment, the improved Further_YOLOv8_F can improve the accuracy and evaluation score of stacked occluded objects recognition.

[0042] In this embodiment, the experimental environment is: win11; memory 16GB; Python version: 3.9.19; torch version: 1.9.1; graphics card: NVIDIA GeForce RTX 4060 Laptop GPU; video memory: 8188MiB, the data set is input into the network improved based on the YOLOv8 network, and the Further_YOLOv8_F network weights are obtained by training. The accuracy curve, recall curve, mAP50 curve and mAP50-90 curve of the YOLOv8 network training before improvement are shown in Figure 3 As shown in the figure, the accuracy curve, recall curve, mAP50 curve and mAP50-90 curve of the improved Further_YOLOv8_F network training are as follows Figure 4 shown.

[0043] S14. Input the stacked object image into the Further_YOLOv8_F network for detection and evaluate the detection results.

[0044] like Figure 5~Figure 8 As shown in FIG. 1 , in this embodiment, the stacked object image is input into the Further_YOLOv8_F network for detection, and the detection result is evaluated. Figure 5~Figure 8 The comparison diagrams of the first experiment and the second experiment of inputting stacked object images into the YOLOv8 network and the Further_YOLOv8_F network for detection are respectively given. The experiments verify that the present invention can ultimately perform higher-precision detection on the input images.

[0045] Reference Fig. 9 ,From the experimental results analysis, the improved model has 2.984% improvement in accuracy, 1.57% improvement in recall, 2.789% improvement in average precision 50%, and 2.173% improvement in mAP50-95 average precision 50-95%, which is generally better than the ,pre-improvement model.

[0046] Fig. 9middle: Precision: Indicates how many of the samples detected as positive are correct, Precision = TP / (TP+FP).

[0047] Recall: Indicates how many truly positive samples are correctly detected as positive, Recall = TP / (TP+FN).

[0048] Among them, TP means True Positive, FP means False Positive, and FN means False Negative.

[0049] Specifically, the Further_YOLOv8_F network model uses the constructed dataset to train, gradually verify and test the improved detection network. This method can ultimately achieve higher-precision detection of the input images.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A target object occlusion stacking recognition and detection method based on YOLOv8 network, characterized in that: The method comprises: S11, take pictures of stacked objects, annotate the objects in the pictures, and build a data set; S12. Improve the YOLOv8 network to construct a Further_YOLOv8_F network, wherein the improvements include replacing the Conv unit of the Backbone module with an AKConv unit, adding an ECA attention mechanism before the SPFF layer, replacing the C2f unit of the Neck module with a VoV-GSCSP unit, and replacing the Conv unit with a GSConv unit, using the DBB re-parameter unit as the detection head of the Head module, and using the GIoU loss function; S13, input the data set into the Further_YOLOv8_F network for training, and obtain the trained Further_YOLOv8_F network weights; S14. Input the stacked object image into the Further_YOLOv8_F network for detection and evaluate the detection results.

2. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network in claim 1, it is characterized in that: In step S11, the target objects in the image are labeled, specifically including: labeling the objects with random occlusion stacking position relationship and the objects without occlusion stacking position relationship respectively.

3. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network in claim 1, it is characterized in that: In your Backbone module do the following: The AKConv unit obtains the offset of the convolution kernel of the corresponding data set through convolution operation, and adjusts the sampling position of convolution kernels of different sizes according to the offset; The ECA attention mechanism uses 1D convolution to perform channel attention calculations, determine the coverage of local cross-channel interactions, and extract features from the dataset.

4. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network according to claim 1, it is characterized in that: In step S12, the Backbone module includes a Conv unit, an AKConv unit, a first C2f unit, an AKConv unit, a second C2f unit, an AKConv unit, a third C2f unit, an AKConv unit, a fourth C2f unit, an ECA attention mechanism unit and an SPPF unit connected in sequence, and the second C2f unit, the third C2f unit and the SPPF unit are respectively connected to the Neck module.

5. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network according to claim 1, it is characterized in that: In the Neck module do the following: The VoV-GSCSP unit uses a one-shot aggregation method to fuse features from different stages of the feature map; The GSConv unit compresses the features in spatial dimension and expands the channel dimension, and evenly exchanges local feature information on different channels, so that the information of the standard 2D convolutional layer of the 2D batch normalization layer and the activation layer can be mixed into the output of the deep convolutional layer of the 2D batch normalization layer and the activation layer.

6. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network according to claim 1, it is characterized in that: In step S13, the Neck module includes an Upsample module, a Concat module, a VoV-GSCSP module, an Upsample module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module, a VoV-GSCSP module, a GSConv module, a Concat module and a VoV-GSCSP module which are connected in sequence.

7. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network according to claim 1, it is characterized in that: In the Head module do the following: The DBB heavy parameter unit enriches the feature representation capability by introducing convolution branches of different sizes and structures; The GIoU loss function is On this basis, the minimum enclosing rectangle of the detection box and the real box is introduced to obtain the proportion of the detection box and the real box in the closure area. The expression of the GIoU loss function is: in, is the ratio of the common part C between the real frame A and the detection frame B to the union, is the real frame, is the detection frame, C is the common part of the real frame and the detection frame, is the IoU loss function, is the GIoU loss function.

8. According to the target object occlusion stacking recognition and detection method based on the YOLOv8 network according to claim 1, it is characterized in that: In the Head module, the Head module includes a first DBB unit, a second DBB unit, and a third DBB unit, and the Neck module is connected to the first DBB unit, the second DBB unit, and the third DBB unit, respectively.

Images