WPIoU loss function construction method and YOLOv8 detection method

Abstract

The WPIoU loss function construction method and the YOLOv8 detection method belong to the field of image information detection technology, and solve the problem that the existing YOLOv8n cannot realize accurate detection of traffic images in complex environments. The following steps are included: Step S1, after processing the WIoUv1 loss function with gradient gain, the WIoUv3 loss function is obtained; Step S2, the PIoU loss function customizes an adaptive penalty factor; Step S3, after improving the WIoUv3 loss function described in Step S1 based on the penalty factor described in Step S2, the WPIoU loss function is obtained; Step S4, after further improving the WPIoU loss function described in Step S3 based on the non-monotonic attention function of the PIOUv2 loss function, the final WPIoU loss function is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of image information detection, and in particular to a WPIoU loss function construction method and a YOLOv8 detection method. Background Art

[0002] The development of detection methods for vehicle flow image information has effectively solved problems such as traffic congestion, traffic safety, and the growth of transportation demand, enabling traffic management departments to formulate and implement traffic management policies more scientifically and provide citizens with a smoother, safer, and more environmentally friendly travel environment.

[0003] At present, the detection methods for traffic images mostly use deep learning methods. For example, deep convolutional neural networks are used to extract features from traffic images and perform target detection, classification and tracking, which has high-precision detection and classification capabilities and can handle vehicle identification in complex environments. In addition, YOLOv8 (You Only Look Once Version 8) can efficiently detect the position of vehicles from traffic images due to its real-time, high-precision and flexibility, and is also widely used in the detection of traffic image information.

[0004] In the prior art, Chinese patent CN117593623A discloses a "lightweight vehicle detection method based on an improved YOLOv8n (You Only Look Once Version 8n) model", which improves the YOLOv8n model, introduces lightweight GSConv (group convolution) and VOVGSCSP (cross-level partial network), designs a new high-speed detection head, adds an attention mechanism, and designs multi-scale convolution blocks to increase the generalization ability and robustness of the model. While improving accuracy, it greatly reduces the number of parameters and FLOPs (floating point operations) and improves speed. However, in the actual detection task of traffic images, there are problems such as dense small targets, multi-size changes, and complex weather background interference. The improved YOLOv8n of this technical solution cannot achieve accurate detection of traffic images in complex environments.

[0005] In summary, the existing YOLOv8n cannot achieve accurate detection of traffic images in complex environments. Summary of the invention

[0006] The present invention solves the problem that the existing YOLOv8n cannot accurately detect vehicle flow images in complex environments.

[0007] The WPIoU loss function construction method of the present invention comprises the following steps:

[0008] Step S1, after performing gradient gain processing on the WIoUv1 loss function, obtain the WIoUv3 loss function;

[0009] Step S2, the PIoU loss function customizes an adaptive penalty factor;

[0010] Step S3, after improving the WIoUv3 loss function described in step S1 based on the penalty factor described in step S2, a WPIoU loss function is obtained;

[0011] Step S4, after further improving the WPIoU loss function described in step S3 based on the non-monotonic attention function of the PIOUv2 loss function, the final WPIoU loss function is obtained.

[0012] Furthermore, in one embodiment of the present invention, in step S1, the WIoUv3 loss function is:

[0013] ;

[0014] in, is the gradient gain, is the WIoUv3 loss function, is the WIoUv1 loss function, is the loss function.

[0015] Furthermore, in one embodiment of the present invention, the WIoUv1 loss function is:

[0016] ;

[0017] in, is the WIoUv1 loss function, assuming The corresponding position in the target frame is , is the width of the annotation box, is the height of the annotation box, Represents a callout box.

[0018] Further, in one embodiment of the present invention, the for:

[0019] ;

[0020] in, is the gradient gain, and are all hyperparameters, is the outlier degree.

[0021] Further, in one embodiment of the present invention, in step S3, the WPIoU loss function is:

[0022] ;

[0023] in, is the WPIoU loss function, is the WIoUv3 loss function, is the penalty factor.

[0024] Further, in one embodiment of the present invention, in step S4, the final WPIoU loss function is:

[0025] ;

[0026] in, is the final WPIoU loss function, is the WPIoU loss function, is the penalty factor, is a single hyperparameter, is a non-monotonic attention function.

[0027] The detection method based on the improved YOLOv8 traffic image information of the present invention is implemented based on the WPIoU loss function construction method described in the above method, specifically:

[0028] The traffic image is sequentially input into the YOLOv8 backbone network, YOLOv8 neck network, and YOLOv8 head network to generate prediction results;

[0029] The prediction results are calculated through the WPIoU loss function and the detected traffic information is output.

[0030] Furthermore, in one embodiment of the present invention, the traffic image is sequentially input into the backbone network of YOLOv8, the neck network of YOLOv8 and the head network of YOLOv8 to generate prediction results, specifically:

[0031] The traffic image is input into the backbone network of YOLOv8 and then reduced in dimension by the Conv layer. The reduced-dimensional traffic image is then input into multiple cascaded C2f_CAST layers and Conv layers.

[0032] In each C2f_CAST layer, a part of the reduced-dimensional traffic flow images remains as is, and another part of the reduced-dimensional traffic flow images is input into multiple cascaded STARMLCA layers;

[0033] In the STARMLCA layer, the other part of the traffic flow images with reduced dimensionality is passed through DW-Conv to keep the spatial dimension unchanged. The other part of the traffic flow images with reduced dimensionality after the DW-Conv layer is divided into two paths, and the number of channels is expanded by GSConv respectively. The two parts of the traffic flow images with reduced dimensionality after GSConv are element-wise multiplied to output feature maps. The feature maps are sequentially passed through MLCA, GSConv and DW-Conv for feature extraction. The traffic flow images after feature extraction are element-wise added with the part of the traffic flow images with reduced dimensionality that remain the same.

[0034] The traffic image after multiple cascaded C2f_CAST layers and Conv layers is subjected to feature extraction again;

[0035] The vehicle flow image after feature extraction is input into the neck network of YOLOv8 for feature fusion, and the fused feature map is output;

[0036] The fused feature map is input into the head network of YOLOv8 for prediction to generate the prediction result.

[0037] An electronic device described in the present invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0038] Memory, used to store computer programs;

[0039] The processor is used to implement the method described in the above method when executing the program stored in the memory.

[0040] The present invention provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method described in the above method is implemented.

[0041] The present invention solves the problem that the existing YOLOv8n cannot accurately detect traffic images in complex environments. Specific beneficial effects include:

[0042] 1. The WPIoU loss function construction method described in the present invention. The existing IoU loss function is affected by unreasonable penalty factors, which causes the anchor box to expand during the regression process, significantly slows down the convergence speed, and produces the technical problem of anchor box enlargement. In order to solve the above technical problems, the present invention is based on the WIoUv1 loss function as a model, adds a gradient gain distribution strategy, and combines an adaptive penalty factor and a gradient adjustment function based on the quality of the anchor box to obtain the final WPIoU loss function, which effectively solves the problem of anchor box enlargement and slow convergence during the regression process, and enhances the focus on medium and high-quality anchor boxes;

[0043] 2. The detection method based on the improved YOLOv8 vehicle flow image information of the present invention improves the average accuracy, reduces the number of parameters and the complexity of calculation, and balances the detection performance and speed;

[0044] 3. The detection method of vehicle flow image information based on the improved YOLOv8 described in the present invention solves the problem that the existing YOLOv8n cannot realize the accurate detection of vehicle flow images in complex environments. In order to solve the above technical problems, the present invention detects vehicle flow image information based on the improved YOLOv8. First, the backbone network of the improved YOLOv8 designs the STARMLCA layer, and interacts and transforms the features from different subspaces through the star operation, and introduces MLCA to strengthen the attention to the significant semantic information of the occluded target. Secondly, the neck network of the improved YOLOv8 is reconstructed, and VOVGSCSP is designed while using GSConv to improve the detection efficiency. Furthermore, the head network of the improved YOLOv8 adds the SDFM layer, self-calibrates the shallow feature map, and integrates the deep semantic information. Finally, the WPIoU loss function is designed. The experimental results show that the improved YOLOv8 can realize the accurate detection of vehicle flow images in complex environments;

[0045] The detection method of vehicle flow image information based on the improved YOLOv8 described in the present invention can be applied to various scenarios to realize accurate detection of vehicle flow image information. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0047] Figure 1 It is a structural diagram of the improved YOLOv8 described in Implementation Method 3;

[0048] Figure 2 It is the C2f_CAST layer diagram described in the third embodiment;

[0049] Figure 3 It is the Detect graph described in the third implementation mode;

[0050] Figure 4 It is the STARMLCA layer diagram described in the third embodiment;

[0051] Figure 5 is the MLCA diagram described in Embodiment 3;

[0052] Figure 6 is the GSConv graph described in Embodiment 3;

[0053] Figure 7is the VOVGSCSP graph described in the third embodiment;

[0054] Figure 8 is the SDFM layer diagram described in the third implementation mode;

[0055] Fig. 9 is a schematic diagram of a predicted frame and a real frame according to the first embodiment;

[0056] Fig.10 This is a detection effect diagram of a traffic image using the original YOLOv8 as described in Implementation Method 2;

[0057] Fig.11 This is a detection effect diagram of a traffic image using the improved YOLOv8 described in Implementation Method 2;

[0058] Fig.12 This is a detection effect diagram of a traffic image using the original YOLOv8 as described in Implementation Method 2;

[0059] Fig.13 This is a detection effect diagram of a traffic image using the improved YOLOv8 described in Implementation Method 2;

[0060] Fig.14 This is a detection effect diagram of a traffic image using the original YOLOv8 as described in Implementation Method 2;

[0061] Fig.15 This is a detection effect diagram of a traffic image using the improved YOLOv8 described in Implementation Method 2;

[0062] Fig.16 This is a detection effect diagram of a traffic image using the original YOLOv8 as described in Implementation Method 2;

[0063] Fig.17 This is a detection effect diagram of a traffic image using the improved YOLOv8 described in Implementation Method 2;

[0064] Fig.18 It is a comparison chart of the experimental effects described in Implementation Method 3. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe various embodiments of the present invention in conjunction with the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0066] Implementation method 1: The WPIoU loss function construction method described in this implementation method includes the following steps:

[0067] Step S1, after performing gradient gain processing on the WIoUv1 loss function, obtain the WIoUv3 loss function;

[0068] Step S2, the PIoU loss function customizes an adaptive penalty factor;

[0069] Step S3, after improving the WIoUv3 loss function described in step S1 based on the penalty factor described in step S2, a WPIoU loss function is obtained;

[0070] Step S4, after further improving the WPIoU loss function described in step S3 based on the non-monotonic attention function of the PIOUv2 loss function, the final WPIoU loss function is obtained.

[0071] In this implementation, the WIoUv3 loss function is:

[0072] ;

[0073] in, is the gradient gain, is the WIoUv3 loss function, is the WIoUv1 loss function, is the loss function.

[0074] In this implementation, the WIoUv1 loss function is:

[0075] ;

[0076] in, is the WIoUv1 loss function, assuming The corresponding position in the target frame is , is the width of the annotation box, is the height of the annotation box, Represents a callout box.

[0077] In this embodiment, the for:

[0078] ;

[0079] in, is the gradient gain, and are all hyperparameters, is the outlier degree.

[0080] In this implementation, in step S3, the WPIoU loss function is:

[0081] ;

[0082] in, is the WPIoU loss function, is the WIoUv3 loss function, is the penalty factor.

[0083] In this implementation, in step S4, the final WPIoU loss function is:

[0084] ;

[0085] in, is the final WPIoU loss function, is the WPIoU loss function, is the penalty factor, is a single hyperparameter, is a non-monotonic attention function.

[0086] In the prior art, since the IoU loss function (intersection over union loss function) is affected by unreasonable penalty factors, the anchor box expands during the regression process, which significantly slows down the convergence speed and causes the technical problem of anchor box enlargement.

[0087] In order to solve the above technical problems, this implementation method designs a WPIoU loss function (weighted multi-intersection-over-union loss function). The WPIoU loss function is based on the WIoU loss function (weighted intersection-over-union loss function) as the loss function of the model, that is, the WPIoU loss function is an improvement based on the WIoU loss function. It includes the following steps:

[0088] Step S1, after performing gradient gain processing on the WIoUv1 loss function (weighted intersection-over-union v1 loss function), the WIoUv3 loss function (weighted intersection-over-union v3 loss function) is obtained, and its calculation formula is:

[0089] ;

[0090] in, is the gradient gain, is the WIoUv3 loss function, is the WIoUv1 loss function, is the loss function.

[0091] ;

[0092] in, is the WIoUv1 loss function, assuming The corresponding position in the target frame is , is the width of the annotation box, is the height of the annotation box, represents the annotation box. To prevent the generation of gradients that hinder convergence, and Separate from the gradient computation (superscript * denotes this operation).

[0093] ;

[0094] in, is the gradient gain, and are all hyperparameters, is the outlier degree. DAIR-V2X-I (dataset) has a large difference in the evaluation of ordinary anchor boxes. and The original coefficients cannot achieve the best effect in this implementation. and The adjustment of is closely related to the proportion of small targets in DAIR-V2X-I, so after comparing the experimental data, when Take 1.7, When the loss value is 2.7, the improved WIoUv3 loss function achieves the best results on DAIR-V2X-I.

[0095] Step S2, the PIoU loss function (polynomial intersection-over-union loss function) defines an adaptive penalty factor ,in, is the penalty factor, and the schematic diagram of the predicted box and the true box is as follows Fig. 9 As shown, , , and are the absolute values ​​of the distances between the edges of the predicted box and the true box. and They are the absolute values ​​of the distances between the corresponding edges of the predicted box and the true box.

[0096] Step S3, based on the gradient adjustment function of the anchor box quality of the PIoU loss function, the WPIoU loss function is improved, and the obtained WPIoU loss function is as follows , which effectively solves the problem of anchor box enlargement during the regression process.

[0097] Step S4, PIOUv2 loss function (weighted polynomial intersection-over-union v2 loss function) is calculated by a single hyperparameter Controlled non-monotonic attention function ,in, It is set to 1.3, which significantly enhances its ability to focus on medium-quality anchor boxes. The WPIoU loss function is further improved to obtain the final WPIoU loss function: ,in, is the final WPIoU loss function.

[0098] Therefore, based on the WIoUv1 loss function as a model, the allocation strategy of increasing the gradient gain solves the regression problem of small and low-quality annotation boxes, and combines the adaptive penalty factor and the gradient adjustment function based on the quality of the anchor box to obtain the final WPIoU loss function, which effectively solves the problem of anchor box enlargement and slow convergence during the regression process, and enhances the focus on medium and high-quality anchor boxes.

[0099] Implementation method 2: The detection method based on the improved YOLOv8 traffic image information described in this implementation method is implemented based on the WPIoU loss function construction method described in implementation method 1, specifically:

[0100] The traffic image is sequentially input into the YOLOv8 backbone network, YOLOv8 neck network, and YOLOv8 head network to generate prediction results;

[0101] The prediction results are calculated through the WPIoU loss function and the detected traffic information is output.

[0102] In the prior art, YOLOv8n is used to detect traffic images. However, in the actual detection task of traffic images, there are complex environments such as dense small targets, multi-size changes, and complex weather background interference. The existing YOLOv8n cannot achieve accurate detection of traffic images.

[0103] In order to solve the above technical problems, this embodiment designs a detection method based on the improved YOLOv8 traffic image information. The traffic image is generated through the YOLOv8 backbone network, YOLOv8 neck network and YOLOv8 head network to generate prediction results. These prediction results are then calculated through the WPIoU loss function and post-processing operations such as NMS (non-maximum suppression), and finally the detected traffic information is output.

[0104] In order to better illustrate the detection method based on the improved YOLOv8 traffic image information described in this embodiment, a detailed description is given through the following examples:

[0105] In simple scenes and large target vehicle detection, such as Fig.10 The detection effect of the original YOLOv8 on the traffic image is shown in the figure. Fig.11 Compared with the detection effect diagram of the vehicle flow image using the improved YOLOv8 of this embodiment, it can be seen that the detection effect of the vehicle flow image using the improved YOLOv8 of this embodiment is basically the same as that of the original YOLOv8. However, for blurred target vehicles with severe occlusion and dense distribution, such as Fig.12The detection effect of the original YOLOv8 on the traffic image is shown in the figure. Fig.13 The comparison of the detection effect of the improved YOLOv8 of this embodiment on the traffic image shows that in a rainy environment, the improved YOLOv8 of this embodiment can detect severely obstructed vehicles without causing any false detection, while the original YOLOv8 cannot. Fig.14 The detection effect of the original YOLOv8 on the traffic image is shown in the figure. Fig.15 The comparison of the detection effect of the vehicle flow image by the improved YOLOv8 using this embodiment shows that the improved YOLOv8 using this embodiment can detect vehicles that are not detected by the original YOLOv8 in a night environment, and has better adaptability in a night environment. Fig.16 The detection effect of the original YOLOv8 on the traffic image is shown in the figure. Fig.17 From the comparison of the detection effect diagram of the traffic image of the improved YOLOv8 using the present embodiment, it can be seen that the original YOLOv8 fails to detect small target vehicles and motorcycles occluded in the distance, while the improved YOLOv8 using the present embodiment can detect occluded targets and small targets in densely populated areas, and has better generalization ability for complex scenes.

[0106] In summary, using the improved YOLOv8 described in this embodiment to improve traffic images can solve the problem that the existing YOLOv8n cannot achieve accurate detection of traffic images.

[0107] Implementation method 3: This implementation method further limits the detection method based on the improved YOLOv8 traffic image information described in implementation method 2. The traffic image is sequentially input into the backbone network of YOLOv8, the neck network of YOLOv8, and the head network of YOLOv8 to generate prediction results, specifically:

[0108] The traffic image is input into the backbone network of YOLOv8 and then reduced in dimension by the Conv layer. The reduced-dimensional traffic image is then input into multiple cascaded C2f_CAST layers and Conv layers.

[0109] In each C2f_CAST layer, a part of the reduced-dimensional traffic flow images remains as is, and another part of the reduced-dimensional traffic flow images is input into multiple cascaded STARMLCA layers;

[0110] In the STARMLCA layer, the other part of the traffic flow images with reduced dimensionality is passed through DW-Conv to keep the spatial dimension unchanged. The other part of the traffic flow images with reduced dimensionality after the DW-Conv layer is divided into two paths, and the number of channels is expanded by GSConv respectively. The two parts of the traffic flow images with reduced dimensionality after GSConv are element-wise multiplied to output feature maps. The feature maps are sequentially passed through MLCA, GSConv and DW-Conv for feature extraction. The traffic flow images after feature extraction are element-wise added with the part of the traffic flow images with reduced dimensionality that remain the same.

[0111] The traffic image after multiple cascaded C2f_CAST layers and Conv layers is subjected to feature extraction again;

[0112] The vehicle flow image after feature extraction is input into the neck network of YOLOv8 for feature fusion, and the fused feature map is output;

[0113] The fused feature map is input into the head network of YOLOv8 for prediction to generate the prediction result.

[0114] In this embodiment, if Figure 1 As shown in the figure, an improved YOLOv8 is designed, specifically:

[0115] First, in the backbone network of YOLOv8, the processed traffic image is input into two Conv layers (convolutional layers) at 640*640 to reduce the dimension of the traffic image. After these two Conv layers, the size of the traffic image is reduced to 160*160, and the number of channels is increased to 128, that is, the traffic image becomes 160*160*128. Subsequently, the traffic image is input into four cascaded C2f_CAST layers (feature decomposition layers) and Conv layers.

[0116] like Figure 2 As shown in Figure 1, within each C2f_CAST layer, the input feature map is first split into two parts. One part remains as is, while the other part is passed through n cascaded STARMLCA layers (feature interaction layers) for feature extraction.

[0117] like Figure 4As shown in the figure, in the STARMLCA layer, a DW-Conv (depthwise separable convolution) is used. DW-Conv applies convolution kernels to each channel of the input independently. The convolution kernel size is 7x7, the step size is 1, and the spatial dimension is kept unchanged. The feature map of 160*160*128 is output. Then the feature map of 160*160*128 is divided into two paths, both of which pass through 1x1 GSConv. This GSConv expands the number of channels from 128 to 384, where the hyperparameter mlpratio parameter is set to 3 and the regularization hyperparameter drop_path is set to 0. The two GSConvs after the sigmod function process the input independently, and then perform the rewritten element-level multiplication, that is, the elements at the corresponding positions are multiplied. The output shape of this step is still 160*160*384, but the value of each position is the product of the values ​​of the corresponding positions of the two convolution outputs.

[0118] The star operation deeply integrates feature maps from different subspaces through element-wise multiplication. It can map the input data to a high-dimensional, nonlinear feature space without broadening the network structure, greatly enriching the complexity and diversity of feature expression.

[0119] Since the star operation itself focuses more on local and multi-scale information extraction, the addition of MLCA (mixed local channel attention) in this implementation can further help the model capture channel, spatial, local and global information between distant pixels, reducing the limitations of the star operation. This multi-dimensional information aggregation helps the model better understand the relationship between the target and its surrounding environment, thereby improving the recognition ability of small targets.

[0120] The feature map after the star operation is input into MLCA, such as Figure 5As shown in the figure, LAP (local average pooling) is first performed to reduce the spatial dimension of the feature map to 5*5*384, and the number of channels remains unchanged. Then it is divided into two paths, the first path and the second path. The first path flattens the spatial dimension to 1*1*384, with a total of 5*5 blocks. It enters the one-dimensional convolution layer to generate a local attention map, and then the local attention map enters the Reshape layer (reshaping layer) for dimension conversion to generate a 5*5*384 attention map. The second path enters the GAP (global average pooling) layer to further compress the local extracted features in space, and the feature map becomes 1*1*384. So that they can enter the Conv1d layer (convolution layer) for processing, learn the correlation between channels, generate a channel attention map of 1*1*384, and then input the channel attention map into the UNAP layer (unpooling layer) to restore the channel attention map size to 5*5*384. The channel attention map and local attention map generated by these two paths are spliced ​​and shuffled to obtain a mixed attention map of 5*5*384. Finally, the hybrid attention map is input into the UNAP layer and adjusted to the same spatial dimension as the original input, 160*160*384. This hybrid attention is then applied to the original input features, and the feature representation is enhanced by element-by-element multiplication, outputting a feature map of size 160*160*384. MLCA is a lightweight attention mechanism that only adds a small amount of parameters and improves detection accuracy.

[0121] like Figure 4 As shown, the feature map 160*160*384 extracted by MLCA is then input into a GSConv for channel dimensionality reduction, and the output traffic image 160*160*128 is input into a DW-Conv for further feature extraction, while keeping the spatial dimension unchanged. This feature transformation and interaction helps to enhance the nonlinear ability of the network, thereby capturing more complex feature relationships. Finally, a part of the feature map input that remains intact is element-wise added to the feature map after a series of transformations, i.e., residual connection, to maintain the flow of information and alleviate the gradient vanishing problem in deep networks.

[0122] like Figure 2 As shown in the figure, after being processed by the STARMLCA layer, a new feature map tensor with a shape of 160*160*128 is output. This new feature map is concatenated with a portion of the original feature map in the channel dimension to achieve feature fusion, and a 160*160*256 feature map is obtained. It is then input into a Conv layer for channel dimensionality reduction, and the resulting feature map is 160*160*128.

[0123] like Figure 1As shown in the figure, the 160*160*128 feature map extracted by the first C2f_CAST layer is input into the Conv layer for size compression and channel number expansion, and the output is a 80*80*256 feature map, which is then input into the C2f_CAST layer for feature transformation and interaction. Then the 80*80*256 feature map passes through the Conv layer and the C2f_CAST layer, and outputs a 40*40*512 feature map. Similarly, the 40*40*512 feature map is input into the next Conv layer and the C2f_CAST layer to obtain a 20*20*1024 feature map.

[0124] The 20*20*1024 feature map is sent to the SPPF layer (fast spatial pooling layer). In the SPPF layer, the feature map is first compressed through a convolution layer to reduce the number of channels. Then, three serial pooling operations are performed using the pooling kernel size k=5, and a portion of the feature map that remains intact is concat-ed with the results of the three poolings along the channel dimension (dim=1). After that, a Conv layer is used to extract features and adjust the number of output channels to 1024. The SPPF layer finally outputs a 20*20*1024 feature map, which provides input for the neck network of the subsequent YOLOv8 model.

[0125] Secondly, in the neck network of the YOLOv8 model, such as Figure 1As shown in the figure, the feature map is first expanded to 40*40*1024 through the Upsample layer, and the Concat layer is performed with the 40*40*512 feature map extracted by the C2f_CAST layer in the backbone network to form a 40*40*1536 feature map. Then, this feature map is sent to VOVGSCSP for feature aggregation and compression, and the output is a 40*40*512 feature map. The feature map is again Upsampled to 80*80*512, and the Concat layer is performed with the feature map of the same scale of the backbone network to obtain a 80*80*768 feature map. Subsequently, VOVGSCSP further compresses it to a 80*80*256 feature map. Then, the 80*80*256 feature map is passed through GSConv to output a feature map with a reduced size but richer feature information and higher computational efficiency, with a scale of 40*40*256. This feature map is concat-ed with the 40*40*512 feature map extracted by VOVGSCSP to obtain a 40*40*768 feature map. Subsequently, this 40*40*768 feature map is processed again by GSConv, VOVGSCSP and Concat layers to finally obtain a 20*20*1024 deep feature map. In the final stage of feature fusion, the 20*20*1024, 40*40*512 and 80*80*256 deep feature maps extracted from the feature fusion network and the shallow feature maps of the same scale of the backbone feature extraction network are sent to the SDFM layer (feature fusion layer) for feature fusion. In the SDFM layer, these feature maps are concat-ed in the channel dimension to generate a fused feature map.

[0126] The following explanations are given for the GSConv, VOVGSCSP, and SDFM layers:

[0127] In the neck network of YOLOv8, GSConv aims to speed up the prediction calculation of the image by maintaining the hidden connections between channels to better capture the feature information, while retaining the hidden connections between channels as much as possible, thereby improving the detection accuracy while maintaining a low time complexity. GSConv has a lower computational cost, about 60% to 70% of the standard convolution. If it is used in all stages of YOLOv8, the model will become too deep, which will increase the resistance to data flow and significantly increase the inference time. When these feature maps are passed to the neck network, the channel dimension reaches the maximum and the width and height dimensions reach the minimum, so this implementation only replaces the convolution in the neck network of YOLOv8.

[0128] like Figure 6As shown in the figure, first, the feature map with the input channel number C1 is preliminarily processed by the Conv layer to reduce the number of channels to C2 / 2 to simplify the information. Subsequently, this feature map enters DW-Conv, and its number of channels remains unchanged, focusing on the extraction of spatial features. The output of the Conv layer and DW-Conv is subjected to the Concat layer operation to achieve preliminary integration of information. Next, the shuffling mechanism is introduced to randomly rearrange the cascaded channels. This process not only ensures the uniform distribution of channel information, but also promotes the interaction and fusion of features from different sources, effectively enhancing the semantic richness and expression ability of the feature map, and finally outputs a feature map with the number of channels C2.

[0129] In the neck network of YOLOv8, VOVGSCSP effectively reduces the computational complexity by reducing the number of channels and using 1*1 convolution for dimensionality reduction and dimensionality increase. The amount of computation will be reduced by about 15% on average compared with the latter, reducing the risk of overfitting and improving the generalization ability of the model. VOVGSCSP realizes the effective integration of multi-scale features by segmenting, transforming and merging feature maps, and improves the fusion efficiency of context information.

[0130] like Figure 7 As shown in the figure, in VOVGSCSP, the feature map with the number of channels C1 first passes through the first Conv layer and performs a 1x1 convolution operation. The output is divided into two paths, namely the first path and the second path, both of which are feature maps with the number of channels C1 / 2. In the first path processing, the feature map is input to the first GSConv and uses a 1x1 convolution kernel for convolution operation to change the number of channels. Subsequently, it is passed to the second GSConv, which uses a 3x3 convolution kernel for convolution operation to further extract and transform features. Then, the feature map is also directly passed to the Conv layer. The output of the Conv layer does not change the spatial dimension of the feature map because it uses a 1x1 convolution kernel and a step size of 1. In the second path processing, the original feature map passes through the Conv layer and also performs a 1x1 convolution operation to match the output of the first path in the channel dimension. Then the feature map processed by the first path and the feature map of the second path are subjected to the Concat layer in the channel dimension to combine the feature information from two different paths, thereby capturing richer feature representations. Finally, the feature map with the number of channels C2 is output.

[0131] In the neck network of YOLOv8, in the feature pyramid structure of the original model, different feature maps are constructed through a series of downsampling and convolution. With the gradual deepening of the downsampling operation and the stacking of multiple layers of convolution, the detail information of the feature map is gradually lost, while the semantic information is gradually enhanced. Shallow feature maps have rich features such as texture and edges. Therefore, shallow networks are more suitable for extracting detail information of small-scale targets. In PANet (path-enhanced feature fusion pyramid network), the top-down feature fusion method may reduce the feature resolution of small targets with fewer pixels after upsampling and layer-by-layer fusion, which makes it difficult to extract the features of small-scale targets. Therefore, the implementation method introduces the SDFM layer. Self-calibration operations are performed on shallow feature maps, and adaptive weighted fusion of deep feature maps is performed according to the importance of channels and spatial positions, which helps the model accurately distinguish small targets and backgrounds in complex backgrounds.

[0132] When adding the SDFM layer in YOLOv8, due to differences in design concepts, network structures, and data processing methods, directly adding it to the shallow feature layer of the backbone network of YOLOv8, although the shallow feature details are rich, it will introduce noise, resulting in performance degradation and increased computational complexity. Therefore, this implementation does not directly add the SDFM layer to the shallow feature layer of the backbone network of YOLOv8, but adopts a progressive fusion strategy to add it after the enhanced feature extraction network layer. At this time, the deep feature map has undergone a series of convolution stacking and downsampling operations, and the original features have been compressed, and the redundant information is further reduced. In addition, the SDFM layer contains multiple convolutional layers, pooling layers, and activation functions. The complex internal structure will increase the corresponding number of parameters and computational complexity. Therefore, this implementation refers to the reconstructed neck network part of YOLOv8, such as Figure 8 As shown in the figure, introducing GSConv in the SDFM layer to replace the point convolution in the SDFM layer can reduce the computational cost and the number of parameters while maintaining the accuracy of the model.

[0133] At this time, the features sent to the SDFM layer for fusion are the shallow feature maps of 20*20*1024, 40*40*512 and 80*80*256 extracted by the C2f_CAST layer and the deep feature maps of the same scale of 20*20*1024, 40*40*512 and 80*80*256 extracted by the stacking of the feature enhancement network. The input shallow feature map and the deep feature map are spliced ​​in the channel dimension to obtain feature maps of 20*20*2048, 40*40*1024 and 80*80*512. The input features are weighted with the weights obtained by the sigmod function (activation function), and residual connection and self-calibration operations are performed.

[0134] like Figure 8As shown in the figure, we first define a feature map with one-fourth of the number of intermediate channels C, GAP the concatenated features to make them 1*1 in size, and input the GAPed feature map into GSConv. We first reduce the feature map to the dimension of the intermediate variable, and then use a convolution GSConv operation to increase the dimension to twice the number of channels C. We use the sigmoid function to get the weights, and the output feature map scales are 20*20*5096, 40*40*2048, and 80*80*1024. We then use the weights obtained from the sigmoid function to weight the input features and perform residual links. One of the branches splits the weighted feature map according to the number of channels to obtain multiple pairs of feature maps of 20*20*2048, 40*40*1024, and 80*80*512. The other branch performs a channel number aggregation on the weighted features, compresses the feature map to the original channels of 20*20*2048, 40*40*1024 and 80*80*512, calculates the local attention weight and the global attention weight based on the feature map, multiplies the local attention weight and the global attention weight, and normalizes them using the sigmoid function. The multiple pairs of feature maps obtained by segmentation are aggregated according to the weights to obtain the fused features.

[0135] Finally, if Figure 1 As shown in the figure, the feature maps of these three sizes are input into the head network of YOLOv8, which generates a set of prediction boxes for each grid point on each feature map. Each prediction box contains its location information (coordinates) and confidence. At the same time, a category probability vector is output for each prediction box, indicating the probability that the prediction box contains each possible category.

[0136] like Figure 3 As shown in the figure, in the Detect (detection head) of YOLOv8, each feature map is subjected to the Conv layer and Conv2d (two-dimensional convolutional layer) operations respectively, and finally the feature map is divided into BboxLoss (regression loss) and CIsLoss (classification loss).

[0137] Then, the improved YOLOv8 is tested and evaluated: ablation experiments are carried out on different improved modules. In order to verify the improvement effect of the improved modules proposed in the implementation method on the model, 8Group ablation experiments are designed, and DAIR-V2X-I is used for testing and evaluation.

[0138] The experimental results are shown in Fig.18As shown in the figure, it includes STARMLCA layer, Neck (Neck network of YOLOv8), SDFM layer and WPIoU loss function, where "√" indicates that the module is added and "—" indicates that the module is not added. The experimental results show that with the addition of each module, YOLOv8 has improved to varying degrees in P (accuracy), R (recall rate), mAP (average precision) and FPS (detection speed). Compared with the original YOLOv8, the number of parameters has basically not increased, and the amount of calculation has decreased.

[0139] Fig.18The ablation experiment results prove the effectiveness of each improved module. The improved YOLOv8 has improved the average precision mAP@0.5, reduced the complexity of GFLOPs and Params, and balanced FPS. Specifically, the results of Group ② show that the backbone network of YOLOv8 adopts the STARMLCA layer of this implementation method, which improves the evaluation indicators mAP@0.5 and FPS by 1.5% and 8% respectively, GFLOPs (computational amount) decreases by 3.5G, and Params (parameter amount) decreases by 1.3 million. Therefore, it is proved that the addition of the STARMLCA layer improves the feature extraction performance and makes YOLOv8 lighter. Group ③ shows that the neck network of YOLOv8 is reconstructed, Params and GFLOPs are reduced by 810,000 and 3.5G respectively, and mAP@0.5 is improved by 0.6%, indicating that the reconstruction of the neck network of YOLOv8 balances the model complexity and detection performance. Group ④ shows that the addition of the improved SDFM layer improves the mAP@0.5 evaluation index by 1.8%. Since the SDFM layer contains internal modules and as the model deepens, Params and GFLOPs also increase by 950,000 and 0.7G respectively. The results of group ⑤ show that the model after replacing the WPIoU loss function has an increase of 2.0% in mAP@0.5, and the improvement results are obvious. Compared with groups ① and ⑥, the neck network of YOLOv8 is reconstructed and the STARMLCA layer is introduced. Compared with the basic model, it is lighter and more efficient, with an increase of 3.6% in mAP@0.5 and a reduction of 5.8G and 1.16 million in GFLOPs and Params respectively. Introducing the STARMLCA layer in the backbone network of YOLOv8, introducing the SDFM layer, and replacing the WPIoU loss function, the three evaluation indicators of mAP@0.5 and FPS are greatly improved. Compared with groups ① and ⑦, after introducing the STARMLCA layer and the SDFM layer into the backbone network of YOLOv8 and replacing the WPIoU loss function, the two evaluation indicators of mAP@0.5 and FPS were greatly improved, and GFLOPs and Params were further reduced. Compared with groups ① and ⑧, after integrating the four improvements, with only a small number of parameters added, the proposed improved model improved mAP@0.5 by 4.4%, GFLOPs and Params decreased by 19.8% and 8.18% respectively, and FPS reached 152frame / s. These results fully demonstrate that the improved YOLOv8 proposed in this implementation method has improved performance and real-time performance.

[0140] Embodiment 4: An electronic device described in this embodiment includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0141] Memory, used to store computer programs;

[0142] The processor is used to implement the method described in the second embodiment when executing the program stored in the memory.

[0143] Embodiment 5: This embodiment describes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method described in embodiment 2 is implemented.

[0144] The WPIoU loss function construction method and the YOLOv8 detection method proposed in the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A detection method based on improved YOLOv8 traffic image information, wherein the detection method is implemented based on the WPIoU loss function construction method and the YOLOv8 improvement method, and is characterized in that: Specifically: The traffic image is sequentially input into the YOLOv8 backbone network, YOLOv8 neck network, and YOLOv8 head network to generate prediction results; The prediction results are calculated through the WPIoU loss function, and the detected traffic information is output; The WPIoU loss function construction method includes the following steps: Step S1, after performing gradient gain processing on the WIoUv1 loss function, obtain the WIoUv3 loss function; Step S2, the PIoU loss function customizes an adaptive penalty factor; Step S3, after improving the WIoUv3 loss function described in step S1 based on the penalty factor described in step S2, a WPIoU loss function is obtained; Step S4, further improving the WPIoU loss function described in step S3 based on the non-monotonic attention function of the PIOUv2 loss function to obtain the final WPIoU loss function; In step S4, the final WPIoU loss function is: Among them, L WPIoU ' is the final WPIoU loss function, L WPIoU is the WPIoU loss function, P is the penalty factor, λ is a single hyperparameter, and u is the non-monotonic attention function; YOLOv8 improvement method, specifically: The image is input into the backbone network of YOLOv8 and after the Conv layer reduces the dimension, the reduced image is input into multiple cascaded C2f_CAST layers and Conv layers; In each C2f_CAST layer, part of the reduced-dimensional images remain as they are, and the other part of the reduced-dimensional images are input into multiple cascaded STARMLCA layers; In the STARMLCA layer, the other part of the reduced-dimensional image passes through DW-Conv to keep the spatial dimension unchanged. The other part of the reduced-dimensional image that passes through the DW-Conv layer is divided into two paths, and the number of channels is expanded by GSConv respectively. The two paths of the other part of the reduced-dimensional image that pass through GSConv are multiplied element-wise to output the feature map. The feature map is sequentially extracted by MLCA, GSConv and DW-Conv, and the image after feature extraction is element-wise added with the part of the reduced-dimensional image that remains unchanged. After multiple cascaded C2f_CAST layers and Conv layers, the image is subjected to feature extraction again; The feature-extracted image is input into the neck network of YOLOv8 for feature fusion, and the fused feature map is output; The fused feature map is input into the head network of YOLOv8 for prediction to generate the prediction result.

2. The detection method based on improved YOLOv8 traffic image information according to claim 1, characterized in that: In the step S1, the WIoUv3 loss function is: L WIoUv3 =rL WIoUv1 ; Among them, r is the gradient gain, L WIoUv3 is the WIoUv3 loss function, L WIoUv1 is the WIoUv1 loss function, and L is the loss function.

3. The detection method based on improved YOLOv8 traffic image information according to claim 2 is characterized in that: The WIoUv1 loss function is: Among them, L WIoUv1 is the WIoUv1 loss function, assuming that the corresponding position of (x, y) in the target box is (x gt ,y gt ), Wg is the width of the annotation box, Hg is the height of the annotation box, and g represents the annotation box.

4. The detection method based on improved YOLOv8 traffic image information according to claim 2, characterized in that: The r is: Among them, r is the gradient gain, δ and α are hyperparameters, and β is the outlier degree.

5. The detection method based on improved YOLOv8 traffic image information according to claim 1, characterized in that: In step S3, the WPIoU loss function is: Among them, L WPIoU is the WPIoU loss function, L WIoUv3 is the WIoUv3 loss function, and P is the penalty factor.

6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the method described in claim 1 when executing the program stored in the memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to claim 1 is implemented.

Images