Multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8

Abstract

The invention relates to the technical field of target detection, and discloses a multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8. According to the method, a YOLOv8 network is improved through a ParmC2f module, an ADCD Head module and a Width IoU loss function, the ParmC2f module is used for replacing the C2f module, the parameter quantity of a model is increased, the detection precision of the model is improved to a certain extent, it is ensured that FLOAPs are not changed, the model reasoning speed is ensured, the model is not directly influenced by the increase of the parameter quantity, the ADCD Head module is used for replacing a standard Head module, and the detection accuracy of the model is improved. According to the method, target information of samples with different scales is effectively extracted by utilizing multi-scale information fusion, effective feature extraction of multi-target samples is realized by adopting adaptive cavity convolution, the attention of a model to the samples in a multi-scale and multi-sample complex traffic environment is dynamically adjusted by using a Wise IoU loss function, and the processing of low-quality samples is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8. Background Art

[0002] With the advancement of science and technology, the field of smart traffic detection has gradually become the focus of people's attention. In recent years, smart traffic detection algorithms based on visible light have continued to emerge, and have achieved good detection results under normal traffic environment conditions. However, under extreme environmental conditions such as smoke, glare, and dust, the detection accuracy of this type of method cannot fully meet the precision of smart traffic detection. Compared with visible light images, thermal infrared radiation images with strong penetration and heat distribution characteristics can more comprehensively adapt to complex traffic conditions.

[0003] However, thermal infrared radiation images have the characteristics of visual blur and no linear relationship between reflected target features, which makes thermal infrared target detection face challenges in detection accuracy. In addition, thermal infrared images also have certain limitations in resolution and detail expression, which increases the difficulty of identifying and classifying targets. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a multi-scale and multi-target thermal infrared traffic target detection method based on improved YOLOv8, which has the advantages of accurate detection and solves the above technical problems.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8, comprising the following steps:

[0008] S1. Obtain real-life traffic images in complex scenes and create a thermal infrared traffic dataset, and divide the dataset into a training set, a validation set, and a test set in a ratio of 7:1:2;

[0009] S2. Improve the YOLOv8 network model based on the Parm_C2f module, ADC_Detect Head module and Wise IoU loss function to obtain an improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model;

[0010] S3, inputting the thermal infrared traffic training set and the validation set into the improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model for training, and obtaining the trained model weight file;

[0011] S4. Perform inference test on the test set data through the weight file to obtain the predicted anchor box and prediction confidence of each target category in each test set image.

[0012] As a preferred technical solution of the present invention, the improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model includes a Backbone layer, a Neck layer and an ADC_Detect Head layer;

[0013] The Backbone layer includes two Conv modules, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module and an SPPF module connected in sequence;

[0014] The Neck layer consists of three branches, including two identical branches consisting of a Concat module, a Parm_C2f module, and a Conv module, a branch consisting of a Concat module, a Parm_C2f module, and an Upsample module, and a branch consisting of a Concat module and a Parm_C2f module;

[0015] The ADC_Detect Head layer includes three Remaping modules, three ADConv and three detection units;

[0016] Each detection unit is divided into two paths, one path is connected to the CBS module, Conv2d module and Bbox Loss module in sequence, and the other path is connected to the CBS module, Conv2d module and Cls Loss module in sequence. Among them, the CIoU loss function and DFL loss function in the Bbox Loss module are replaced by the Wise IoU loss function.

[0017] As a preferred technical solution of the present invention, two Parm_Block modules are connected in series in the Parm_C2f module, and the Parm_C2f module is used in step S2 to improve the YOLOv8 network model: the Parm_C2f module is used to replace the C2f module in the YOLOv8 network, and the specific steps are as follows:

[0018] S2a.1. Set the size of the input Parm_C2f module feature map to H×W×C, where H is the height, W is the width, and C is the number of channels. Use the CBS module and Split module in the YOLOv8 network model to perform preliminary feature extraction on the input image to obtain a feature map of size H×W×C_out and divide it into two equal parts according to the number of channels. One part is used as the input of the Parm_Block module, and the other part is used as the partial input of the residual connection of the YOLOv8 network model, that is, the first feature map.

[0019] S2a.2, set the feature size of the input Parm_Block module to H×W×0.5C_out. The Parm_Block module receives the feature map extracted by the CBS module and uses it as the input of the dynamic convolution Dy_Conv in the Parm_Block module. The dynamic convolution Dy_Conv passes the input feature to the MoE Layer and finally generates a second feature map of size H×W×0.5C_out. 0.5C_out represents half of the number of channels. The calculation expression of the MoE Layer is as follows:

[0020] Y=X * W′

[0021]

[0022] in, represents the parameters of the i-th dynamic convolution kernel, M represents the number of convolution kernels, α i represents the corresponding dynamic hyperparameters, X represents the input feature map information, W' represents the dynamic convolution kernel calculated in a parameterized manner, and Y represents the output features of the MoE Layer;

[0023] S2a.3. Input the second feature map of size H×W×0.5C_out into the second Parm_Block, generate a third feature map of size H×W×0.5C_out, perform channel cascade operation on the first feature map, the second feature map and the third feature map, and perform feature fusion as the output of the Parm_C2f module.

[0024] As a preferred technical solution of the present invention, the ADC_Detect Head module improves the YOLOv8 network model: the ADC_Detect Head module is used to replace the standard Detect Head module in the YOLOv8 network. The specific steps are as follows:

[0025] S2b.1, ADC_Detect Head module will receive three feature maps of different sizes transmitted from three different Parm_C2f modules, perform linear interpolation up and down sampling operations through Remaping module, unify the three feature maps of different sizes, and concatenate the three feature maps of the same size through channel cascading, and input them into three corresponding ADConv respectively;

[0026] S2b.2, ADConv samples the input feature map according to a regular grid and extracts information from independent grids, eventually generating a feature map of size H×W×0.5C_out, where the expression extracted from each independent grid in ADConv is as follows:

[0027]

[0028] Among them, F0(x n )and Represent the output features and input features of the specified block, Δx i It is equivalent to x n The convolution kernel offset position, x n Represents a specified block of the input feature map, represents the adaptive void ratio, and G represents the position coordinate set of each divided area after regular grid division. i ) represents the weighted calculation operation for the convolution kernel offset position.

[0029] As a preferred technical solution of the present invention, the expression of the Wise IoU loss function is as follows:

[0030]

[0031] Among them, L IoU Represents the Wise IoU loss function value, W i and H i Respectively represent the width and height of the overlapping part, w gt and h gt They represent the width and height of the label category box, respectively, and w and h represent the width and height of the predicted anchor box, respectively.

[0032] Compared with the prior art, the present invention provides a multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8, which has the following beneficial effects:

[0033] The present invention improves the YOLOv8 network through the Parm_C2f module, the ADC_Detect Head module and the Wise IoU loss function, and replaces the C2f module with the Parm_C2f module, which not only increases the parameter amount of the model and improves the detection accuracy of the model to a certain extent, but also ensures that the FLOAPs remain unchanged, and the model reasoning speed is guaranteed, which is not directly affected by the increase in the parameter amount. The ADC_Detect Head module is used to replace the standard Detect Head module, and multi-scale information fusion is used to effectively extract target information of samples of different scales, and adaptive void convolution is used to realize effective feature extraction of multi-target samples. The Wise IoU loss function is used to dynamically adjust the model's attention to samples in multi-scale and multi-sample complex traffic environments, and optimize the processing of low-quality samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the process of the present invention;

[0035] Figure 2 This is a schematic diagram of the improved YOLOv8 network model structure of the present invention;

[0036] Figure 3 It is a schematic diagram of the Parm_C2f structure of the present invention;

[0037] Figure 4 It is a schematic diagram of the detection results of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1-Figure 4 , a multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8 includes the following steps:

[0040] S1. Obtain real-life traffic images in complex scenes and create a thermal infrared traffic dataset, and divide the dataset into a training set, a validation set, and a test set in a ratio of 7:1:2;

[0041] S2. Improve the YOLOv8 network model based on the Parm_C2f module, ADC_Detect Head module and Wise IoU loss function to obtain an improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model;

[0042] The improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model includes Backbone layer, Neck layer and ADC_Detect Head layer;

[0043] The Backbone layer includes two Conv modules, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module, and an SPPF module connected in sequence;

[0044] The Neck layer consists of three branches, including two identical branches consisting of Concat module, Parm_C2f module, and Conv module, one branch consisting of Concat module, Parm_C2f module, and Upsample module, and one branch consisting of Concat module and Parm_C2f module;

[0045] The ADC_Detect Head layer consists of three Remaping modules, three ADConv and three detection units;

[0046] Each detection unit is divided into two paths, one path is connected to the CBS module, Conv2d module and Bbox Loss module in sequence, and the other path is connected to the CBS module, Conv2d module and Cls Loss module in sequence. Among them, the CIoU loss function and DFL loss function in the Bbox Loss module are replaced by the Wise IoU loss function;

[0047] Two Parm_Block modules are connected in series in the Parm_C2f module. In step S2, the Parm_C2f module is used to improve the YOLOv8 network model: the Parm_C2f module is used to replace the C2f module in the YOLOv8 network. The specific steps are as follows:

[0048] S2a.1. Set the size of the input Parm_C2f module feature map to H×W×C, where H is the height, W is the width, and C is the number of channels. Use the CBS module and Split module in the YOLOv8 network model to perform preliminary feature extraction on the input image to obtain a feature map of size H×W×C_out and divide it into two equal parts according to the number of channels. One part is used as the input of the Parm_Block module, and the other part is used as the partial input of the residual connection of the YOLOv8 network model, that is, the first feature map.

[0049] S2a.2, set the feature size of the input Parm_Block module to H×W×0.5C_out, the Parm_Block module receives the feature map extracted by the CBS module and uses it as the input of the dynamic convolution Dy_Conv in the Parm_Block module. The dynamic convolution Dy_Conv passes the input features to the MoE Layer and uses multiple expert convolution kernels to train simultaneously to adaptively adjust the parameter α i (i=1,2,...,M), thereby dynamically fitting the optimal weight of the corresponding convolution kernel. It also achieves the use of a larger number of parameters to enhance the feature extraction capability while ensuring the model's reasoning speed, and finally generates a second feature map of size H×W×0.5C_out, where 0.5C_out represents half the number of channels. The calculation expression of the MoE Layer is as follows:

[0050] Y=X * W′

[0051]

[0052] in, Represents the parameters of the i-th dynamic convolution kernel. M represents the number of convolution kernels. αi Indicates the corresponding dynamic hyperparameters. X represents the input feature map information. W' represents the dynamic convolution kernel calculated in a parameterized way. Y represents the output features of the MoE Layer;

[0053] That is, for input X, global average pooling is applied to fuse the information into a vector, and then a two-layer MLP module with softmax activation is used to dynamically generate coefficients, where α can be expressed as:

[0054] α=softmax(MLP(Pool(X)))

[0055] In the formula,

[0056] S2a.3, input the second feature map of size H×W×0.5C_out into the second Parm_Block, generate a third feature map of size H×W×0.5C_out, perform channel cascade operation on the first feature map, the second feature map and the third feature map, and perform feature fusion as the output of the Parm_C2f module;

[0057] Two Parm_Blocks are connected in series in the entire Parm_C2f, and the number of output channels of the two is consistent with the number of input channels, that is, the feature map of 0.5C_out channels, and the Concat operation is performed with the original feature map of 0.5C_out channels. The entire Parm_C2f uses the characteristics of its residual structure to stably perform deeper feature extraction. In addition, Parm_Block uses MoELayer to increase the number of parameters and improve the model's feature mining capabilities while ensuring that FLOAPs does not increase with the increase in the number of parameters, ensuring that the model's inference speed is not directly affected by the number of parameters;

[0058] The ADC_DetectHead module improves the YOLOv8 network model: Use the ADC_Detect Head module to replace the standard DetectHead module in the YOLOv8 network. The specific steps are as follows:

[0059] S2b.1, ADC_DetectHead module will receive three feature maps of different sizes transmitted from three different Parm_C2f modules, perform linear interpolation up and down sampling operations through Remaping module, unify the three feature maps of different sizes, and concatenate the three feature maps of the same size through channel cascading, and input them into three corresponding ADConv respectively;

[0060] S2b.2, ADConv samples the input feature map according to a regular grid, the regular grid is:

[0061] G={(-1,-1),(-1,0),...,(0,1),(1,1)}

[0062] In the formula, each coordinate is the coordinate information of an independent grid in the grid divided by standard convolution, and the independent grid information is extracted to finally generate a feature map of size H×W×0.5C_out, where the expression extracted from each independent grid in ADConv is as follows:

[0063]

[0064] Among them, F0(x n )and Represent the output features and input features of the specified block, Δx i It is equivalent to x n The convolution kernel offset position, x n Represents a specified block of the input feature map, represents the adaptive void ratio, and G represents the position coordinate set of each divided area after regular grid division. i ) represents the weighted calculation operation for the convolution kernel offset position;

[0065] The expression of the Wise IoU loss function is as follows:

[0066]

[0067] Among them, L IoU Represents the Wise IoU loss function value, W i and H i Respectively represent the width and height of the overlapping part, w gt and h gt Represent the width and height of the label category box, w and h represent the width and height of the predicted anchor box respectively;

[0068] The core of Wise IoU is to dynamically adjust the model's attention to the anchor box by evaluating the quality difference between the predicted anchor box and the label box, that is, the dynamic non-monotonic focusing mechanism. The quality of each anchor box is judged according to its abnormality β, which can be expressed as:

[0069]

[0070] In the formula, LIoU represents the overlap between the predicted anchor box and the label box. is the exponential moving average of LIoU.

[0071] The gain is adjusted according to the abnormality β by using a specific gradient gain r. The gradient gain r can be expressed as:

[0072]

[0073] Where α and δ are hyperparameters used to control the shape of r. This formula is designed to make r change with β rather than monotonically to achieve the following goals:

[0074] For high-quality anchor boxes (small β), r is set to a smaller value to reduce their weight in the total loss, thereby avoiding the model from overfitting these well-matched cases.

[0075] For anchor boxes of normal quality, r is larger, which increases the weight of these anchor boxes in the total loss and encourages the model to learn more from these improveable cases.

[0076] For low-quality anchor boxes (β is large), r is set to a small value again to reduce the contribution of these samples that may be noise or difficult to match to the total loss, avoiding the model wasting too much energy on these suboptimal data.

[0077] S3, inputting the thermal infrared traffic training set and the validation set into the improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model for training, and obtaining the trained model weight file;

[0078] S4. Perform inference test on the test set data through the weight file to obtain the predicted anchor box and prediction confidence of each target category in each test set image;

[0079] S4.1. Read an image from the test set and detect it using the weight file. The detection result is as follows: Figure 4 As shown;

[0080] S4.2, re-read the images in the test set until all images in the test set have been detected. Using the prepared dataset, set the confidence threshold to 0.5 and get the overall performance of the model;

[0081] The infrared traffic target detection method of this application is compared with the performance of YOLOv7 and YOLOv8 in three categories: people, cars, and bicycles. These methods are the existing technologies that have very good feature matching capabilities. Using these methods as comparison methods can more objectively prove the ability and effect of the infrared traffic target detection method of this application.

[0082]

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8, characterized by: The following steps are involved: S1. Obtain real-life traffic images in complex scenes and create a thermal infrared traffic dataset, and divide the dataset into a training set, a validation set, and a test set in a ratio of 7:1:2; S2. Improve the YOLOv8 network model based on the Parm_C2f module, ADC_Detect Head module and Wise IoU loss function to obtain an improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model; S3, inputting the thermal infrared traffic training set and the validation set into the improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model for training, and obtaining the trained model weight file; S4. Perform inference test on the test set data through the weight file to obtain the predicted anchor box and prediction confidence of each target category in each test set image.

2. The multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8 according to claim 1 is characterized in that: The improved YOLOv8 multi-scale multi-target thermal infrared traffic target detection model includes a Backbone layer, a Neck layer and an ADC_Detect Head layer; The Backbone layer includes two Conv modules, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module, a Conv module, a Parm_C2f module and an SPPF module connected in sequence; The Neck layer consists of three branches, including two identical branches consisting of a Concat module, a Parm_C2f module, and a Conv module, a branch consisting of a Concat module, a Parm_C2f module, and an Upsample module, and a branch consisting of a Concat module and a Parm_C2f module; The ADC_Detect Head layer includes three Remaping modules, three ADConv and three detection units; Each detection unit is divided into two paths, one path is connected to the CBS module, Conv2d module and Bbox Loss module in sequence, and the other path is connected to the CBS module, Conv2d module and Cls Loss module in sequence. Among them, the CIoU loss function and DFL loss function in the Bbox Loss module are replaced by the Wise IoU loss function.

3. The multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8 according to claim 2 is characterized in that: The Parm_C2f module connects two Parm_Block modules in series. The Parm_C2f module is used in step S2 to improve the YOLOv8 network model: the Parm_C2f module is used to replace the C2f module in the YOLOv8 network. The specific steps are as follows: S2a.

1. Set the size of the input Parm_C2f module feature map to H×W×C, where H is the height, W is the width, and C is the number of channels. Use the CBS module and Split module in the YOLOv8 network model to perform preliminary feature extraction on the input image to obtain a feature map of size H×W×C_out and divide it into two equal parts according to the number of channels. One part is used as the input of the Parm_Block module, and the other part is used as the partial input of the residual connection of the YOLOv8 network model, that is, the first feature map. S2a.2, set the feature size of the input Parm_Block module to H×W×0.5C_out. The Parm_Block module receives the feature map extracted by the CBS module and uses it as the input of the dynamic convolution Dy_Conv in the Parm_Block module. The dynamic convolution Dy_Conv passes the input feature to the MoE Layer and finally generates a second feature map of size H×W×0.5C_out. 0.5C_out represents half of the number of channels. The calculation expression of the MoE Layer is as follows: Y=X * W′ in, represents the parameters of the i-th dynamic convolution kernel, M represents the number of convolution kernels, α i represents the corresponding dynamic hyperparameters, X represents the input feature map information, W' represents the dynamic convolution kernel calculated in a parameterized manner, and Y represents the output features of the MoE Layer; S2a.

3. Input the second feature map of size H×W×0.5C_out into the second Parm_Block, generate a third feature map of size H×W×0.5C_out, perform channel cascade operation on the first feature map, the second feature map and the third feature map, and perform feature fusion as the output of the Parm_C2f module.

4. The multi-scale multi-target thermal infrared traffic target detection method based on improved YOLOv8 according to claim 2 is characterized in that: The ADC_Detect Head module improves the YOLOv8 network model: the ADC_DetectHead module is used to replace the standard Detect Head module in the YOLOv8 network. The specific steps are as follows: S2b.1, ADC_Detect Head module will receive three feature maps of different sizes transmitted from three different Parm_C2f modules, perform linear interpolation up and down sampling operations through Remaping module, unify the three feature maps of different sizes, and concatenate the three feature maps of the same size through channel cascading, and input them into three corresponding ADConv respectively; S2b.2, ADConv samples the input feature map according to a regular grid and extracts information from independent grids, eventually generating a feature map of size H×W×0.5C_out, where the expression extracted from each independent grid in ADConv is as follows: Among them, F0(x n )and Represent the output features and input features of the specified block, Δx i It is equivalent to x n The convolution kernel offset position, x n Represents a specified block of the input feature map, represents the adaptive void ratio, G represents the position coordinate set of each divided area after regular grid division, w(Δx i ) represents the weighted calculation operation for the convolution kernel offset position.

5. The multi-scale multi-target thermal infrared traffic target detection method based on improved YOL0v8 according to claim 2 is characterized in that: The expression of the Wise IoU loss function is as follows: Among them, L IoU Represents the WiseIoU loss function value, W i and H i Respectively represent the width and height of the overlapping part, w gt and h gt They represent the width and height of the label category box, respectively, and w and h represent the width and height of the predicted anchor box, respectively.

Images