Night infrared detection method and system based on improved lightweight YOLOv8 model

Abstract

The invention provides a night infrared detection method and system based on an improved lightweight YOLOv8 model, and relates to the technical field of deep learning and target detection. The method comprises the following steps: firstly, acquiring an infrared image data set, preprocessing the infrared image data set, and dividing the preprocessed infrared image data set into a training set, a verification set and a test set; an improved lightweight YOLOv8 model is constructed; according to the improved lightweight YOLOv8 model, on the basis of a YOLOv8 model, a DWR module used for improving the multi-scale and feature extraction capability of an infrared image, a CGAFusion module used for improving the feature fusion capability of the image, and an LSDECDDetect module are added. Training the improved lightweight YOLOv8 model by using the training set, and verifying the effectiveness of the improved lightweight YOLOv8 model by using the verification set in the training process to obtain a trained improved lightweight YOLOv8 model; and inputting the test set into the trained improved lightweight YOLOv8 model to obtain an infrared detection result, thereby improving the vehicle detection accuracy under the condition of poor visibility at night.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and target detection, and more specifically relates to a nighttime infrared detection method and system based on an improved lightweight YOLOv8 model. Background Art

[0002] With the development of science and technology, the development of autonomous driving technology has made significant progress. From the initial assisted driving system to fully unmanned driving, the technical path is clear and steadily advancing. At present, autonomous vehicles are able to achieve relatively complex driving tasks in specific scenarios. However, existing survey data show that accidents caused by night driving account for a large proportion of all traffic accidents. At present, in order to facilitate night driving, many commercial vehicle manufacturers have deployed night vision systems using thermal imaging. The use of automated equipment such as sensors and cameras combined with advanced algorithms of deep learning to detect safety risks inside and outside the vehicle body has become an indispensable key technology in the field of autonomous driving.

[0003] Deep learning is an important branch of artificial intelligence. It has made significant progress in image recognition, speech recognition, natural language processing, etc. With the development of deep learning, detection technology using artificial neural networks is constantly being updated. Among them, single-stage algorithms that do not need to generate candidate target areas, such as YOLO and SSD, have more advantages in real time and are suitable for complex traffic scenes. YOLOv8 is built on the success of previous YOLO versions. Compared with its predecessor, YOLOv8 has made a series of optimizations: using deep separable convolution, fusing multi-scale features, and using more efficient activation functions. However, the current YOLOv8 model faces defects such as large amount of calculation and parameters, low detection accuracy and real-time performance, and difficulty in detecting small targets. If it is used as a means of target detection for night infrared detection, it will lead to low vehicle detection accuracy in the case of poor night visibility. Summary of the invention

[0004] In order to solve the current problem of low vehicle detection accuracy under poor nighttime visibility conditions, the present invention proposes a nighttime infrared detection method and system based on an improved lightweight YOLOv8 model to improve the accuracy of vehicle detection under poor nighttime visibility conditions.

[0005] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0006] A nighttime infrared detection method based on an improved lightweight YOLOv8 model includes the following steps:

[0007] S1: Acquire an infrared image dataset, preprocess the infrared image dataset, and divide the preprocessed infrared image dataset into a training set, a validation set, and a test set;

[0008] S2: construct an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a LSDECD_Detect module on the basis of the YOLOv8 model;

[0009] S3: Use the training set to train the improved lightweight YOLOv8 model, and during the training process, use the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model;

[0010] S4: Input the test set into the trained improved lightweight YOLOv8 model to obtain the infrared detection results.

[0011] Furthermore, the acquired infrared image dataset includes: FLIR raw dataset and Multi-spectral dataset;

[0012] The preprocessing operations on the infrared image dataset are as follows: the invalid images and images with few categories in the FLIR original dataset are deleted, and the number of categories in the Multi-spectral dataset remains unchanged; the preprocessed FLIR original dataset and Multi-spectral dataset are divided into training set, validation set and test set in the ratio of 8:1:1, respectively.

[0013] Furthermore, the improved lightweight YOLOv8 model consists of three parts: Backbone network, Neck network and Head network;

[0014] The Backbone network includes: a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a first C2f_DWR module, a fifth Conv module, a second C2f_DWR module and a first SPPF module connected sequentially from top to bottom;

[0015] The Neck network includes: a first Upsample module, a first Concat module, a third C2f module, a second Upsample module, a second Concat module and a fourth C2f module connected sequentially from bottom to top; a sixth Conv module, a third Concat module, a fifth C2f module, a seventh Conv module, a fourth Concat module and a sixth C2f module connected sequentially from top to bottom; a first CGAFusion module, a second CGAFusion module and a third CGAFusion module; the output end of the second C2f module is connected to the first Concat module and the first CGAFusion module respectively, the output end of the first C2f_DWR module is connected to the second CGAFusion module, the output end of the first SPPF module is connected to the first Upsample module, the fourth Concat module and the third CGAFusion module respectively, the output end of the fourth C2f module is connected to the sixth Conv module and the first CGAFusion module respectively, the output end of the fifth C2f module is connected to the second CGAFusion module, and the output end of the sixth C2f module is connected to the third CGAFusion module;

[0016] The head network includes a LSDECD_Detect module; output ends of the first CGAFusion module, the second CGAFusion module and the third CGAFusion module are all connected to the LSDECD_Detect module.

[0017] Furthermore, the C2f_DWR module includes: a seventh Conv module, a Split module, an n-layer DWR module and an eighth Conv module connected in sequence; the output ends of the seventh Conv module, the Split module and the n-layer DWR module are spliced ​​and connected to the eighth Conv module.

[0018] Furthermore, the DWR module includes: regional residual RR and semantic residual SR; regional residual RR includes: the ninth Conv module, the first combined batch normalization layer, and the ReLU activation function layer; the semantic residual SR includes: the first DConv module and the second combined batch normalization layer.

[0019] Furthermore, the dilation rates of the three dilated convolution branches taken in the semantic residual SR of the DWR module are 1, 3, and 5, respectively.

[0020] Furthermore, in the CGAFusion module: let X∈R C×H×W Represents the input infrared image features. The input infrared image features are processed by the spatial attention mechanism and the channel attention mechanism to obtain the channel spatial information weight matrix W respectively. s and the channel related information matrix Wc ; Among them, the channel space information weight matrix W s and the channel related information matrix W c Satisfy the following formulas respectively:

[0021]

[0022] In the formula, c k×k (·) indicates that the convolution kernel size is k×k, max(0,x) indicates the ReLU activation function, [·] indicates the channel-by-channel cascade operation, GAP indicates the global average pooling operation, GMP indicates the global maximum pooling operation, and X s ,X c denote the cross-spatial dimension and channel dimension respectively;

[0023] W c and W s Fusion obtains rough feature information W coa =W c +W s ; W coa Shuffle and rearrange each channel of X to obtain refined feature information W. The process satisfies:

[0024] W=σ(gc 7×7 (CS([x,W coa ])))

[0025] In the formula, σ represents the sigmoid activation function, CS(·) represents channel shuffle, and gc k×k (·) represents a group convolution kernel of size k×k.

[0026] Furthermore, the LSDECD_Detect module adopts the LSDECD detection head to realize the shared use of convolution, adopts the GN group normalization strategy in the convolution, and calculates the mean variance of all channels in groups to realize normalization.

[0027] Furthermore, during training, the ATFL adaptive threshold focus loss function is used as the classification loss function of the improved lightweight YOLOv8 model; and the PIoUv2 loss function is used as the classification regression function of the improved lightweight YOLOv8 model.

[0028] The present invention also provides a nighttime infrared detection system based on an improved lightweight YOLOv8 model, comprising:

[0029] A data acquisition and processing module is used to acquire an infrared image data set, preprocess the infrared image data set, and divide the preprocessed infrared image data set into a training set, a verification set, and a test set;

[0030] A model building module, used to build an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a lightweight detection head module on the basis of the YOLOv8 model;

[0031] The training module uses the training set to train the improved lightweight YOLOv8 model, and during the training process, uses the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model;

[0032] The detection module is used to input the test set into the trained improved lightweight YOLOv8 model to obtain the infrared detection results.

[0033] Compared with the prior art, the beneficial effects of this method are:

[0034] The present invention proposes a nighttime infrared detection method and system based on an improved lightweight YOLOv8 model. First, an infrared image data set is obtained, the infrared image data set is preprocessed, and the preprocessed infrared image data set is divided into a training set, a verification set, and a test set. The preprocessing deletes the categories that affect the detection performance in the infrared image data set, thereby improving the generalization ability of the improved lightweight YOLOv8 model and the detection accuracy of infrared images. Then, an improved lightweight YOLOv8 model is constructed, and a DWR module is added to the improved lightweight YOLOv8 model to improve the feature extraction ability of multi-scale and different category targets of infrared images, while improving the problems of false detection and missed detection of infrared weak targets; the CGAFusion module is added to further enhance the ability of the feature fusion network; and the LSDECD_Detect module is added to further improve the accuracy on the basis of lightweight. The present invention adopts an improved lightweight YOLOv8 model as a whole to improve the vehicle detection accuracy under the condition of poor visibility at night. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart showing a nighttime infrared detection method based on an improved lightweight YOLOv8 model proposed in an embodiment of the present invention;

[0036] Figure 2 An example diagram of an infrared image dataset used in an embodiment of the present invention;

[0037] Figure 3 A structural diagram of an improved lightweight YOLOv8 model proposed in an embodiment of the present invention is shown;

[0038] Figure 4A schematic diagram showing the structure of a C2f_DWR module proposed in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of a DWR module proposed in an embodiment of the present invention is shown;

[0040] Figure 6 A schematic diagram of a CGAFusion module proposed in an embodiment of the present invention is shown;

[0041] Figure 7 A schematic diagram of the LSDECD_Detect module proposed in an embodiment of the present invention is shown;

[0042] Figure 8 A schematic diagram showing the nighttime infrared detection results of the test set proposed in an embodiment of the present invention;

[0043] Fig. 9 The figure shows a structural diagram of a nighttime infrared detection system based on an improved lightweight YOLOv8 model proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0045] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the actual size;

[0046] It is understandable to those skilled in the art that descriptions of certain well-known contents in the drawings may be omitted.

[0047] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0048] The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present patent;

[0049] Example 1

[0050] This embodiment proposes a night infrared detection method based on an improved lightweight YOLOv8 model. Figure 1 The flowchart of the method comprises the following steps:

[0051] S1: Acquire an infrared image dataset, preprocess the infrared image dataset, and divide the preprocessed infrared image dataset into a training set, a validation set, and a test set;

[0052] S2: construct an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a LSDECD_Detect module on the basis of the YOLOv8 model;

[0053] S3: Use the training set to train the improved lightweight YOLOv8 model, and during the training process, use the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model;

[0054] S4: Input the test set into the trained improved lightweight YOLOv8 model to obtain the infrared detection results.

[0055] In this embodiment, a process of obtaining an infrared image data set and preprocessing the infrared image data set is proposed. An example of the infrared image data set is shown in FIG. Figure 2 shown.

[0056] The acquired infrared image datasets include: FLIR original dataset and Multi-spectral dataset.

[0057] The FLIR original dataset is the ADAS dataset v2.0.0 released by Teledyne FLIR on January 19, 2022. It is captured by a thermal imaging camera and has a total of 26,442 annotated frames. It is a dataset specifically for thermal infrared image processing and contains rich thermal infrared images and corresponding label information. The FLIR original dataset includes 15 categories: pedestrians, bicycles, cars, motorcycles, buses, trains, trucks, traffic lights, fire hydrants, road signs, dogs, skateboards, strollers, scooters, and other vehicles.

[0058] A multi-spectral dataset is a dataset that contains multiple spectral bands, usually including image data of multiple bands such as red, green, and near infrared. The far infrared data is selected for use.

[0059] After removing invalid images without label information and images with less than 500 labels, the FLIR original dataset is composed of five categories of images: cars, pedestrians, buses, bicycles, and motorcycles. The number of categories in the Multi-spectral dataset remains unchanged.

[0060] The preprocessed dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0061] In this embodiment, a model of an improved lightweight YOLOv8 model is proposed, and the structure is as follows Figure 3shown.

[0062] The improved lightweight YOLOv8 model consists of three parts: Backbone network, Neck network and Head network;

[0063] The Backbone network includes: a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a first C2f_DWR module, a fifth Conv module, a second C2f_DWR module and a first SPPF module connected sequentially from top to bottom;

[0064] The Neck network includes: a first Upsample module, a first Concat module, a third C2f module, a second Upsample module, a second Concat module and a fourth C2f module connected sequentially from bottom to top; a sixth Conv module, a third Concat module, a fifth C2f module, a seventh Conv module, a fourth Concat module and a sixth C2f module connected sequentially from top to bottom; a first CGAFusion module, a second CGAFusion module and a third CGAFusion module; the output end of the second C2f module is connected to the first Concat module and the first CGAFusion module respectively, the output end of the first C2f_DWR module is connected to the second CGAFusion module, the output end of the first SPPF module is connected to the first Upsample module, the fourth Concat module and the third CGAFusion module respectively, the output end of the fourth C2f module is connected to the sixth Conv module and the first CGAFusion module respectively, the output end of the fifth C2f module is connected to the second CGAFusion module, and the output end of the sixth C2f module is connected to the third CGAFusion module;

[0065] The head network includes a LSDECD_Detect module; output ends of the first CGAFusion module, the second CGAFusion module and the third CGAFusion module are all connected to the LSDECD_Detect module.

[0066] For the Backbone network, the C2f module is replaced by the C2f_DWR module. Infrared images are affected by atmospheric scattering, clutter interference, and various noises, and have problems such as less detailed texture information and low signal-to-noise ratio. The C2f_DWR module can improve the feature extraction capability and adapt to the problems of weak semantic information of infrared image target features and difficulty in detecting multi-category and multi-scale targets in complex scenes.

[0067] Figure 4: is a structural diagram of the C2f_DWR module, the C2f_DWR module comprises: a seventh Conv module, a Split module, an n-layer DWR module and an eighth Conv module connected in sequence; the output ends of the seventh Conv module, the Split module and the n-layer DWR module are spliced ​​and connected to the eighth Conv module.

[0068] Figure 5 It is a schematic diagram of a DWR module, wherein the DWR module includes: a regional residual RR and a semantic residual SR; the regional residual RR includes: a ninth Conv module, a first combined batch normalization layer, and a ReLU activation function layer; the semantic residual SR includes: a first DConv module and a second combined batch normalization layer.

[0069] First, a common 3×3 convolution is used to extract features, and the regional features are activated by combining the batch normalization layer and the ReLU activation function layer. The generated regional type feature maps of different sizes are passed as residual features to the second step; the second step groups the regional feature maps and uses deep convolution expansion at different rates on different groups. Each channel in this step only uses a single receptive field to avoid repetition of the receptive field, and at the same time completes morphological filtering for regional features of different sizes.

[0070] After two key steps, the complex process of dilated convolution is transformed into morphological filtering using a single receptive field on the regional feature map, so that multi-scale context information can be more effectively obtained. Then all outputs are aggregated, batch normalization is performed again through the aggregated feature map, and the point convolution operation merges the features to form the final residual. This residual will be added to the input feature map to form a more comprehensive feature representation.

[0071] In the first step, the feature maps of different region sizes generated by RR are not only related to the dilation rate of the convolution in SR, but also limited by the receptive field required in the current stage. As the number of network layers increases, the receptive field will be increased accordingly, but too large a receptive field, such as a dilation rate exceeding 7, will not produce ideal results. Therefore, the dilation rates of the three dilated convolution branches taken in the semantic residual SR of the DWR module are 1, 3, and 5 respectively.

[0072] like Figure 6 is a schematic diagram of a CGAFusion module. In the CGAFusion module: let X∈R C×H×W Represents the input infrared image features. The input infrared image features are processed by the spatial attention mechanism and the channel attention mechanism to obtain the channel spatial information weight matrix W respectively. s and the channel related information matrix W c ; Among them, the channel space information weight matrix W s and the channel related information matrix W cSatisfy the following formulas respectively:

[0073]

[0074]

[0075] In the formula, c k×k (·) indicates that the convolution kernel size is k×k, max(0,x) indicates the ReLU activation function, [·] indicates the channel-by-channel cascade operation, GAP indicates the global average pooling operation, GMP indicates the global maximum pooling operation, and X S ,X C denote the cross-spatial dimension and channel dimension respectively;

[0076] W c and W s Fusion obtains rough feature information W coa =W c +W s ; W coa Shuffle and rearrange each channel of X to obtain refined feature information W. The process satisfies:

[0077] W=σ(gc 7×7 (CS([x,W coa ])))

[0078] In the formula, σ represents the sigmoid activation function, CS(·) represents channel shuffle, and gc k×k (·) represents a group convolution kernel of size k×k.

[0079] like Figure 7 As shown, the LSDECD_Detect module adopts the LSDECD detection head to realize the shared use of convolution, adopts the GN group normalization strategy in the convolution, and calculates the mean variance of all channels in groups to achieve normalization.

[0080] The LSDECD_Detect module includes: a first Conv_GN module, a second Conv_GN module, a third Conv_GN module, a first DEConv-GN module, a second DEConv-GN module, a first Conv_Reg module, a first Conv_Cls module, a second Conv_Reg module, a second Conv_Cls module, a third Conv_Reg module, a third Conv_Cls module, a first Scale module, a second Scale module and a third Scale module. The output ends of the first Conv_GN module, the second Conv_GN module and the third Conv_GN module are all connected to the first DEConv-GN module; the output end of the first DEConv-GN module is connected to the second DEConv-GN module; the output end of the second DEConv-GN module is respectively connected to the first Conv_Reg module, the first Conv_Cls module, the second Conv_Reg module, the second Conv_Cls module, the third Conv_Reg module and the third Conv_Cls module; the output end of the first Conv_Reg module is connected to the first Scale module; the output end of the second Conv_Reg module is connected to the second Scale module; the output end of the third Conv_Reg module is connected to the third Scale module.

[0081] The use of the DEConv module can extract richer features without incurring additional computing costs and memory burdens, meeting the requirements of the lightweight detection head of the present invention, improving the ability to capture infrared details, and improving the accuracy of the lightweight detection head. Due to the addition of the DEConv module, the regression task will have the problem of inconsistent target scales for each detection head. The Scale module is added after the corresponding module to scale the features, making the model more stable.

[0082] During training, the ATFL adaptive threshold focus loss function is used as the classification loss function of the improved lightweight YOLOv8 model; the PIoUv2 loss function is used as the classification regression function of the improved lightweight YOLOv8 model.

[0083] In order to solve the problem that the background features in infrared images occupy too much space, the main part of the image will dominate the gradient update direction, and some target features in training are characterized as difficult samples, the binary cross entropy (BCE) loss function of the original classification loss is improved. This function can decouple the target features and the background, so that the improved model can allocate extra attention to the target features. The specific implementation is as follows:

[0084] First, threshold setting is used to decouple the easily identifiable background and the difficult-to-identify target. Then, by strengthening the loss associated with the target and reducing the loss associated with the background, the model allocates more attention to the target features. Finally, an adaptive mechanism is applied within the hyperparameters to improve real-time performance. Since BCE cannot handle the problem of sample imbalance, the focal loss function adds a modulation factor (1-p t ) γ , the relevant formula is as follows:

[0085]

[0086]

[0087] FL(p t )=-(1-p t ) γ log(p t )

[0088] In the formula, p represents the predicted probability, y represents the true label, and the focusing parameter γ is adjusted to reduce the contribution of easy-to-classify samples.

[0089] However, the traditional focal loss function also has the problem of difficulty in learning difficult samples. To address this, a threshold mechanism is added, and the prediction probability above 0.5 is defined as a simple sample and the prediction probability below this threshold is defined as a difficult sample. At the same time, an adaptive mechanism is used to express it as the following formula, where λ is a hyperparameter:

[0090]

[0091] The original classification regression function of the YOLOv8 model is changed using the PIoUv2 loss function. The Powerful-IoU (PIoU) loss is adopted, which uses a size-adaptive penalty factor to guide the anchor box to regress directly and efficiently to the target box. The core advantage of the PIoU loss is that by combining the penalty factor with the gradient adjustment function of the anchor box quality, the distance between the four sides of the anchor box and the corresponding sides of the target box is directly minimized, thereby significantly improving the efficiency and accuracy of the regression. The PIoU loss function can guide the anchor box to move toward the target box in an almost straight path, effectively avoiding unnecessary anchor box enlargement. At the same time, compared with the CIoU loss function, the PIoU loss shows a faster convergence speed during training. The penalty factor P is defined as follows:

[0092]

[0093] Where dw1, dw2, dh1 and dh2 represent the absolute distances between the corresponding edges of the prediction box and the target box, and w gt and h gtRepresents the width and height of the target box. Using P to represent the penalty factor of the loss function will not increase the size of the anchor box, because its denominator only contains the target box parameters, and P also represents the adaptability to the target size. Finally, the regression function of the improved model adopts PIoU v2, which applies the focus mechanism to PIoU. This function is obtained by combining a non-monotonic attention function with PIoU controlled by a single hyperparameter. By adding the attention layer, the ability of the loss function to focus on medium and high-quality anchor boxes has been significantly improved, as follows:

[0094]

[0095] PIoU=IoU-f(P),-1<=PIoU<=1

[0096] L PIoU =1-PIou=L IoU +f(P),0<=L PIoU <=2

[0097] Q=e -P ,q∈(0,1]

[0098]

[0099] In the formula, q is used to replace the traditional penalty factor P, where the value range of q is (0,1], which is used to quantify the quality of the anchor box. When q reaches its maximum value of 1, it means that the anchor box is completely aligned with the target box, and the corresponding penalty factor P is 0. As P increases (that is, the alignment between the anchor box and the target box decreases), the value of q gradually decreases, reflecting the decrease in the quality of the anchor box. Through its unique design, PIoU strengthens its ability to focus on medium-quality anchor boxes during regression. In addition, PIoU v2 only requires a hyperparameter λ to control the behavior of the attention function, and the parameter value selected in this model is 1.3.

[0100] Embodiment 2

[0101] In this embodiment, the improved lightweight YOLOv8 model is trained using the training set, and during the training process, the effectiveness of the improved lightweight YOLOv8 model is verified using the verification set.

[0102] Training process:

[0103] Initialize various parameters of the improved lightweight YOLOv8 model; input the infrared image in the preprocessed infrared image data set into the improved lightweight YOLOv8 model for recognition to obtain category information and detection box; input the above information together with the annotation information of the infrared image into the optimized loss function to calculate the loss value.

[0104] The training round is set to 200 epochs and the batch is 16, and the improved lightweight YOLOv8 model after training is obtained. The superiority of the improved infrared detection network is evaluated by Precision, Recall and mAP (average precision), and the training results of the improved infrared detection network are compared with the original network without improvement. The results show that the lightweight YOLOv8 model shows better accuracy and real-time performance in infrared target detection, as shown in Table 1.

[0105] Table 1

[0106]

[0107] The infrared test set images to be detected are input into the trained lightweight YOLOv8 model for recognition to obtain the detection results, such as Figure 8 As shown, the infrared image includes various target categories in the traffic scene, which are marked with rectangular boxes in the image and the corresponding confidence levels are indicated.

[0108] Embodiment 3

[0109] Fig. 9 For this embodiment, a nighttime infrared detection system based on an improved lightweight YOLOv8 model is proposed, including:

[0110] A data acquisition and processing module is used to acquire an infrared image data set, preprocess the infrared image data set, and divide the preprocessed infrared image data set into a training set, a verification set, and a test set;

[0111] The acquired infrared image datasets include: FLIR raw dataset and Multi-spectral dataset. Invalid images and images with few categories in the FLIR raw dataset are deleted, and the number of categories in the Multi-spectral dataset remains unchanged; the preprocessed FLIR raw dataset and Multi-spectral dataset are divided into training set, validation set and test set in the ratio of 8:1:1.

[0112] A model building module, used to build an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a lightweight detection head module on the basis of the YOLOv8 model;

[0113] The improved lightweight YOLOv8 model consists of three parts: Backbone network, Neck network and Head network;

[0114] The Backbone network includes: a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a first C2f_DWR module, a fifth Conv module, a second C2f_DWR module and a first SPPF module connected sequentially from top to bottom;

[0115] The Neck network includes: a first Upsample module, a first Concat module, a third C2f module, a second Upsample module, a second Concat module and a fourth C2f module connected sequentially from bottom to top; a sixth Conv module, a third Concat module, a fifth C2f module, a seventh Conv module, a fourth Concat module and a sixth C2f module connected sequentially from top to bottom; a first CGAFusion module, a second CGAFusion module and a third CGAFusion module; the output end of the second C2f module is connected to the first Concat module and the first CGAFusion module respectively, the output end of the first C2f_DWR module is connected to the second CGAFusion module, the output end of the first SPPF module is connected to the first Upsample module, the fourth Concat module and the third CGAFusion module respectively, the output end of the fourth C2f module is connected to the sixth Conv module and the first CGAFusion module respectively, the output end of the fifth C2f module is connected to the second CGAFusion module, and the output end of the sixth C2f module is connected to the third CGAFusion module;

[0116] The head network includes a LSDECD_Detect module; output ends of the first CGAFusion module, the second CGAFusion module and the third CGAFusion module are all connected to the LSDECD_Detect module.

[0117] The training module uses the training set to train the improved lightweight YOLOv8 model, and during the training process, uses the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model;

[0118] Initialize the parameters of the improved lightweight YOLOv8 model; input the infrared images in the preprocessed infrared image dataset into the improved lightweight YOLOv8 model for recognition, obtain category information and detection box; input the above information together with the annotation information of the infrared image into the optimized loss function, and calculate the loss value. Set the training round z to 200 epochs and the batch to 16, and obtain the trained improved lightweight YOLOv8 model.

[0119] The detection module is used to input the test set into the trained improved lightweight YOLOv8 model to obtain the infrared detection results.

Claims

1. A nighttime infrared detection method based on an improved lightweight YOLOv8 model, characterized in that: The following steps are involved: S1: Acquire an infrared image dataset, preprocess the infrared image dataset, and divide the preprocessed infrared image dataset into a training set, a validation set, and a test set; S2: construct an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a LSDECD_Detect module on the basis of the YOLOv8 model; S3: Use the training set to train the improved lightweight YOLOv8 model, and during the training process, use the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model; S4: Input the test set into the trained improved lightweight YOLOv8 model to obtain the infrared detection results.

2. According to claim 1, a nighttime infrared detection method based on an improved lightweight YOLOv8 model is characterized in that: The acquired infrared image datasets include: FLIR raw dataset and Multi-spectral dataset; The preprocessing operations on the infrared image dataset are as follows: the invalid images and images with few categories in the FLIR original dataset are deleted, and the number of categories in the Multi-spectral dataset remains unchanged; the preprocessed FLIR original dataset and Multi-spectral dataset are divided into training set, validation set and test set in the ratio of 8:1:1, respectively.

3. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 1 is characterized in that: The improved lightweight YOLOv8 model consists of three parts: Backbone network, Neck network and Head network; The Backbone network includes: a first Conv module, a second Conv module, a first C2f module, a third Conv module, a second C2f module, a fourth Conv module, a first C2f_DWR module, a fifth Conv module, a second C2f_DWR module and a first SPPF module connected sequentially from top to bottom; The Neck network includes: a first Upsample module, a first Concat module, a third C2f module, a second Upsample module, a second Concat module and a fourth C2f module connected sequentially from bottom to top; a sixth Conv module, a third Concat module, a fifth C2f module, a seventh Conv module, a fourth Concat module and a sixth C2f module connected sequentially from top to bottom; a first CGAFusion module, a second CGAFusion module and a third CGAFusion module; the output end of the second C2f module is connected to the first Concat module and the first CGAFusion module respectively, the output end of the first C2f_DWR module is connected to the second CGAFusion module, the output end of the first SPPF module is connected to the first Upsample module, the fourth Concat module and the third CGAFusion module respectively, the output end of the fourth C2f module is connected to the sixth Conv module and the first CGAFusion module respectively, the output end of the fifth C2f module is connected to the second CGAFusion module, and the output end of the sixth C2f module is connected to the third CGAFusion module; The head network includes a LSDECD_Detect module; output ends of the first CGAFusion module, the second CGAFusion module and the third CGAFusion module are all connected to the LSDECD_Detect module.

4. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 3 is characterized in that: The C2f_DWR module comprises: a seventh Conv module, a Split module, an n-layer DWR module and an eighth Conv module connected in sequence; the output ends of the seventh Conv module, the Split module and the n-layer DWR module are spliced ​​and connected to the eighth Conv module.

5. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 3 is characterized in that: The DWR module includes: regional residual RR and semantic residual SR; regional residual RR includes: the ninth Conv module, the first combined batch normalization layer, and the ReLU activation function layer; the semantic residual SR includes: the first DConv module and the second combined batch normalization layer.

6. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 5 is characterized in that: The dilation rates of the three dilated convolution branches taken in the semantic residual SR of the DWR module are 1, 3, and 5 respectively.

7. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 3 is characterized in that: In the CGAFusion module: Let X∈R C×H×W Represents the input infrared image features. The input infrared image features are processed by the spatial attention mechanism and the channel attention mechanism to obtain the channel spatial information weight matrix W respectively. s and the channel related information matrix W c ; Among them, the channel space information weight matrix W s and the channel related information matrix W c Satisfy the following formulas respectively: In the formula, c k×k (·) indicates that the convolution kernel size is k×k, max(0,x) indicates the ReLU activation function, [·] indicates the channel-by-channel cascade operation, GAP indicates the global average pooling operation, GMP indicates the global maximum pooling operation, and X S ,X C denote the cross-spatial dimension and channel dimension respectively; W c and W s Fusion obtains rough feature information W coa =W c +W s ;W coa Shuffle and rearrange each channel of X to obtain refined feature information W. The process satisfies: W=σ(gC7×7(CS([x,Wcoa]))) In the formula, σ represents the sigmoid activation function, CS(·) represents channel shuffle, and gc k×k (·) represents a group convolution kernel of size k×k.

8. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 1 is characterized in that: The LSDECD_Detect module adopts the LSDECD detection head to realize the shared use of convolution, adopts the GN group normalization strategy in the convolution, and calculates the mean variance of all channels in groups to realize normalization.

9. The nighttime infrared detection method based on the improved lightweight YOLOv8 model according to claim 1, characterized in that: During training, the ATFL adaptive threshold focus loss function is used as the classification loss function of the improved lightweight YOLOv8 model; the PIoUv2 loss function is used as the classification regression function of the improved lightweight YOLOv8 model.

10. A nighttime infrared detection system based on an improved lightweight YOLOv8 model, characterized in that: include: A data acquisition and processing module is used to acquire an infrared image data set, preprocess the infrared image data set, and divide the preprocessed infrared image data set into a training set, a verification set, and a test set; A model building module is used to build an improved lightweight YOLOv8 model; the improved lightweight YOLOv8 model is based on the YOLOv8 model and adds a DWR module for improving the multi-scale and feature extraction capabilities of infrared images, a CGAFusion module for improving the image feature fusion capability, and a lightweight detection head module; The training module uses the training set to train the improved lightweight YOLOv8 model, and during the training process, uses the verification set to verify the effectiveness of the improved lightweight YOLOv8 model to obtain a trained improved lightweight YOLOv8 model; The detection module is used to input the test set into the trained improved lightweight YOLOv8 model to obtain infrared detection results.

Images