Road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion

Abstract

A road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion comprises the following steps: S1, data set preparation: on the basis of an FLIR infrared-visible light data set, constructing a vehicle-mounted scene road infrared target detection data set through data cleaning, filtering, data enhancement and label remapping operation; s2, module replacement: designing a C3MBC module to replace a C3k2 module in the network backbone; s3, reconstructing a neck network, designing a small target feature enhancement module STFEM to be applied to the neck of the network, and reconstructing the neck network; s4, optimizing a down-sampling structure, and replacing a down-sampling structure before P4 and P5 detection heads in the neck network with a down-sampling structure combining CBS and MP; and S5, updating a loss function, and introducing PIOUv2 as a bounding box regression loss function in the positioning task. The invention aims to provide a solution for the detection problems of fuzzy features, numerous small targets, multi-scale and diversity of the targets and the like in a vehicle-mounted scene.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion. Background Technology

[0002] In recent years, autonomous driving technology has become closely intertwined with people's lives. How to quickly and effectively detect traffic participants in in-vehicle scenarios is a key challenge for autonomous driving technology. Using deep learning algorithms for object detection is a common method to address this issue. However, with the continued increase in the density of road traffic participants and the frequent occurrence of extreme weather events due to El Niño and La Niña phenomena, the complexity of driving scenarios has further increased. This has led to deep learning algorithms trained using visible light image data becoming insufficient to meet the demands of all-weather object detection and object detection in complex driving scenarios such as severe weather.

[0003] Infrared images are characterized by their independence from light sources and ability to effectively penetrate visual interference, making them highly capable of target detection in all-weather and complex driving scenarios, including adverse weather conditions. However, when using infrared scanners for thermal imaging, the imaging distance between the device and the target is generally quite far. This results in a high number of small-sized targets appearing in the obtained infrared images. Furthermore, because infrared scanners cannot identify the external color and internal texture of the target, the infrared images obtained by thermal imaging technology have low contrast in environments with uniform temperature. In such cases, the brightness of the target is similar to the background, leading to the loss of internal features and blurring of characteristics. This makes target detection of road traffic participants using infrared images in vehicular scenarios extremely difficult.

[0004] To address the low target detection accuracy issue in infrared target detection, researchers both domestically and internationally have conducted extensive research. In 2016, Y. Wei et al. proposed a contrast measurement algorithm based on multi-scale blocks, which enhances multi-scale target features by utilizing the local contrast between the target and the background to solve the feature blurring problem and improve infrared target detection accuracy. In 2019, Han J et al. proposed a method using adaptive background estimation to optimize the local contrast of infrared images, thereby improving the accuracy of infrared small target detection. In 2020, Moradi S et al. constructed a new directional small target detection algorithm, Absolute Directional Average Difference, using a similar mean absolute grayscale difference, which effectively enhanced the target region features and, to some extent, eliminated the influence of background clutter on detection accuracy. In 2024, Liu et al. proposed Trans-YOLO, an improved YOLOv8 model based on RT-DETR, which replaces the YOLOv8 Head with the Decoder & Head in the RT-DETR network, and designs a CAFMFusion mechanism as a new feature fusion strategy between the neck network and the detection head. In the same year, Ni Mengqi et al. proposed a lightweight infrared small target detection model, YOLO-IRLight, based on YOLOv8s. They introduced an effective multi-scale EMA attention mechanism to improve feature extraction capabilities, and added a P2 detection layer at the neck of the network to improve the accuracy of small target detection.

[0005] In summary, using both traditional and deep learning algorithms for infrared target detection in vehicular scenarios has effectively improved target detection accuracy. While researchers have proposed some improved detection schemes to address challenges such as the large number of small targets and feature ambiguity in current infrared target detection methods, existing schemes still suffer from high model complexity, insufficient accuracy in infrared target detection, and an inability to effectively balance real-time performance and accuracy. Therefore, proposing more effective algorithms to address the challenges of infrared detection in vehicular scenarios is of practical significance. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion, which aims to provide a solution to the detection problems in vehicle scenarios such as feature ambiguity, numerous small targets, and targets with multiple scales and diversity. The invention proposes a road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion, namely EITD-YOLO.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion, comprising the following steps: S1. Dataset preparation: Based on the FLIR infrared-visible light dataset, a vehicle scene road infrared target detection dataset is constructed through data cleaning, filtering, data augmentation, and label remapping operations. S2. Module replacement: Design the C3MBC module to replace the C3k2 module in the network backbone; S3. Neck network reconstruction: A small target feature enhancement module (STFEM) is designed and applied to the neck of the network to reconstruct the neck network. S4. Downsampling structure optimization: Replace the downsampling structure before the P4 and P5 detection heads in the neck network with a downsampling structure that combines CBS and MP. S5. Loss function update: PIOUv2 is introduced as the bounding box regression loss function in the localization task. S6. Model Training and Testing: Input the constructed dataset into the improved EITD-YOLO model according to the preset training strategy for training, and use the test set to evaluate the model performance.

[0008] Preferably, in step S1, the filtering effect is evaluated using the peak signal-to-noise ratio (PSNR) and the similarity structure index (SSIM). The formulas for calculating PSNR and SSIM are as follows: ; ; in, Indicates the maximum pixel value of the image. This represents the mean squared error between the pixels of the original image and the filtered image. A higher value indicates better image quality and less noise, but may result in over-smoothing. , as well as These represent the brightness contrast, contrast contrast, and structural contrast of the images before and after filtering, respectively. Multiplying these three values ​​yields the similarity structure index. , The range of values ​​is Furthermore, the closer the image is to the boundary, the closer it is to the boundary.

[0009] Preferably, the infrared images that have undergone data preprocessing are relabeled and divided into training, testing, and validation sets in a 7:2:1 ratio for model training and testing.

[0010] Preferably, the construction process of the C3MBC module includes: S201: The traditional residual module bottleneck inside the C3k2 module is replaced by the MBConv structure. The core of the MBConv structure is the inverse residual IRB design combined with the depthwise separable convolution DWConv. S202: An effective channel attention (eSE) mechanism is added to the IRB structure containing DWConv to recalibrate the feature channel weights by dynamically modeling the inter-channel dependencies.

[0011] Preferably, the data processing procedure of the MBConv structure satisfies: ; ; ; ; ; Where X represents the initial input feature map X^(C×H×W) of the MBConv module, and Wp1 and bp1 represent the pointwise convolution weights and biases of the first layer, respectively. The first layer of pointwise convolution outputs the feature map, where the number of channels is expanded from C to C×factor, and H and Remain unchanged; and They represent Depthwise convolution (DW) weights and biases This represents the output of the depthwise convolutional (DW) layer. correspond , as well as All maintain with Consistent; Wp2 and bp2 represent the pointwise convolution weights and biases of the second layer, respectively. The second layer of pointwise convolution outputs a feature map, where the number of output channels is compressed to Cout. Indicates to among The output feature map after the channel feature map has been regularized; This indicates that the MBConv module ultimately outputs a feature map when residual connections exist and the number of channels in the input feature map differs from the number of channels in the output feature map of the second-layer pointwise convolution. When in a relationship, equal to the initial input feature map The output feature map after regularization Add them together, otherwise they are equal. .

[0012] Preferably, the effective channel attention (eSE) mechanism is as follows: ; in, Representative input channel feature map, This represents the output compressed vector obtained through global pooling, also known as the channel descriptor. ; in, This represents the weights of a fully connected layer. This represents the Hard sigmoid activation function. This indicates that the weight is After linear transformation of the fully connected layer, and applying the Hard sigmoid activation function to generate... The corresponding channel attention weights; ; in, Indicates the use of channel attention weights Input channel feature map The weighted feature map is obtained by recalibrating.

[0013] Preferably, the operation of the STFEM module includes: S301. Use Space to Depth Convolution (SPD-Conv) to process the feature maps from the P2 feature layer of the backbone network to achieve feature extraction of small targets. S302. Design the CSPOKM module for feature extraction and feature fusion. The features rich in small target information obtained in the previous step are concatenated with the features of the P3 feature layer and fed into the CSPOKM module, which adopts a CSP (Cross Stage Partial) structure and combines a full-core module OKM (Omni-Kernel Module), for feature extraction and feature fusion. Finally, the fused feature map containing rich small target features and multi-scale features is transmitted to the P3 detection head to achieve small target detection.

[0014] Preferably, the data processing procedure implemented by SPD-Conv is as shown in the following formula: ; in, This is the initial feature map. As a scaling factor, To pass The resulting sub-feature maps are composed of all features that satisfy... and Divisible by all of Composition; the process of dividing the feature map into sub-feature maps is equivalent to... Proportion Perform downsampling; The CSPOKM module and the OKM module data processing flow are shown in the following formula.

[0015] ; in, This indicates the final output feature map of the CSPOKM module. This represents the initial output feature map of the CSPOKM module. This represents the feature map after it has been partitioned and processed by the OKM module. This indicates the division ratio, which is 0.25 in this case. This represents the sub-feature map before OKM processing, which here accounts for 0.75 of the initial input feature map. Indicates to as well as Perform the splicing operation.

[0016]

[0017] in, This represents the output feature map after processing by the full-core module OKM. , as well as Representing sub-feature maps The output feature map after processing by the three branches within the OKM module.

[0018] Preferably, step S4 employs the V7DownSample downsampling structure and applies it before the network neck P4 and P5 detection heads; the V7DownSample is internally designed as a dual-branch parallel downsampling design, which first uses a step size of 2 and a size of The MaxPool max pooling branch preserves salient features in the feature map, while the CBS convolution branch, composed of convolutional modules, BN modules, and SiLU activation functions, learns complex features in the feature map. Finally, Concat is used to concatenate the outputs of the two branches to achieve better feature fusion.

[0019] Preferably, the formula for calculating the PIoUv2 loss function in step S5 is as follows: ; in, This represents the adaptive penalty factor calculated based on the sizes of the predicted bounding box and the ground truth bounding box. This represents the calculated parameters for measuring the quality of the anchor frame. This represents the non-monotonic attention function in a non-monotonic attention layer. Indicates control A single hyperparameter of shape, which will be used in this invention Set to 1.3.

[0020] Preferably, step S6 involves training and testing the vehicle-mounted scene road infrared target detection dataset using the improved EITD-YOLO model, specifically including the following sub-steps: S601: The dataset is configured according to a specific training strategy for the EITD-YOLO model to read and train. The training strategy of this invention is as follows: the image size input to the network is uniformly set to 640×640; the training epochs are 200; the batch size is set to 32; the number of data loading processes (workers) is set to 4; the loss function optimizer is a stochastic gradient optimizer (SGD); automatic mixing precision (amp) is disabled; the initial learning rate used by the model is set to 0.01, the momentum parameter is set to 0.937, and the weight decay parameter is set to 0.0005; the model's built-in Mosaic and MixUp data augmentation weights are set to 1.0 and 0.0 respectively, and Mosaic data augmentation is enabled throughout the training process.

[0021] S602: After training, the EITD-YOLO model is evaluated using a test set. To verify the performance of the improved model, this invention uses precision (P), recall (R), mean average precision (mAP), average precision (AP), number of model parameters (Params), and gigabit floating-point operations per second (GFLOPs) as evaluation metrics.

[0022] This invention provides a road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion, which has the following beneficial effects: 1. This invention effectively improves the accuracy of infrared target detection in vehicle-mounted scenarios, achieving an accuracy rate of 74.3% and an average detection accuracy of 61.5%. Its detection performance surpasses that of many mainstream target detection algorithms, better meeting the detection needs of vehicle-mounted scenarios. By replacing the C3k2 module in the network backbone with a C3MBC module, and combining an inverse residual structure, depthwise separable convolution, and an effective channel attention mechanism, the feature channel weights are dynamically recalibrated, significantly enhancing the feature extraction capability of blurred objects in low-contrast infrared images and solving the problem of feature loss in infrared images.

[0023] 2. A specially designed small target feature enhancement module, STFEM, is applied to the network neck. It extracts small target features through spatial-to-depth convolution and combines it with the CSPOKM module to achieve feature fusion, effectively improving the problem of small target feature loss during the sampling process of traditional neck networks and significantly improving the small target detection performance. The V7DownSample dual-branch downsampling structure is adopted, which retains salient features in the feature map while learning complex features. This reduces feature loss during downsampling and lowers the number of model parameters, resulting in a model with only 3.04M parameters, achieving lightweight model design.

[0024] 3. The PIOUv2 loss function is introduced, and through a non-monotonic attention function and adaptive penalty factor, it adapts to the multi-scale characteristics and aspect ratio differences of targets in vehicle scenarios, improving the accuracy of target localization and alleviating the difficulty of localization in multi-scale target detection. The model's gigabit floating-point operations per second are controlled within a reasonable range, achieving a balance between detection accuracy and real-time performance, and meeting the requirements of real-time detection in vehicle scenarios.

[0025] 4. The constructed vehicle-mounted scene road infrared target detection dataset has undergone data cleaning, filtering, and enhancement processes. The data processing effect has been verified by peak signal-to-noise ratio and similarity structure index. The dataset is robust, which improves the model's adaptability to complex driving scenarios such as extreme weather and complex road conditions, and further ensures the stability of detection performance. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a diagram showing the overall structure of the EITD-YOLO algorithm proposed in this invention. Figure 2 This is a flowchart of the method of the present invention; Figure 3 This is a structural diagram of the C3MBC module proposed in this invention; Figure 4 This diagram shows the neck network structure including the Small Target Feature Enhancement Module (STFEM) proposed in this invention. Figure 5 A diagram showing the neck network structure including the V7DownSample structure used in this invention; Figure 6 This is a comparison chart showing the detection performance of the baseline model YOLOv11 and the improved EITD-YOLO model on the test set of the dataset created in S1 of this invention. Detailed Implementation

[0027] This invention designs a road infrared target detection algorithm based on feature adaptive enhancement and multi-scale feature fusion. The algorithm is implemented in Python version 3.10, using YOLOv11n as the baseline model. The overall network architecture of the improved EITD-YOLO algorithm is as follows. Figure 1 As shown. Implementing road target detection in vehicle-mounted scenes using deep learning algorithms requires sufficient relevant data for algorithm training. This invention filters and processes raw infrared images from the FLIR infrared-visible light dataset from FLIR Systems to obtain the dataset used in this invention, which is then used as the training dataset for the baseline model YOLOv11n. By improving the YOLOv11n model, the improved EITD-YOLO algorithm is obtained. Finally, EITD-YOLO is trained on the dataset of this invention to obtain weight parameters for testing detection performance.

[0028] This invention proposes a road infrared target detection algorithm based on feature adaptive enhancement and multi-scale feature fusion. The design flowchart is as follows: Figure 2 As shown, it includes the following steps: S1. Dataset preparation: Data cleaning, enhancement, and filtering are used to improve data robustness and enhance the representation of small target features in order to construct a vehicle-mounted road infrared target detection dataset. Preferably, the process S1 involves using the FLIR infrared-visible light dataset as a sample and constructing a road infrared target detection dataset for vehicle scenarios through the system's data processing flow, for subsequent model training and testing. Specifically, it includes the following sub-steps: S101: The dataset samples of this invention come from the FLIR infrared-visible light dataset open sourced by FLIR Systems. This dataset contains 11,886 infrared images of road targets, including 16 common road traffic participants such as pedestrians, bicycles, cars, motorcycles, buses, trams, trucks, traffic lights, fire hydrants, road signs, dogs, deer, skateboards, strollers, scooters, and other vehicles.

[0029] S102: This invention uses the FLIR infrared-visible light dataset as a sample, and performs data cleaning, filtering, and data enhancement on the infrared images in the original dataset to obtain processed infrared images. In the filtering operation, two metrics, Peak Signal-to-Noise Ratio (PSNR) and Similarity Structure Index (SSIM), are used to evaluate the filtering effect. The formulas for calculating PSNR and SSIM are as follows:

[0030]

[0031] in, Indicates the maximum pixel value of the image. This represents the mean squared error between the pixels of the original image and the filtered image. A higher value indicates better image quality and less noise, but may result in over-smoothing. , as well as These represent the brightness contrast, contrast contrast, and structural contrast of the images before and after filtering, respectively. Multiplying these three values ​​yields the similarity-structure index. , The range of values ​​is Furthermore, the closer the image is to the boundary, the closer it is to the boundary.

[0032] S103: The infrared images that have undergone data preprocessing are relabeled to obtain a vehicle scene road infrared target detection dataset consisting of 11,598 infrared images, including labels for 8 common road detection objects such as pedestrians, bicycles, cars, motorcycles, buses, traffic lights, fire hydrants, and road signs. The dataset is then divided into training, testing, and validation sets in a 7:2:1 ratio for model training and testing.

[0033] S2. Design the C3MBC module to replace the C3k2 module in the network backbone. The C3MBC module structure is as follows: Figure 3 As shown; Preferably, the process of S2 involves designing a C3MBC (C3k2 Mobile Inverted Bottleneck Convolution) module to replace the C3k2 module in the network backbone, specifically comprising the following sub-processes: S201: First, the traditional residual module bottleneck inside the original C3k2 module is replaced with the MBConv (Mobile Inverted Bottleneck Convolution) structure. The core of the MBConv structure uses an inverted residual (IRB) design to mitigate gradient vanishing from feature maps and combines it with depthwise separable convolution (DWConv) to reduce computational complexity and enhance feature representation. The corresponding data processing can be represented by the following formula:

[0034]

[0035]

[0036]

[0037]

[0038] Where X represents the initial input feature map X^(C×H×W) of the MBConv module, Wp1 and bp1 represent the weights and biases of the first-layer pointwise convolution, respectively, and X1 is the output feature map of the first-layer pointwise convolution. At this time, the number of channels is expanded from C to C×factor, and H and Remain unchanged; and They represent Depthwise convolution (DW) weights and biases This represents the output of the depthwise convolutional (DW) layer. correspond , as well as All maintain with Consistent; Wp2 and bp2 represent the weights and biases of the second-layer pointwise convolution, respectively, and X3 is the output feature map of the second-layer pointwise convolution. At this time, the number of output channels is compressed to Cout. Indicates to among The output feature map after the channel feature map has been regularized; This indicates that the MBConv module ultimately outputs a feature map when residual connections exist and the number of channels in the input feature map differs from the number of channels in the output feature map of the second-layer pointwise convolution. When in a relationship, equal to the initial input feature map The output feature map after regularization Add them together, otherwise they are equal. .

[0039] S202: Next, an effective channel attention (eSE) mechanism is introduced and applied to the inverse residual IRB containing the MBConv structure. The eSE mechanism dynamically recalibrates the feature channel weights by adaptively and dynamically modeling the inter-channel dependencies to enhance important feature channels. The corresponding data processing is shown in the following formula. Finally, the MBConv structure used in this invention is obtained, achieving better feature extraction performance for feature-fuzzy objects compared to the C3k2 module.

[0040]

[0041] in, Representative input channel feature map, This represents the output compressed vector obtained through global pooling, also known as the channel descriptor.

[0042]

[0043] in, This represents the weights of a fully connected layer. This represents the Hard sigmoid activation function. This indicates that the weight is After linear transformation of the fully connected layer, and applying the Hard sigmoid activation function to generate... The corresponding channel attention weights.

[0044]

[0045] in, Indicates the use of channel attention weights Input channel feature map The weighted feature map is obtained by recalibrating.

[0046] S3. Design a Small Target Feature Enhancement Module (STFEM) to be applied to the network neck to reconstruct the neck network. The reconstructed neck network structure including the STFEM structure is as follows: Figure 4 As shown; Preferably, the S3 process involves designing a small target feature enhancement module (STFEM) and applying it to the network neck. This module uses a novel convolutional structure to avoid detail loss during traditional downsampling and pooling processes. Simultaneously, a multi-scale feature learning module is designed to enhance the model's ability to learn multi-scale features, achieving better multi-scale feature extraction and fusion. Specifically, this includes the following sub-processes: S301: First, the feature map of the backbone network P2 feature layer is processed using Space-to-Depth Convolution (SPD-Conv) to achieve shallow feature extraction of small targets. When using the SPD-Conv structure for arbitrary sizes... Feature map The process for partitioning is shown in the following formula.

[0047]

[0048] in, This is the initial feature map. As a scaling factor, Indicates passage The resulting sub-feature maps are composed of all features that satisfy... and Divisible by all of Composition, the entire process of dividing the feature map into sub-feature maps is equivalent to... Proportion Perform downsampling.

[0049] S302: Then, the CSP (Cross Stage Partial) structure is adopted, combined with the Omni-Kernel Module (OKM) to design the CSPOKM module for feature extraction and feature fusion. In this step, the features rich in small target information obtained in the previous step are concatenated with the features from the P3 feature layer and fed into the CSPOKM module. This results in a fused feature map containing rich small target features and multi-scale features, which is then transmitted to the P3 detection head, ultimately achieving small target detection. The data processing flow of the CSPOKM module and the OKM module is shown in the following formula.

[0050]

[0051] in, This represents the feature map that is finally output by the CSPOKM module. This represents the initial output feature map of the CSPOKM module. This indicates the division ratio; in this invention, this value is set to 0.25. This represents the feature map after it has been partitioned and processed by the OKM module; therefore, it occupies 0.25 of the initial input feature map. This represents the sub-feature map before OKM processing, therefore it occupies 0.75 of the initial input feature map. Indicates to as well as Perform the splicing operation.

[0052]

[0053] in, This represents the feature map output by the OKM module. , as well as Representing sub-feature maps The output feature map after processing by the three branches within the OKM module.

[0054] S4. Replace the downsampling modules before the P4 and P5 detection heads with the V7DownSample downsampling structure combining CBS and MP. The V7DownSample structure is as follows: Figure 5 As shown; Preferably, the process S4 involves setting the step size of the original channels (256 and 512) to 2, and the size to be... The strided convolutional downsampling module was replaced with the V7DownSample structure, which is internally designed as a dual-branch parallel downsampling module. It first uses a stride of 2 and a size of [missing information]. The MaxPool maximum pooling branch preserves salient features in the feature map, while the CBS convolution branch, composed of convolutional modules, BN modules, and SiLU activation functions, learns complex features in the feature map. Finally, Concat is used to concatenate the outputs of the two branches to achieve feature fusion.

[0055] S5. Introduce the boundary regression loss function PIoUv2, which combines a non-monotonic attention layer, as the bounding box regression loss function in the localization task. Furthermore, step S5 involves introducing the PIoUv2 loss function to enhance the model's attention to medium-quality anchor boxes, while simultaneously adapting to anchor boxes of different sizes. This alleviates the localization difficulties caused by the diverse scales and significant aspect ratio differences of targets detected in vehicular scenes. The PIoUv2 loss function is calculated as follows:

[0056] in, This represents the adaptive penalty factor calculated based on the sizes of the predicted bounding box and the ground truth bounding box. This represents the calculated parameters for measuring the quality of the anchor frame. This represents the non-monotonic attention function in a non-monotonic attention layer. Indicates control A single hyperparameter of shape, which will be used in this invention Set to 1.3.

[0057] S6. Train and test the improved EITD-YOLO model on the dataset, and finally obtain a comparison of the actual detection accuracy before and after the improvement. Figure 6 As shown.

[0058] S601: Configure the dataset created in S1 according to a specific training strategy for the EITD-YOLO model to read and train. The training strategy of this invention is as follows: the image size input to the model for training is uniformly set to 640×640; the training epochs are 200; the batch size is set to 32; the number of data loading processes (workers) is set to 4; the loss function optimizer is the stochastic gradient optimizer (SGD); automatic mixing of precision (amp) is disabled; the initial learning rate used by the model is set to 0.01, the momentum parameter is set to 0.937, and the weight decay parameter is set to 0.0005; the model's built-in Mosaic is set to 1.0, and Mosaic data augmentation is enabled throughout the training process.

[0059] S602: After training, the EITD-YOLO model is evaluated using a test set. To verify the performance of the improved EITD-YOLO model, this invention uses precision (P), recall (R), mean average precision (mAP), average precision (AP), number of model parameters (Params), and gigabit floating-point operations per second (GFLOPs) as evaluation metrics for detection accuracy.

[0060]

[0061] This invention constructs a vehicle-mounted road infrared target detection dataset based on the FLIR infrared-visible light dataset through data cleaning, filtering, enhancement, and label remapping. The training, testing, and validation sets are divided in a 7:2:1 ratio. The C3MBC module replaces the C3k2 module in the network backbone. The feature extraction capability of blurred objects is enhanced by combining inverse residual structure depthwise separable convolution and effective channel attention mechanism. The STFEM module is proposed and applied to the network neck. The spatial-to-depth convolution and CSPOKM module improve the detection effect of small targets. The V7DownSample dual-branch downsampling structure is adopted to reduce feature loss and reduce the number of parameters. The PIOUv2 loss function is introduced to adapt to multi-scale targets and aspect ratio differences to improve localization accuracy. Finally, the improved EITD-YOLO model is formed, achieving a balance between detection accuracy and real-time performance.

[0062] This invention addresses the challenges of road target detection in vehicular scenarios. First, it replaces the C3k2 module in the network backbone with a C3MBC module, dynamically updating feature channel weights to highlight important feature channels and enhancing feature extraction of blurred objects in low-contrast infrared images. Second, it proposes a small target feature enhancement module, STFEM, and applies it to the network neckline to improve the loss of small target features during traditional neckline upsampling and enhance the multi-scale feature fusion capability of the neckline network, thereby improving the model's detection performance for small infrared targets. Third, it replaces the neckline network downsampling with a downsampling structure combining CBS and MP from the YOLOv7 network, further avoiding feature loss during downsampling and reducing the number of model parameters. Finally, considering the scale diversity of targets to be detected in vehicular scenarios, it introduces the boundary regression loss function PIOUv2, which combines a non-monotonic attention layer, to replace the original loss function CIoU. PIOUv2 combines a gradient adjustment function based on anchor box quality and a target scale adaptive penalty factor, achieving better multi-scale target focusing. In summary, the EITD-YOLO algorithm proposed in this invention effectively improves the accuracy of infrared target detection on roads in vehicle-mounted scenarios, and has a better detection effect, especially for objects with blurred features and small targets.

[0063] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion, characterized in that, Includes the following steps: S1. Dataset preparation: Based on the FLIR infrared-visible light dataset, a vehicle scene road infrared target detection dataset is constructed through data cleaning, filtering, data augmentation, and label remapping operations. S2. Module replacement: Design the C3MBC module to replace the C3k2 module in the network backbone; S3. Neck network reconstruction: The small target feature enhancement module STFEM is designed and applied to the network neck to reconstruct the neck network. S4. Downsampling structure optimization: Replace the downsampling structure before the P4 and P5 detection heads in the neck network with a downsampling structure that combines CBS and MP. S5. Loss function update: PIOUv2 is introduced as the bounding box regression loss function in the localization task. S6. Model Training and Testing: Input the constructed dataset into the improved EITD-YOLO model according to the preset training strategy for training, and use the test set to evaluate the model performance.

2. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1, characterized in that, In step S1, the filtering effect is evaluated using the peak signal-to-noise ratio (PSNR) and the similarity structure index (SSIM). The formulas for calculating PSNR and SSIM are as follows: ； ； in, Indicates the maximum pixel value of the image. This represents the mean squared error between the pixels of the original image and the filtered image. , as well as These represent the brightness comparison, contrast comparison, and structure comparison of the images before and after filtering, respectively.

3. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1 or 2, characterized in that, The preprocessed infrared images are relabeled and divided into training, testing, and validation sets in a 7:2:1 ratio for model training and testing.

4. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1, characterized in that, The construction process of the C3MBC module includes: S201: The traditional residual module bottleneck inside the C3k2 module is replaced by the MBConv structure. The core of the MBConv structure is the inverse residual IRB design combined with the depthwise separable convolution DWConv. S202: An effective channel attention (eSE) mechanism is added to the IRB structure containing DWConv to recalibrate the feature channel weights by dynamically modeling the inter-channel dependencies.

5. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 4, characterized in that, The data processing procedure of the MBConv structure satisfies: ； ； ； ； ； Where X represents the initial input feature map X^(C×H×W) of the MBConv module, and Wp1 and bp1 represent the pointwise convolution weights and biases of the first layer, respectively. The first layer of pointwise convolution outputs the feature map, where the number of channels is expanded from C to C×factor, and H and Remain unchanged; and They represent Depthwise convolution (DW) weights and biases This represents the output of the depthwise convolutional (DW) layer. correspond , as well as All maintain with Consistent; Wp2 and bp2 represent the pointwise convolution weights and biases of the second layer, respectively. The second layer of pointwise convolution outputs a feature map, where the number of output channels is compressed to Cout. Indicates to among The output feature map after the channel feature map has been regularized; This indicates that the MBConv module ultimately outputs a feature map when residual connections exist and the number of channels in the input feature map differs from the number of channels in the output feature map of the second-layer pointwise convolution. When in a relationship, equal to the initial input feature map The output feature map after regularization Add them together, otherwise they are equal. .

6. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 4, characterized in that, The effective channel attention (eSE) mechanism is as follows: ； in, Representative input channel feature map, This represents the output compressed vector obtained through global pooling, also known as the channel descriptor. ； in, This represents the weights of a fully connected layer. This represents the Hard sigmoid activation function. This indicates that the weight is After linear transformation of the fully connected layer, and applying the Hard sigmoid activation function to generate... The corresponding channel attention weights; ； in, Indicates the use of channel attention weights Input channel feature map The weighted feature map is obtained by recalibrating.

7. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1, characterized in that, The working process of the STFEM module includes: S301. Spatial-to-depth convolution SPD-Conv is used to process the feature map from the P2 feature layer of the backbone network to achieve small target-related feature extraction. S302. Design the CSPOKM module for feature extraction and feature fusion. The features rich in small target information obtained in the previous step are concatenated with the features of the P3 feature layer and sent to the CSPOKM module designed with CSP structure and combined with the full-core module OKM for feature extraction and feature fusion. Finally, the fused feature map containing rich small target features and multi-scale features is transmitted to the P3 detection head to finally realize small target detection.

8. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 7, characterized in that, The data processing procedure implemented by SPD-Conv is shown in the following formula: ； in, This is the initial feature map. As a scaling factor, To pass The resulting sub-feature maps are composed of all features that satisfy... and Divisible by all of Composition; the process of dividing the feature map into sub-feature maps is equivalent to... Proportion Perform downsampling; The CSPOKM module and the OKM module data processing flow are shown in the following formula; ； in, This indicates the final output feature map of the CSPOKM module. This represents the initial output feature map of the CSPOKM module. This represents the feature map after it has been partitioned and processed by the OKM module. This indicates the division ratio, which is 0.25 in this case. This represents the sub-feature map before OKM processing, which here accounts for 0.75 of the initial input feature map. Indicates to as well as Perform the splicing operation; in, This represents the output feature map after processing by the full-core module OKM. , as well as Representing sub-feature maps The output feature map after processing by the three branches within the OKM module.

9. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1, characterized in that, Step S4 employs the V7DownSample downsampling structure and applies it before the network neckline P4 and P5 detection heads. The V7DownSample is internally designed as a dual-branch parallel downsampling design, initially using a step size of 2 and a size of [missing information]. The MaxPool max pooling branch preserves salient features in the feature map, while the CBS convolution branch, composed of convolutional modules, BN modules, and SiLU activation functions, learns complex features in the feature map. Finally, Concat is used to concatenate the outputs of the two branches to achieve better feature fusion.

10. The road infrared target detection method based on feature adaptive enhancement and multi-scale feature fusion according to claim 1, characterized in that, The formula for calculating the PIoUv2 loss function in step S5 is as follows: ； in, This represents the adaptive penalty factor calculated based on the sizes of the predicted bounding box and the ground truth bounding box. This represents the calculated parameters for measuring the quality of the anchor frame. This represents the non-monotonic attention function in a non-monotonic attention layer. Indicates control A single hyperparameter of shape, which will be used in this invention Set to 1.3.

Images