Low-visibility environment pedestrian detection method based on improved YOLOv8n model

Abstract

The invention provides a low-visibility environment pedestrian detection method based on an improved YOLOv8n model. A double-branch fusion attention network is adopted in a backbone network of an original YOLOv8n model, and a CBFuse module and a CBLinear module are introduced for feature fusion between different branches; a feature aggregation and calibration pyramid network is introduced, multi-scale feature fusion is carried out through an up-sampling module, a down-sampling module and a feature aggregation and calibration module, and the feature aggregation and calibration module carries out feature calibration and enhancement through a local attention mechanism, a global attention mechanism and a pixel attention mechanism; an adaptive task alignment detection head is introduced to execute a dynamic convolution mechanism, a task decomposition mechanism and a dynamic feature alignment mechanism; an improved YOLOv8n model is formed based on the improvement and serves as a foggy day pedestrian detection network model; according to the method, the detection accuracy and stability of the network in processing shielded and background complex images can be enhanced, and the boundary and detail features of a fuzzy target can be extracted more accurately in low-visibility environments such as foggy days and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection and relates to a method for detecting pedestrians in foggy weather, specifically a method for detecting pedestrians in low-visibility environments based on an improved YOLOv8n model. Background Art

[0002] In recent years, with the continuous acceleration of urbanization, road congestion has become increasingly prominent. Vehicle and pedestrian detection is widely used in intelligent transportation and autonomous driving. Haze, a common inclement weather condition, poses significant risks and challenges to vehicle operation. Visual sensors can capture a wide range of target information, but are susceptible to interference from factors such as haze, rain, snow, and dust. These objective factors can severely impact target detection algorithms. Therefore, improving the accuracy of vehicle and pedestrian detection in low-visibility environments has become a hot research topic in the field of target detection.

[0003] The single-stage detection algorithm directly classifies and regresses the input image without generating candidate boxes. Compared with the two-stage detection algorithm, it has significant advantages in speed and is particularly suitable for real-time applications. Representative models in the single-stage detection algorithm are the YOLO series and SSD. Ma et al. combined DehazeNet with the SSD target detection algorithm, and performed target detection after dehazing using the DehazeNet algorithm, thereby improving detection accuracy. Huang et al. proposed a target detection algorithm DSNet for severe weather conditions, which first enhances the image and then performs target detection through the detection subnetwork of RetianNet. Qin et al. proposed a detection-driven enhancement network (DENet) consisting of three key modules to enhance the features of the input image. However, none of these methods take into account the problem that target features in low-visibility environments are affected by fog noise, resulting in blurred images and difficulty for the network to extract effective features, which has a significant impact on the overall detection performance. Summary of the Invention

[0004] To address the problem that existing methods have unclear features in image model extraction in low-visibility environments, the present invention provides a pedestrian detection method in low-visibility environments based on an improved YOLOv8n model. A dual-branch fusion attention network, a feature aggregation and calibration pyramid network, and an adaptive task alignment detection head are proposed in the original YOLOv8n model. This method can enhance the network's detection accuracy and stability when processing occluded and complex background images, and more accurately extract the boundaries and detail features of blurred targets in low-visibility environments such as foggy days.

[0005] The technical solution adopted by the present invention to solve its technical problems is:

[0006] A pedestrian detection method in low-visibility environments based on an improved YOLOv8n model, comprising the following steps:

[0007] YOLOv8n is used as the original model. A dual-branch fusion attention network is adopted in the backbone network of the original YOLOv8n model. The feature structure of the original YOLOv8n model is retained in the dual-branch fusion attention network, and the CBFuse module and CBLinear module are introduced for feature fusion between different branches. A feature aggregation and calibration pyramid network containing a feature aggregation and calibration module is introduced into the original YOLOv8n model. The feature aggregation and calibration pyramid network performs multi-scale feature fusion through upsampling, downsampling and feature aggregation and calibration modules. The feature aggregation and calibration module calibrates and enhances features through local attention mechanism, global attention mechanism and pixel attention mechanism. An adaptive task alignment detection head is introduced into the original YOLOv8n model. The adaptive task alignment detection head implements dynamic convolution mechanism, task decomposition mechanism and dynamic feature alignment mechanism. Based on the above improvements, an improved YOLOv8n model is constructed as a network model for pedestrian detection in foggy weather.

[0008] Preprocessing of images from real foggy datasets;

[0009] The preprocessed foggy dataset is input into the improved YOLOv8n model for model training;

[0010] The trained improved YOLOv8n model is used to detect actual foggy pedestrian images.

[0011] Furthermore, in the dual-branch fusion attention network, the features fused by the CBFuse module and the CBLinear module generate a feature pyramid through multiple shared convolution kernels and convolutions with different dilation coefficients, and the attention mechanism is applied at the output to further enhance the features. Finally, the obtained feature map is input into the multi-expansion hierarchical attention module, which replaces the SPPF module of the original YOLOv8n model. It uses dilated convolutions with three different dilation factors to capture features under different receptive fields and generate multi-scale feature maps.

[0012] Furthermore, the feature aggregation and calibration module includes:

[0013] ADown downsampling module, which reduces the spatial dimension of feature maps by adjusting the stride of convolutional layers;

[0014] DySample dynamic upsampling mechanism, which adaptively processes complex input features and preserves multi-scale feature details through dynamic sampling, offset adjustment, and group convolution;

[0015] The feature diffusion module captures multi-scale features through deep convolutional layers with different kernel sizes and fuses information from different channels through point-by-point convolution.

[0016] The feature calibration module integrates local attention, global attention, and pixel attention mechanisms to calibrate features through weighted sums and residual connections. The local attention mechanism strengthens features in specific spatial regions, the global attention mechanism captures long-range dependencies, and the pixel attention mechanism refines pixel-by-pixel information.

[0017] The output features of the feature diffusion module and the output features of the feature calibration module are fused through deep convolution to obtain the final output features of the feature aggregation and calibration module.

[0018] Furthermore, the feature diffusion module performs the following calculation process:

[0019] First, multi-scale features are captured through deep convolutional layers with different kernel sizes. Each convolutional layer performs group convolution, and the calculation formula is as follows:

[0020] feature i =Conv i (x),i∈{5,7,9,11}

[0021] Among them, Conv i Represents the depth convolution operation with different convolution kernel sizes, x is the input feature;

[0022] Then the feature features processed by the deep convolution operation with different convolution kernel sizes are i Stack and fuse the information of different channels through point-by-point convolution. The calculation formula is as follows:

[0023]

[0024] Where feature1 represents the feature after point-by-point convolution fusion; Conv represents the point-by-point convolution operation; and the value range of n is {1, 2, 3, 4}.

[0025] Furthermore, the feature calibration module performs the following calculation process:

[0026] The local attention mechanism is implemented by the SpatialAttention module, which captures the local spatial information of the feature map; the global attention mechanism is implemented by the ChannelAttention module, which captures the global channel information of the feature map;

[0027] The pixel attention mechanism is implemented by the PixelAttention module. The pixel attention mechanism refines the information of each pixel in the feature map by combining local spatial information with global channel information. The pixel attention weight W pixelThe calculation is as follows:

[0028] W pixel =σ(pa(x,W local +W global ))

[0029] Where σ represents the sigmoid activation function; pa represents the pixel attention mechanism; W local represents the local attention weight; W global represents the global attention weight;

[0030] Through the above three attention mechanisms, feature calibration is completed. The calibrated feature is the weighted sum of the input feature x and the pixel attention weight, expressed as:

[0031] feature2=W pixel x+x

[0032] Finally, through deep convolution and fusion, the feature 1 after point-by-point convolution fusion and the calibrated feature 2 are added to obtain the final output feature, which is expressed as:

[0033] Out=feature1+feature2.

[0034] Furthermore, the adaptive task alignment detection head consists of five parts: a shared convolutional layer, a task decomposition module, a dynamic convolution offset and mask branch, a classification alignment branch, and an output prediction module. The shared convolutional layer consists of two consecutive depth-wise separable convolutions. The adaptive task alignment detection head performs the following steps:

[0035] Assume that the input feature maps P3, P4 and P5 have a size of H×W×C, where H, W and C represent the height, width and number of channels respectively. These three input feature maps are processed by the shared convolution layer and then concatenated to obtain the fused feature maps.

[0036]

[0037] Among them, Cat represents the feature splicing operation; DWConv represents the shared convolution layer processing operation;

[0038] The fused features Decomposed by the task decomposition module and input into the dynamic convolution offset and mask branch, and the classification alignment branch respectively:

[0039]

[0040] in represents the regression feature; Represents classification features; TaskDecomposition represents task decomposition module processing;

[0041] In the dynamic convolution offset and mask branch, Perform convolution to generate offset and mask:

[0042]

[0043] The generated offset and mask are the same as Input into dynamic convolution together and output the final regression features

[0044]

[0045] In the classification alignment branch, after the convolution layer, ReLU and Sigmoid activation function, the weight cls_weight is generated and then compared with Perform element-by-element multiplication to obtain the final classification features

[0046]

[0047] Where σ represents the sigmoid activation function;

[0048] Finally, the regression features and classification features are processed by convolution respectively through the output prediction module to generate bounding box predictions and category predictions, and adjusted according to the predetermined scale. The regression features and classification features are then spliced ​​together as the final detection output.

[0049] Furthermore, the specific steps of model training include: automatically generating a priori boxes using the K-mean clustering method, obtaining the bounding box size through bounding box regression prediction, classifying the bounding boxes using a classifier, obtaining the probability of defect types corresponding to each bounding box, and then sorting the classification probability of each bounding box through the non-maximum suppression method to obtain the bounding box prediction value with the highest confidence, and then calculating the loss value between the predicted value and the true value through the loss function, and performing backpropagation according to the loss value until the preset number of iterations is reached, and the network model training is completed.

[0050] Furthermore, the image data is preprocessed, specifically: when the width or height is proportionally scaled to 640, the remaining part is filled with background grayscale.

[0051] The beneficial effects of the present invention include:

[0052] The YOLOv8n model was improved for pedestrian detection in foggy weather. A dual-branch fusion attention network was adopted, and a multi-expanded hierarchical attention module was designed to replace the spatial pyramid pooling fast structure SPPF. This allows for more effective extraction and fusion of multi-scale features, enhancing the network's detection accuracy and stability when dealing with occlusion and complex background images. A feature aggregation and calibration module was proposed, and a new feature aggregation and calibration pyramid network was designed to further improve target detection accuracy in complex weather conditions. An adaptive task alignment detection head was designed. Through dynamic convolutional alignment and task separation processing, it more accurately extracts the boundaries and detailed features of blurred targets in low-visibility environments such as foggy weather, thereby improving classification and regression performance. The improved model improves the accuracy of pedestrian detection in foggy weather, meets the requirements for pedestrian detection in foggy weather, and lays the technical foundation for the ultimate establishment of a pedestrian detection system in foggy weather. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the pedestrian detection method in low-visibility environments based on the improved YOLOv8n model of the present invention;

[0054] Figure 2 This is a schematic diagram of the dual-branch fusion attention network structure proposed in this invention;

[0055] Figure 3 Schematic diagram of the multi-expansion hierarchical attention module proposed in the present invention;

[0056] Figure 4 It is a structural diagram of the feature aggregation and calibration module proposed in the present invention;

[0057] Figure 5 Schematic diagram of the feature aggregation and calibration pyramid network structure proposed in this invention;

[0058] Figure 6 Schematic diagram of the structure of the adaptive task alignment detection head proposed in the present invention;

[0059] Figure 7 1 is a schematic diagram of the improved algorithm structure of the present invention;

[0060] Figure 8 It is a comparison chart of the experimental results of the original model and the improved model of the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0062] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0063] As a single-stage detector, YOLO has faster detection speeds and is more suitable for real-time detection tasks such as autonomous driving. This application uses YOLOv8n as the detection network for pedestrians and vehicles in foggy conditions. YOLOv8n has high detection accuracy, fewer parameters, and lower computational overhead compared to other versions. By improving YOLOv8n, a more robust detection network is proposed, addressing the problems of low network accuracy and poor robustness.

[0064] Example 1

[0065] This embodiment proposes a pedestrian detection method for low-visibility environments based on an improved YOLOv8n model. It integrates artificial intelligence technology with foggy pedestrian detection technology. It aims to strengthen the adaptive feature fusion of the neural network during feature extraction and improve the detection effect. Without increasing the number of parameters and FLOPs, the model's detection accuracy for foggy pedestrians is improved, meeting the needs of foggy pedestrian detection.

[0066] The specific operations are as follows:

[0067] 1. Build a foggy pedestrian detection network model based on the improved YOLOv8n model:

[0068] (1) Environment configuration: Download the official source code of the YOLOv8n model from GitHub, then use Anaconda to build a virtual YOLOv8n environment. According to the requirements file of the YOLOv8n source code, install matplotlib>=3.2.2, opencv-python>=4.6.0, Pillow>=7.1.2, PyYAML>=5.3.1, requests>=2.23.0, scipy>=1.4.1, torc h>=1.7.0, torchvision>=0.8.1, tqdm>=4.64.0, tensorboard>=2.13.0, dvclive>=2.11.0, clearml, comet , pandas>=1.1.4, seaborn>=0.11.0, coremltools>=6.0, onnx>=1.12.0, onnxsim>=0.4.1, nvidia-pyindex nvidia-tensorrt, scikit-learn==0.19.2, tensorflow>=2.4.1, tflite-support, tensorflowjs>=3.9.0openvino-dev>=2022.3, psutil, thop>=0.1.1, ipython, algorithms>=1.0.3, pycocotools>=2.0.6, roboflow;

[0069] (2) Data input: Input the real foggy data set into the YOLOv8n network structure model, including obtaining foggy pedestrian images, foggy pedestrian image labels, YOLOv8n network structure model pre-training weights, and YOLOv8n network structure model configuration file;

[0070] (3) Configure hyperparameters: Set the optimal hyperparameter combination of learning rate, batch size, iteration number, image channels, image crop size, and learning rate momentum, specifically: learning rate lr = 0.01, batch size Batchsize = 16, iteration number Epoch = 150, image channels Channels = 3, image crop size Cropsize = 640 × 640, learning rate momentum = 0.937;

[0071] (4) Model improvement: The original model used in this invention is YOLOv8n. Specifically, the YOLOv8n network structure model is improved in the following three aspects:

[0072] 1) A dual-branch fusion attention network is designed in the backbone network to enhance the feature extraction capability and multi-scale information fusion. The CBLinual module and CBFuse module, dual-branch structure and CBLinear module are used in the backbone network. The network extracts features at different scales to ensure that both large structures and small details can be captured. The CBFuse module effectively combines the features of different branches, integrates features at different levels, and bridges the gap between semantic information and low-level details. Finally, a multi-expanded hierarchical attention module is designed to replace the SPPF module of the original YOLOv8n model to improve the network's multi-scale feature extraction capability. This fusion mechanism can enable the network to maintain information consistency in the presence of blur and distortion, adapt to more diverse weather environments, and improve detection accuracy in complex scenes. The structure of the proposed dual-branch fusion attention network is shown in the figure below. Figure 2 shown.

[0073] In the dual-branch fusion attention network, the feature structure in the original YOLOv8n network is retained, and the CBFuse module and CBLinear module are introduced to realize feature fusion between different branches, ensuring that the feature extraction in the two trunks can be effectively combined. The final feature fusion integrates features from different scales through a shared convolution layer to construct a multi-scale feature pyramid. This application uses shared convolution kernels and convolutions with different expansion coefficients to generate a feature pyramid, and applies an attention mechanism at the output to further enhance the features. Finally, the obtained feature map is input into the multi-expansion layered attention module, which uses three different expansion factors (1, 3, 5) of expansion convolution to capture features under different receptive fields. This allows the model to obtain a wider range of contextual information without losing spatial resolution, thereby effectively processing targets in complex scenes; at the same time, the use of a shared convolution strategy reduces the number of model parameters, and multiple expansion convolutions share the same convolution kernel weights, reducing storage and computational costs; splicing the features extracted by different expansion convolutions can effectively combine multiple feature information; in this way, the model can consider features at different scales, thereby enhancing the ability to detect targets. The structure of the multi-expansion layered attention module is as follows Figure 3 shown.

[0074] 2) A feature aggregation and calibration module is designed, and based on this, a feature aggregation and calibration pyramid network is designed. The proposed feature aggregation and calibration module structure is as follows: Figure 4 As shown, the feature aggregation and calibration pyramid network structure designed based on this module is as follows Figure 5 shown.

[0075] In complex weather conditions, the traditional PAN architecture is only suitable for feature fusion under general conditions. In severe weather scenarios, the PAN architecture lacks sufficient mechanisms to suppress irrelevant background noise, such as raindrops and fog, resulting in poor robustness in complex environments. While the top-down sampling mechanism in the PAN architecture enables feature fusion, the feature interaction path is relatively simple. This results in a lack of more refined feature interaction methods when processing targets of varying scales in complex weather scenarios, potentially losing key information about small or blurred targets.

[0076] To overcome these issues, this application proposes a Feature Focus and Calibration Module (FFAC), and based on this, designs a Feature Focus and Calibration Pyramid Network (FFCPN). The FFAC module not only supports the capture of high-level semantic features, but also maintains the integrity of low-level semantic spatial information, enhancing the effect of multi-scale feature fusion. The use of local and global attention mechanisms in the FFAC module can more effectively extract effective features in complex scenarios and suppress irrelevant background noise.

[0077] The ADown downsampling module is introduced in the FFAC module. By adjusting the stride of the convolutional layer, it reduces the spatial dimension of the feature map, reduces model complexity, and improves operational efficiency. Traditional upsampling is interpolated based on spatial position and cannot utilize the content of the input image or feature map. It is prone to feature loss under complex conditions. The DySample dynamic upsampling mechanism is introduced in the FFAC module. Through dynamic sampling, offset adjustment, group convolution and other mechanisms, it provides more powerful upsampling capabilities than traditional upsampling. It can maintain multi-scale features while adaptively processing complex input features, especially in preserving and enhancing details in visual tasks.

[0078] The Feature Diffusion Module is designed to perform feature fusion through deep convolutional layers with different kernel sizes, and finally through point-by-point convolution. This module uses convolution kernels of different sizes (5x5, 7x7, 9x9, 11x11) to capture feature information under different receptive fields. This design allows the model to extract features at multiple scales: small convolution kernels capture local information, and large convolution kernels capture larger global structural information. Each convolution layer performs group convolution to ensure channel-independent calculations, reduce the number of parameters, and improve computational efficiency. The corresponding formula is:

[0079] feature i =Conv i(x),i∈{5,7,9,11}

[0080] Among them, Conv i represents the depthwise convolution operation with different kernel sizes, and x is the input feature.

[0081] The feature features processed by the depth convolution operation with different convolution kernel sizes i The features are stacked and the information of different channels is fused through point-by-point convolution. This method uniformly processes the features extracted by different receptive fields, making the final output features more expressive. The formula for feature fusion is:

[0082]

[0083] Where feature1 represents the feature after point-by-point convolution fusion; Conv represents the point-by-point convolution operation; and the value range of n is {1, 2, 3, 4}.

[0084] The Feature Calibration Module primarily calibrates and enhances features through three different attention mechanisms: local attention, global attention, and pixel attention. This design aims to fuse and calibrate features at different scales and ranges, thereby improving the overall performance of the model, especially in complex scenarios.

[0085] The local attention is implemented by the SpatialAttention module, which is responsible for capturing the local spatial information of the feature map. It can enhance the features of specific spatial regions and reduce the interference of irrelevant redundant information. Define the local weight:

[0086] W local =span(x)

[0087] Global attention is implemented through ChannelAttention, which focuses on the global channel information of the entire feature map and can better capture long-range dependencies and global context information. Define global weights:

[0088] W global =chan(x)

[0089] Pixel attention is implemented by PixelAttention, which further refines the information of each pixel in the feature map. By combining local and global attention information, pixel attention can calibrate features more carefully. The pixel-level weight is calculated as follows:

[0090] W pixel =σ(pa(x,W local +W global ))

[0091] Where σ represents the sigmoid activation function; pa represents the pixel attention mechanism;

[0092] Through the above three attention mechanisms, the feature calibration is completed. The calibrated feature is the weighted sum of the input feature map x and the pixel attention weight:

[0093] feature2=W pixel x+x

[0094] The features corrected by pixel attention are added to the original input x to realize residual connection, thereby maintaining the original features while enhancing the attention effect.

[0095] Finally, through deep convolution and fusion, the feature 1 after point convolution fusion and the feature 2 after calibration are added to obtain the final output feature:

[0096] Out=feature1+feature2

[0097] This overall design, through the combination of efficient convolution, multi-scale processing and attention mechanism, can significantly improve the performance of the model in complex scenarios, especially in multi-target detection and multi-scale object detection tasks.

[0098] 3) Designed an adaptive task alignment detection head (ATADH) such as Figure 6 As shown, the traditional detection head processes classification and bounding box regression tasks at the same time. Since these tasks share features, the classification task and the positioning task affect each other in complex weather conditions. The adaptive task alignment detection head of this application has the following main advantages: 1) Adaptive feature extraction: In order to solve the problem that traditional convolution is not flexible enough for feature extraction in complex weather conditions, ATADH introduces a dynamic convolution mechanism, which enables the convolution kernel to adaptively adjust the receptive field and convolution parameters according to the input image, so as to better deal with blurred and low-contrast targets and improve the accuracy of target feature extraction; 2) Task decomposition and feature alignment: In response to the mutual interference problem caused by the coupling of classification tasks and positioning tasks in YOLOv8n, ATADH separates classification and bounding box regression through task decomposition, and ensures that each task can obtain more accurate feature support through a dynamic feature alignment mechanism;

[0099] Specifically, the adaptive task alignment detection head (ATTDH) consists of five parts: a shared convolutional layer, a task decomposition module, a dynamic convolution offset and mask branch, a classification alignment branch, and an output prediction module. First, in the feature extraction stage, a shared convolutional layer (SharedLayer) is applied. This layer consists of two consecutive depthwise separable convolutions (Depthwise SeparableConvolution). Through convolution and normalization operations, the input feature map is transformed into implicit features, and the number of feature channels is reduced to reduce the amount of calculation. The entire adaptive task alignment detection head process is as follows: Assuming that the input feature maps P3, P4, and P5 are each of size H×W×C, H, W, and C represent height, width, and number of channels, respectively; these three input feature maps are subjected to separable convolution processing through a shared convolutional layer, and then spliced ​​to obtain the fused features.

[0100]

[0101] Among them, Cat represents the feature splicing operation; DWConv represents the shared convolution layer processing operation;

[0102] The obtained fused features It is decomposed through the task decomposition module and input into two branches respectively: dynamic convolution offset and mask branch RegAlignment and classification alignment branch ClsAlignment:

[0103]

[0104] in represents the regression feature; Represents classification features; TaskDecomposition represents task decomposition module processing;

[0105] In the RegAlignment branch, Perform convolution to generate offset and mask:

[0106]

[0107] In the RegAlignment part, the generated offset and mask are the same as Input into dynamic convolution together and output the final regression features

[0108]

[0109] In the Cls Alignment branch, after the convolution layer, ReLU and Sigmoid activation function, the weight cls_weight is generated and then combined with Perform element-by-element multiplication to obtain the final classification features

[0110]

[0111] Where σ represents the sigmoid activation function;

[0112] Finally, the aligned regression features and classification features are processed by convolution respectively through the output prediction module to generate bounding box predictions and category predictions, and adjusted according to the predetermined scale. The regression features and classification features are then spliced ​​together as the final detection output.

[0113] This module uses the shared features obtained for classification and regression tasks, assigning independent feature processing paths to each task. It uses adaptive average pooling to obtain global features, and then processes the classification and regression features separately based on this, ensuring that the tasks do not interfere with each other.

[0114] Based on the above solution, YOLOv8n is used as the base model, a dual-branch fusion attention network is employed, and a multi-scale dilated self-attention network (MDSAN) is designed to replace the spatial pyramid pooling fast structure (SPPF). This allows for more efficient extraction and fusion of multi-scale features, enhancing the network's detection accuracy and stability when handling images with occlusion and complex backgrounds. In the neck feature fusion component, a feature aggregation and calibration module (FFAC module) is proposed, and a novel feature aggregation and calibration pyramid network (FFCPN) is designed to further improve object detection accuracy in complex weather conditions. Finally, an adaptive task alignment detection head is designed. Through dynamic convolutional alignment and task separation, it more accurately extracts the boundaries and detailed features of blurred objects in low-visibility environments such as fog, thereby improving classification and regression performance. Image data augmentation and preprocessing improve the model's robustness and generalization, enabling it to better handle foggy pedestrian images in various real-world scenarios.

[0115] The CBLinear and CBFuse modules are introduced to achieve feature fusion between different branches, ensuring that the feature extraction of the two main branches can be effectively combined. Furthermore, a multi-expansion hierarchical attention module is designed to replace the SPPF module in the original network structure. By sharing convolution kernels multiple times and combining different expansion rates, multi-scale feature maps are generated. The FFCPN network is redesigned to perform multi-scale feature fusion through upsampling, downsampling, and FFAC modules. The feature flow is not only bidirectional, but also includes a feature enhancement step, allowing for more effective fusion of features at different scales. A dynamically aligned adaptive task alignment detection head is proposed, which uses dynamic deformable convolution to improve localization accuracy. A task decomposition module is used to separate the classification and regression tasks, improving classification accuracy and bounding box localization precision, reducing interference between tasks, and showing significant advantages in complex weather scenarios where multiple interference factors are simultaneously processed. These improvements, by optimizing the model's feature extraction, enhancing texture perception, and strengthening the attention mechanism, collectively improve the performance of the modified YOLOv8n model in foggy pedestrian detection tasks, resulting in higher accuracy and better robustness.

[0116] 2. Image preprocessing:

[0117] The specific steps of preprocessing include:

[0118] When the width / height is scaled proportionally to 640, the remaining part is filled with background grayscale.

[0119] 3. Training using the improved YOLOv8n model

[0120] (1) The image data enhancement and pre-processed foggy pedestrian dataset are input into the improved YOLOv8n model with set hyperparameters for training. The configuration used in the present invention is as follows: CPU: Intel Core i9-11900K; CPU main frequency: 3.50GHz; memory: 32G; GPU: NVIDIA GeForce RTXA4000; video memory: 8G; deep learning framework is PyTorch, and the development environment is Pytoch 2.0.1, Python 3.11, and Cuda 11.8;

[0121] (2) Model training: The training set of the foggy pedestrian dataset is input into the improved YOLOv8n model for training. The K-mean clustering method is used to automatically generate a priori boxes. The bounding box size is predicted by bounding box regression. The bounding boxes are classified by the classifier to obtain the probability of defect types corresponding to each bounding box. The classification probability of each bounding box is then sorted by the non-maximum suppression (NMS) method to obtain the bounding box prediction value with the highest confidence. The confidence threshold is set to 0.25 and the IOU threshold is set to 0.7. The loss value between the predicted value and the true value is then calculated by the loss function. Backpropagation is performed according to the loss value until the preset number of iterations is reached and the network model training is completed.

[0122] Based on the above scheme, the use of K-means clustering method to automatically generate prior boxes helps the model better adapt to the sizes and shapes of different targets, which can improve the generalization ability of the model and enable it to detect a variety of targets; through bounding box regression, the model can more accurately predict the bounding box size of the target, which helps to improve the accuracy of detection, especially for targets of different sizes; the classifier is used to classify the bounding boxes, thereby determining the probability of the defect type corresponding to each bounding box, allowing the model to not only detect the existence of the target but also classify the type of the target; the non-maximum suppression method is used to sort the classification probability of each bounding box, remove highly overlapping bounding boxes, thereby reducing repeated detection and improving the quality of the detection results; setting the confidence threshold and IOU threshold helps to control the output results of the model, ensuring that only bounding boxes with sufficiently high confidence are retained, improving the robustness and accuracy of the model; the difference between the predicted value and the true value is calculated through the loss function, and the model can perform backpropagation to update the model parameters to reduce the loss;

[0123] 4. Test and evaluate the improved YOLOv8n model:

[0124] (1) Model testing: The performance changes caused by changes in network structure were verified through ablation experiments. Among them, we will add the Dual-Branch Fusion Attention Network module (Dual-Branch Fusion Attention Network) that replaces the original model as DBFAN, and the proposed Feature Focus and Calibration Pyramid Network (Feature Focus and Calibration Pyramid Network) as FFCPN. The proposed Adaptive Task-Aligned Detection Head (Adaptive Task-Aligned Detection Head) is called ATADH. A total of four experiments were trained: YOLOv8n, YOLOv8n-DBFAN, YOLOv8n-DBFAN-FFCPN, and YOLOv8n-DBFAN-FFCPN-ATADH (this invention). The experimental results are as follows: Figure 8 and as shown in Table 1.

[0125] Table 1 Ablation experiment evaluation table of the present invention

[0126]

[0127] As shown in Table 1, the DBFAN network can better fuse information from different scales and use dilated convolutions to expand the receptive field, thereby capturing rich contextual information. Furthermore, the designed FFAC module and the model's feature aggregation and calibration pyramid network effectively capture contextual information and perform feature fusion and correction. This enhances target feature extraction capabilities and improves accuracy to 74.2%. The designed ATADH detection head can adaptively adjust detection strategies based on different detection tasks, thereby improving model robustness. The overall model average accuracy increased from 71.2% to 76.2%. Figure 8 This figure shows a performance comparison between the proposed method and YOLOv8n in different scenarios. The image on the left shows the detection results of YOLOv8n, and the image on the right shows the detection results of the proposed method. In scenes with clear distinction between the background and pedestrians in fog, there is almost no difference between the proposed method and YOLOv8n, but the proposed method achieves higher detection accuracy. Under occlusion conditions, YOLOv8n may experience false detections or missed detections, while the proposed method does not experience these false detections or missed detections, and achieves higher detection accuracy.

[0128] 5. Use the improved YOLOv8n model to detect pedestrian images in foggy weather:

[0129] The pedestrian image in foggy weather is input into the improved YOLOv8n model to complete the detection of pedestrian images in foggy weather.

[0130] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A pedestrian detection method in low-visibility environment based on the improved YOLOv8n model, its characteristic steps include: YOLOv8n is used as the original model. A dual-branch fusion attention network is adopted in the backbone network of the original YOLOv8n model. The feature structure of the original YOLOv8n model is retained in the dual-branch fusion attention network, and the CBFuse module and CBLinear module are introduced for feature fusion between different branches. A feature aggregation and calibration pyramid network containing a feature aggregation and calibration module is introduced into the original YOLOv8n model. The feature aggregation and calibration pyramid network performs multi-scale feature fusion through upsampling, downsampling, and feature aggregation and calibration modules. The feature aggregation and calibration module calibrates and enhances features through local attention mechanism, global attention mechanism, and pixel attention mechanism. An adaptive task alignment detection head is introduced into the original YOLOv8n model. The adaptive task alignment detection head implements dynamic convolution mechanism, task decomposition mechanism, and dynamic feature alignment mechanism. Based on the above improvements, an improved YOLOv8n model is constructed as a network model for pedestrian detection in foggy weather. Preprocessing of images from real foggy datasets; The preprocessed foggy dataset is input into the improved YOLOv8n model for model training; The trained improved YOLOv8n model is used to detect actual foggy pedestrian images.

2. The pedestrian detection method in low-visibility environment based on the improved YOLOv8n model according to claim 1 is characterized in that: In the dual-branch fusion attention network, the features fused by the CBFuse module and the CBLinear module are used to generate a feature pyramid through multiple shared convolution kernels and convolutions with different dilation coefficients. The attention mechanism is applied at the output to further enhance the features. Finally, the obtained feature map is input into the multi-expansion hierarchical attention module, which replaces the SPPF module of the original YOLOv8n model. It uses dilated convolutions with three different dilation factors to capture features under different receptive fields and generate multi-scale feature maps.

3. The pedestrian detection method in low-visibility environment based on the improved YOLOv8n model according to claim 1 is characterized in that: The feature aggregation and calibration module includes: ADown downsampling module, which reduces the spatial dimension of feature maps by adjusting the stride of convolutional layers; DySample dynamic upsampling mechanism, which adaptively processes complex input features and preserves multi-scale feature details through dynamic sampling, offset adjustment, and group convolution; The feature diffusion module captures multi-scale features through deep convolutional layers with different kernel sizes and fuses information from different channels through point-by-point convolution. The feature calibration module integrates local attention, global attention, and pixel attention mechanisms to calibrate features through weighted sums and residual connections. The local attention mechanism strengthens features in specific spatial regions, the global attention mechanism captures long-range dependencies, and the pixel attention mechanism refines pixel-by-pixel information. The output features of the feature diffusion module and the output features of the feature calibration module are fused through deep convolution to obtain the final output features of the feature aggregation and calibration module.

4. The pedestrian detection method in low-visibility environment based on the improved YOLOv8n model according to claim 3 is characterized in that: The feature diffusion module performs the following calculation process: First, multi-scale features are captured through deep convolutional layers with different kernel sizes. Each convolutional layer performs group convolution, and the calculation formula is as follows: feature i =Conv i (x),i∈{5,7,9,11} Among them, Conv i Represents the depth convolution operation with different convolution kernel sizes, x is the input feature; Then the feature features processed by the deep convolution operation with different convolution kernel sizes are i Stack and fuse the information of different channels through point-by-point convolution. The calculation formula is as follows: Where feature1 represents the feature after point-by-point convolution fusion; Conv represents the point-by-point convolution operation; and the value range of n is {1, 2, 3, 4}.

5. The pedestrian detection method in low-visibility environment based on the improved YOLOv8n model according to claim 4 is characterized in that: The feature calibration module performs the following calculation process: The local attention mechanism is implemented by the SpatialAttention module, which captures the local spatial information of the feature map; the global attention mechanism is implemented by the ChannelAttention module, which captures the global channel information of the feature map; The pixel attention mechanism is implemented by the PixelAttention module. The pixel attention mechanism refines the information of each pixel in the feature map by combining local spatial information with global channel information. The pixel attention weight W pixel The calculation is as follows: IN pixel =σ(pa(x,W local +W global )) Where σ represents the sigmoid activation function; pa represents the pixel attention mechanism; W local represents the local attention weight; W global represents the global attention weight; Through the above three attention mechanisms, feature calibration is completed. The calibrated feature is the weighted sum of the input feature x and the pixel attention weight, expressed as: feature2=W pixel ·x+x Finally, through deep convolution and fusion, the feature 1 after point-by-point convolution fusion and the calibrated feature 2 are added to obtain the final output feature, which is expressed as: Out=feature1+feature2.

6. The pedestrian detection method in low-visibility environment based on the improved YOLOv8n model according to claim 1 is characterized in that: The adaptive task alignment detection head consists of five parts: a shared convolutional layer, a task decomposition module, a dynamic convolution offset and mask branch, a classification alignment branch, and an output prediction module. The shared convolutional layer consists of two consecutive depthwise separable convolutions. The adaptive task alignment detection head performs the following steps: Assume that the input feature maps P3, P4 and P5 have a size of H×W×C, where H, W and C represent the height, width and number of channels respectively. These three input feature maps are processed by the shared convolution layer and then concatenated to obtain the fused feature maps. Among them, Cat represents the feature splicing operation; DWConv represents the shared convolution layer processing operation; The fused features Decomposed by the task decomposition module and input into the dynamic convolution offset and mask branch, and the classification alignment branch respectively: in represents the regression feature; Represents classification features; TaskDecomposition represents the task decomposition module processing; In the dynamic convolution offset and mask branch, Perform convolution to generate offset and mask: The generated offset and mask are the same as Input into dynamic convolution together and output the final regression features In the classification alignment branch, after the convolution layer, ReLU and Sigmoid activation function, the weight cls_weight is generated and then compared with Perform element-by-element multiplication to obtain the final classification features Where σ represents the sigmoid activation function; Finally, the regression features and classification features are processed by convolution respectively through the output prediction module to generate bounding box predictions and category predictions, and adjusted according to the predetermined scale. The regression features and classification features are then spliced ​​together as the final detection output.

7. The method for pedestrian detection in low-visibility environments based on the improved YOLOv8n model according to any one of claims 1 to 6, characterized in that: The specific steps of model training include: using the K-mean clustering method to automatically generate a priori boxes, obtaining the bounding box size through bounding box regression prediction, using a classifier to classify the bounding boxes, obtaining the probability of defect types corresponding to each bounding box, and then sorting the classification probability of each bounding box through the non-maximum suppression method to obtain the bounding box prediction value with the highest confidence. Then, the loss value between the predicted value and the true value is calculated through the loss function, and backpropagation is performed according to the loss value until the preset number of iterations is reached, and the network model training is completed.

8. The method for pedestrian detection in low-visibility environments based on the improved YOLOv8n model according to any one of claims 1 to 6, characterized in that: Image data preprocessing, specifically: when the width or height is proportionally scaled to 640, the remaining part is filled with background grayscale.

Images