Intelligent driving pedestrian detection method, system and device and vehicle

Abstract

The invention provides an intelligent driving pedestrian detection method, system and device and a vehicle, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining a road environment image, carrying out the size normalization preprocessing to generate a normalized road image, analyzing the images through an improved YOLOv8 model, and carrying out the recognition of the normalized road image, the model comprises a backbone network module, a feature pyramid fusion module and a detection head module, the detection head module particularly integrates a C2f residual unit, a mixed attention mechanism unit and a decoupling detection unit, and the mixed attention mechanism unit is formed by connecting a channel attention subunit and a space attention subunit in series. Therefore, the pedestrian detection precision and sensitivity are effectively improved. And on the basis of the obtained pedestrian detection result, an obstacle avoidance instruction is generated to control steering or braking of the vehicle, so that the safety and reliability of intelligent driving in a complex traffic environment are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to an intelligent driving pedestrian detection method, system and device, and vehicle. BACKGROUND

[0002] With the rapid development of intelligent driving technology, pedestrian detection as a core part of environmental perception is directly related to the safety decision of the vehicle, such as emergency braking and path planning. However, small target detection, occlusion interference and light mutation in complex road environment pose severe challenges to the robustness of detection algorithms. In the prior art, although the two-stage detection model such as the R-CNN series performs well in accuracy, it has high computational complexity and slow processing speed, which is difficult to meet the real-time requirement; the single-stage detection model, especially the YOLO series, is fast and efficient, but has a high miss rate for small target objects. In order to pursue speed, the accuracy is sacrificed to a certain extent. Therefore, these technologies have obvious deficiencies in pedestrian detection, especially in the recognition of small target pedestrians in complex scenes. SUMMARY

[0003] The purpose of the present application is to provide an intelligent driving pedestrian detection method, system, device and vehicle to solve one or more technical problems in the prior art and at least provide a beneficial choice or create conditions.

[0004] The solution to the technical problem of the present application is: on the one hand, the present application provides an intelligent driving pedestrian detection method, comprising the following steps: obtaining a road environment image in the intelligent driving process; performing size normalization preprocessing on the road environment image to obtain a normalized road image; inputting the normalized road image into an improved YOLOv8 model to obtain a pedestrian detection result; the pedestrian detection result includes a pedestrian position bounding box and a confidence; generating an obstacle avoidance instruction according to the pedestrian detection result to control the vehicle to turn or brake; the improved YOLOv8 model includes a backbone network module, a feature pyramid fusion module and a detection head module; the detection head module includes a C2f residual unit, a mixed attention mechanism unit and a decoupling detection unit; the mixed attention mechanism unit is connected after the C2f residual unit; the mixed attention mechanism unit is composed of a channel attention subunit and a spatial attention subunit in series; the improved YOLOv8 model is optimized by a distance attention Wiou loss function.

[0005] Further, the normalized road image is input into an improved YOLOv8 model to obtain a pedestrian detection result, including: The normalized road image is input into the backbone network module, and multi-scale feature maps are extracted through the backbone network module; The multi-scale feature maps are input into the feature pyramid fusion module, and high-resolution fusion feature maps are generated through upsampling and cross-level connection; The fusion feature maps are input into the detection head module, and are sequentially processed through a C2f residual unit, a mixed attention mechanism unit and a decoupling detection unit to output a pedestrian detection result.

[0006] Further, the fusion feature maps are input into the detection head module, and are sequentially processed through a C2f residual unit, a mixed attention mechanism unit and a decoupling detection unit to output a pedestrian detection result, including: The fusion feature maps are input into the C2f residual unit, and shallow detail features and deep semantic features are fused through cross-stage partial connection to generate multi-scale optimized feature maps; The multi-scale optimized feature maps are input into the channel attention subunit to generate a channel weight matrix, which is multiplied element by element with the multi-scale optimized feature maps to output a channel weighted feature map; The multi-scale optimized feature maps are input into the spatial attention subunit to generate a spatial weight matrix, which is multiplied element by element with the multi-scale optimized feature maps to output a spatial weighted feature map; The spatial weighted feature map and the channel weighted feature map are adaptively fused to obtain a spatial channel feature map, which is input into the decoupling detection unit and processed through a separate classification branch and a regression branch to output a pedestrian detection result.

[0007] Further, the road environment image is subjected to size normalization preprocessing to obtain a normalized road image, including The road environment image is scaled to a fixed resolution according to the original aspect ratio to obtain a scaled adaptive image; The blank area in the scaled adaptive image due to the difference in aspect ratio is filled using a bilinear interpolation algorithm, and the pixel value is the average of the edge pixels of the scaled adaptive image to generate an edge average filled image; The edge average filled image is subjected to pixel value normalization processing, and the pixel value is linearly mapped to the [0, 1] interval to generate the normalized road image.

[0008] Further, the distance attention Wiou loss function adjusts the gradient of the IoU loss function by introducing a center distance dynamic weight coefficient; The distance attention Wiou loss function satisfies the following formula: ; wherein, denotes the distance attention Wiou loss function, denotes the center distance dynamic weight coefficient, denotes the IoU loss function; The center distance dynamic weight coefficient satisfies the following formula: ; wherein, is the center point coordinate of the predicted pedestrian position bounding box; is the center point coordinate of the real pedestrian position bounding box; is the normalized width of the real pedestrian position bounding box; is the normalized height of the real pedestrian position bounding box.

[0009] Further, the generation of the obstacle avoidance instruction according to the pedestrian detection result, the control of the vehicle steering or braking, comprises: When the confidence is greater than a preset steering confidence threshold, and the overlap area proportion of the pedestrian position bounding box and the vehicle driving track is greater than a preset steering overlap area threshold, a steering instruction is generated to control the vehicle steering; When the confidence is greater than a preset braking confidence threshold, and the overlap area proportion of the pedestrian position bounding box and the vehicle driving track is greater than a preset braking overlap area threshold, a braking instruction is generated to control the vehicle braking.

[0010] In another aspect, the present application provides an intelligent driving pedestrian detection system for executing the aforementioned intelligent driving pedestrian detection method.

[0011] In another aspect, the present application provides an intelligent driving pedestrian detection device, comprising a processor, a memory, a vehicle-mounted camera and a vehicle control module; the vehicle-mounted camera is used to acquire road environment images in the intelligent driving process; the memory is used to store programs; when the programs are executed by the processor, the aforementioned intelligent driving pedestrian detection method is realized, the pedestrian detection result is obtained, and the obstacle avoidance instruction is generated according to the pedestrian detection result; The vehicle control module is used to control the vehicle steering or braking according to the obstacle avoidance instruction.

[0012] Further, the vehicle-mounted camera is a multispectral camera supporting RGB and infrared image fusion input.

[0013] In another aspect, the present application provides a vehicle integrated with the aforementioned intelligent driving pedestrian detection device.

[0014] The beneficial effects of the present invention are as follows: the present invention proposes a method for intelligent driving pedestrian detection, which obtains road environment images and performs size normalization preprocessing to generate normalized road images, and then uses an improved YOLOv8 model to analyze these images. The model includes a backbone network module, a feature pyramid fusion module and a detection head module, wherein the detection head module particularly integrates a C2f residual unit, a hybrid attention mechanism unit and a decoupling detection unit, wherein the hybrid attention mechanism unit is composed of a channel attention subunit and a spatial attention subunit in series, thereby effectively improving the detection accuracy and sensitivity of pedestrians. In addition, the model further enhances the positioning accuracy of the pedestrian position bounding box through the optimization of the distance attention WIoU loss function; based on the obtained pedestrian detection results, obstacle avoidance instructions are generated to control the vehicle steering or braking, thereby effectively improving the safety and reliability of intelligent driving in complex traffic environments. The present application also provides corresponding devices, systems and vehicles. The beneficial effects of the devices, systems and vehicles are similar to those of the method and will not be repeated here.

[0015] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation to the technical solution of the present invention.

[0017] Figure 1 This is a flow chart of the intelligent driving pedestrian detection method provided by this application; Figure 2 This is a structural diagram of the improved YOLOv8 model provided by this application; Figure 3 This is a structural diagram of the intelligent driving pedestrian detection system provided by this application; Figure 4 This is a structural diagram of the intelligent driving pedestrian detection device provided in this application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0019] The application will be further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be considered as limiting the application, and all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the application.

[0020] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the application and are not intended to limit the application.

[0022] With the rapid development of intelligent driving technology, pedestrian detection as the core of environmental perception is directly related to the safety decision of the vehicle, such as emergency braking and path planning. However, in practical applications, complex road environment brings many challenges to pedestrian detection algorithm, especially for small target detection (such as distant pedestrians), occlusion interference (such as partially visible pedestrians between vehicles) and light mutation scene (such as tunnel entrance and exit). These problems pose a severe test to the robustness of the detection algorithm.

[0023] In the prior art, two-stage detection models such as R-CNN series perform well in accuracy, but their high computational complexity and slow processing speed make it difficult to meet the real-time requirements. Single-stage detection models such as YOLO series are known for their speed and efficiency, but they have a high miss rate in detecting small target objects (such as pedestrians), which is mainly because the accuracy is sacrificed to some extent in pursuit of speed. In addition, although there have been attempts to apply attention mechanisms to target detection to improve performance, most work has focused on a single type of attention mechanism, which cannot fully capture the key features of input data, especially in pedestrian detection tasks that require attention to different scales and location information at the same time, limiting the final performance of the model.

[0024] These technologies are not up to the task of pedestrian detection in complex scenarios, especially in the case of small target pedestrian detection, occlusion interference and light mutation, the robustness and positioning accuracy of existing models are obviously insufficient. These problems lead to high miss rate and inaccurate positioning, and there is an urgent need to optimize the algorithm to improve the reliability and accuracy of detection. Therefore, how to improve the detection accuracy of small target pedestrians while ensuring processing speed and enhance the adaptability of the model in complex environments has become a problem to be solved in the field of intelligent driving pedestrian detection.

[0025] In response to the challenges of pedestrian detection in complex road environments, such as small target detection, occlusion interference, and sudden changes in illumination, the embodiments of the present application propose an intelligent driving pedestrian detection method and its system, device, and vehicle. The method first obtains images of the road environment during intelligent driving and performs size normalization preprocessing on them to generate normalized road images. Subsequently, these images are input into an improved YOLOv8 model for analysis. The model is specially designed with a structure including a backbone network module, a feature pyramid fusion module, and a detection head module to improve detection accuracy. Among them, the detection head module further enhances the sensitivity to pedestrians and the detection accuracy by introducing a C2f residual unit, a hybrid attention mechanism unit (composed of a channel attention subunit and a spatial attention subunit in series), and a decoupled detection unit.

[0026] In addition, this application also uses the distance attention WIoU loss function to optimize the model, and adjusts the gradient of the IoU loss function by introducing a dynamic weight coefficient of the center distance, thereby improving the accuracy of the bounding box positioning. Based on the obtained pedestrian detection results, that is, the pedestrian position bounding box and confidence, the system can generate corresponding obstacle avoidance instructions to control the vehicle steering or braking. In order to implement the above method, this application also designs an intelligent driving pedestrian detection system, device and a vehicle integrated with the device, wherein the device includes a processor, a memory, a multispectral camera that supports RGB and infrared image fusion input, and a vehicle control module to ensure that pedestrians can be effectively identified and timely safety decisions can be made in various complex environments. This series of innovations not only improves the accuracy and robustness of pedestrian detection, but also provides strong support for the development of intelligent driving technology.

[0027] First, the intelligent driving pedestrian detection method provided by the embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0028] Reference Figure 1 The implementation process of the intelligent driving pedestrian detection method provided in the embodiment of the present application includes but is not limited to the following steps.

[0029] Step S100: Acquire a road environment image during the intelligent driving process.

[0030] In step S100, the vehicle's onboard camera or other image acquisition device acquires real-time image information of the road environment while the vehicle is in motion. This provides raw visual data for subsequent pedestrian recognition and analysis, forming the foundation for environmental perception. High-quality road environment images effectively ensure accurate and timely pedestrian detection, a prerequisite for the intelligent driving system to make safe decisions.

[0031] Step S200 , performing size normalization preprocessing on the road environment image to obtain a normalized road image.

[0032] In step S200, the original images are uniformly resized and normalized, so that images of different resolutions or sources have a uniform format and scale before being input into the model. This step is meaningful in improving the generalization ability of the model, reducing detection errors caused by inconsistent image sizes, and speeding up model inference and improving detection efficiency, providing standardized input for subsequent deep learning model processing.

[0033] In step S300, the normalized road images are input into the improved YOLOv8 model to obtain pedestrian detection results.

[0034] The pedestrian detection results include pedestrian position bounding boxes and confidence.

[0035] In step S300, the improved YOLOv8 model is used for efficient and accurate pedestrian detection on the preprocessed images. Compared with traditional target detection models, the improved model introduces C2f residual units, hybrid attention mechanism units (including channel attention and spatial attention), and decoupled detection structures, and uses distance attention Wiou loss function to optimize the training process, thereby effectively improving the detection accuracy and robustness of small target pedestrians, occluded pedestrians, and complex lighting conditions. The output pedestrian position bounding boxes are used to locate the position of the pedestrians, and the confidence is used to evaluate the reliability of the detection results.

[0036] In step S400, according to the pedestrian detection results, obstacle avoidance instructions are generated to control the vehicle to turn or brake.

[0037] In step S400, the pedestrian detection results are used to realize active safety control of the intelligent driving system. When pedestrians are detected in front and it is judged that there is a risk of collision, the system will generate corresponding obstacle avoidance instructions, such as emergency braking, deceleration, or turning to bypass, based on the pedestrian position bounding boxes and confidence. The significance of this step is to apply the results of the detection algorithm directly to the actual control of the vehicle, improving the safety and intelligence level of autonomous driving, thereby effectively preventing traffic accidents and protecting the lives and safety of pedestrians and passengers.

[0038] In some embodiments of the present application, with reference to Figure 2 The improved YOLOv8 model includes a backbone network module 101, a feature pyramid fusion module 102, and a detection head module 103. The detection head module 103 includes C2f residual units, hybrid attention mechanism units, and decoupled detection units. The hybrid attention mechanism unit is connected after the C2f residual unit. The hybrid attention mechanism unit is composed of a channel attention sub-unit and a spatial attention sub-unit in series. The improved YOLOv8 model is optimized by a distance attention Wiou loss function.

[0039] Backbone network module 101, the first component of the improved YOLOv8 model, is responsible for extracting multi-layer semantic features from the input image. Through a series of convolution operations and sampling, it transforms raw pixel information into high-dimensional feature representations, providing the foundational feature maps for subsequent object detection. This module's design draws on efficient feature extraction mechanisms, enabling it to extract rich visual features while maintaining computational efficiency, thereby improving the model's overall expressiveness and detection performance.

[0040] The feature pyramid fusion module 102 is used to fuse and enhance multi-scale features. Its purpose is to effectively integrate feature maps from different levels, enabling the model to simultaneously perceive information about both large and small objects. This is particularly important for pedestrian detection in complex road scenes, especially when facing distant pedestrians (small objects) or occluded pedestrians. This module helps improve the model's ability to recognize multi-scale objects, enhancing the robustness and generalization of detection.

[0041] The detection head module 103 is a key component for the final object detection task. It includes a C2f residual unit, a hybrid attention mechanism unit, and a decoupled detection unit. Its function is to further process the feature map and output detection results, such as the bounding box coordinates and confidence score of pedestrians. By introducing advanced structural design, this module enhances the model's ability to focus on key features, improving detection accuracy and stability.

[0042] The Cross Stage Partial with Feature Fusion (C2f) residual unit is a highly efficient feature extraction structure that enhances feature propagation and reuse while maintaining lightweightness, mitigating the vanishing gradient problem. It retains low-level details through residual connections and incorporates high-level semantic information, improving model training stability and detection performance, making it particularly suitable for real-time detection scenarios.

[0043] The hybrid attention mechanism unit consists of a channel attention sub-unit and a spatial attention sub-unit connected in series. Its function is to enable the model to learn which features are more worthy of attention in the channel and spatial dimensions, respectively, thereby improving the effectiveness of feature representation. Channel attention focuses on the importance of each channel in the feature map, while spatial attention focuses on the effectiveness of specific regions in the image. The combination of these two makes the model more adaptable to complex environments such as lighting changes and occlusion.

[0044] A hybrid attention mechanism unit is inserted after the C2f residual unit. The model network first performs convolution calculations and feature extraction, and then uses the C2f residual structure to eliminate gradients during the operation. This elimination reduces the model's computational complexity. The addition of the attention mechanism module at this time can better extract the perceptual domain from the image, thereby improving detection accuracy.

[0045] Attention mechanism is a technique commonly used in computer science and machine learning that can make the model more accurate and efficient when processing sequence data. It is also a flexible and efficient mechanism that can help machine learning models better handle sequence data. In traditional neural networks, the output of each neuron only depends on the output of all neurons in the previous layer, while in attention mechanism, the output of each neuron depends on the output of all neurons in the previous layer. This allows the model to pay more attention to key information in the input sequence, thereby improving the accuracy and efficiency of the model.

[0046] Hybrid attention mechanism combines the advantages of spatial attention mechanism and channel attention mechanism. Channel attention mechanism improves network performance by assigning different weights to different feature regions. Spatial attention mechanism improves network performance by establishing an intermediate layer and using parameter convolution to give different weights to feature maps. By combining these two attention mechanisms, the model can absorb the advantages of both spatial attention mechanism and channel attention mechanism, improving detection accuracy. In the hybrid attention mechanism algorithm, the input features are first sent to the channel attention mechanism for operation, and then the channel attention mechanism operation results are convolved with the original input features to obtain the preliminary feature extraction map. Then the preliminary feature extraction map obtained in the previous step is input into the spatial attention mechanism module for operation, and the obtained results are convolved with the preliminary feature extraction map. In this way, the output feature map is obtained.

[0047] The decoupled detection unit separates the bounding box regression task and the class classification task to reduce the mutual interference between them and improve the accuracy and convergence speed of detection. Compared with the traditional coupled detection method, the decoupled design can improve the stability of the model in complex scenarios, especially in dense pedestrian or variable posture situations, effectively reducing false positives and missed detections.

[0048] The decoupled detection unit separates the regression branch and the prediction branch, and uses the integral form of expression proposed in the Distribution Focal Loss strategy for the regression branch. The decoupled head decouples the classification and regression, and splits them into two independent branches. Experiments show that for the same detection task, using the decoupled detection head can speed up the convergence and improve the detection accuracy.

[0049] The distance attention WIoU loss function is used to optimize the model training process, and its core role is to introduce the center point distance factor when calculating the boundary box regression loss, dynamically adjust the gradient weight, and make the prediction box approach the real box faster and more accurately. Compared with the traditional IoU loss function, this method has better positioning ability when facing large target offset or small target detection, which helps to improve the detection accuracy and model convergence efficiency.

[0050] Specifically, to improve the performance of the YOLOv8 model, the original loss function is replaced with the Wise-IoU (WIoU) loss function. This improvement is mainly based on the dynamic non-monotonic focusing mechanism in WIoU, which evaluates the quality of anchor boxes by introducing an outlying degree, and accordingly implements a gradient gain distribution strategy. This strategy aims to reduce the competitive pressure of high-quality anchor boxes while reducing the negative gradient impact of low-quality samples, so that WIoU can focus on anchor boxes of ordinary quality, thereby enhancing the overall performance of the detector.

[0051] Considering that the training data set inevitably contains some low-quality samples, which are often affected by geometric factors such as distance and aspect ratio, and thus have an adverse effect on the generalization ability of the model. In this context, a good loss function needs to appropriately reduce the punishment for these geometric factors when the anchor box and the target box are highly overlapped, so that the model can achieve better generalization performance. Therefore, the distance attention mechanism is introduced in this framework, which dynamically adjusts the weight coefficient according to the distance between the center points, amplifies certain loss values, and guides the model to pay more attention to learning samples of medium or high quality. In addition, combined with the application of the IoU loss function, the deviation caused by the dynamic weight coefficient of the center distance can be balanced during optimization, ensuring that when the anchor box and the target box are well overlapped, the distance between their center points is optimized, avoiding interference from irrelevant geometric factors, and further improving the accuracy and robustness of the model.

[0052] In summary, the above-mentioned modules work together to build an improved YOLOv8 model that is efficient, accurate, and robust, effectively improving the pedestrian detection ability of the intelligent driving system in complex road environments, meeting the dual demands of real-time performance and safety for autonomous driving.

[0053] In some embodiments of the present application, the implementation process of performing size normalization preprocessing on the road environment image in step S200 to obtain a normalized road image includes but is not limited to the following steps.

[0054] In step S210, the road environment image is scaled to a fixed resolution according to the original aspect ratio to obtain a scaled and adapted image.

[0055] In step S210, the input image is uniformly adjusted to a fixed size required by the model while maintaining the original proportion of the image. This avoids introducing noise or information loss due to image proportion distortion, thereby ensuring the accuracy of subsequent feature extraction. By preserving the original aspect ratio, the deformation of the pedestrian target during scaling can be effectively prevented, improving detection accuracy.

[0056] In step S220, the blank areas generated due to the difference in aspect ratio in the scaled and adapted image are filled using a bilinear interpolation algorithm, and the pixel value of the filled pixels is the average of the edge pixels of the scaled and adapted image, to generate an edge average filled image.

[0057] In step S220, since the target size may not be completely filled after scaling in step S210, blank areas appear around the image. This step is used to fill these blanks. The bilinear interpolation combined with the average of the edge pixels is used for filling, which can maintain the consistency of the whole image, avoid the interference caused by random filling, reduce the influence of background noise on model judgment, and help to improve the robustness and stability of the model.

[0058] In step S230, the pixel value of the edge average filled image is normalized to linearly map the pixel value to the interval [0, 1] to generate a normalized road image.

[0059] In step S230, the pixel value of the image is standardized to make its distribution range consistent with the data set used during model training. By mapping the pixel value from [0, 255] to [0, 1], it helps to speed up the model convergence, improve the training efficiency, and enhance the model's adaptability to different lighting conditions, thereby further improving the accuracy and generalization performance of pedestrian detection.

[0060] In some embodiments of the present application, the road environment image is scaled to a fixed resolution according to the original aspect ratio to obtain a scaled and adapted image. The scaled and adapted image satisfies the following formula (1): (1), In formula (1), denotes the scaled and adapted image, denotes the road environment image, is the fixed width of the scaled and adapted image, is the fixed height of the scaled and adapted image, denotes the scaling operation.

[0061] In some embodiments of the present application, the resolution of the scaled and adapted image is 640x640.

[0062] In some embodiments of the present application, in step S300, the normalized road image is input into the improved YOLOv8 model to obtain the pedestrian detection result, and the implementation process includes but is not limited to the following steps.

[0063] In step S310, the normalized road image is input into the backbone network module 101, and the multi-scale feature map is extracted through the backbone network module 101.

[0064] In step S310, rich hierarchical feature information is extracted from the input image to provide strong support for subsequent processing. CSPDarknet, as the core component of the backbone network, can effectively capture details and complex patterns in the image; and the SPPF (Spatial Pyramid Pooling Fast) unit further enhances the model's perception ability for objects of different scales. The combination of the two not only improves the diversity of feature expression, but also enhances the recognition accuracy of the network for pedestrians of various sizes.

[0065] In step S320, the multi-scale feature map is input into the feature pyramid fusion module 102, and a high-resolution fused feature map is generated through upsampling and cross-level connection.

[0066] In step S320, the feature pyramid structure is used to integrate feature information from different layers to address the limitations of single-scale feature maps in handling multi-scale targets. Through upsampling and cross-level connection, detailed information from low layers and semantic information from high layers are effectively fused to generate a fused feature map with higher resolution. This step is particularly crucial for improving the detection performance of small targets in the pedestrian detection task, as small targets often require more precise spatial location information for accurate recognition.

[0067] In step S330, the fused feature map is input into the detection head module 103, which is processed in sequence through the C2f residual unit, the hybrid attention mechanism unit, and the decoupled detection unit, and the pedestrian detection result is output.

[0068] In step S330, the fused feature map is sent to the specially designed detection head module 103 for fine processing. The C2f residual unit helps to alleviate the gradient vanishing problem in deep networks, ensuring a more stable and efficient training process; the hybrid attention mechanism unit adjusts the importance weights of different regions adaptively, highlighting key areas and improving detection accuracy; finally, the decoupled detection unit independently handles the classification and bounding box regression tasks, simplifying the optimization process and improving the overall detection effect. Through this series of carefully designed processing procedures, the precise positioning and recognition of pedestrian targets are finally achieved.

[0069] In some embodiments of the present application, in step S330, the fused feature map is input into the detection head module 103, sequentially processed through a C2f residual unit, a mixed attention mechanism unit and a decoupled detection unit, and the pedestrian detection result is output, including: In step S331, the fused feature map is input into the C2f residual unit, and the shallow layer detail feature and the deep layer semantic feature are fused through the cross-stage partial connection to generate a multi-scale optimization feature map.

[0070] In step S331, the C2f residual unit is introduced to effectively combine the shallow layer and deep layer features. The design of the C2f residual unit not only helps to alleviate the gradient vanishing problem in deep networks, but also enhances the expressiveness of the model through local information sharing, so that the generated multi-scale optimization feature map can better capture the subtle structures and complex patterns in the image, laying a solid foundation for subsequent processing.

[0071] In step S332, the multi-scale optimization feature map is input into the channel attention subunit to generate a channel weight matrix, and the channel weight matrix is multiplied element by element with the multi-scale optimization feature map to output a channel weighted feature map.

[0072] In step S332, the channel attention mechanism is used to enhance the model's attention to important feature channels. First, the multi-scale optimization feature map is input into the channel attention subunit, which evaluates the importance of each channel by performing global average pooling on the information of each channel, thereby generating a channel weight matrix reflecting the importance of each channel. Next, this channel weight matrix is multiplied element by element with the original multi-scale optimization feature map to highlight channels containing key information while suppressing irrelevant or redundant information. The purpose of this is to improve the model's learning ability for useful features, thereby improving the accuracy of the pedestrian detection task.

[0073] In step S333, the multi-scale optimization feature map is input into the spatial attention subunit to generate a spatial weight matrix, and the spatial weight matrix is multiplied element by element with the multi-scale optimization feature map to output a spatial weighted feature map.

[0074] In step S333, the spatial attention mechanism is used to emphasize important local regions in the image. First, the multi-scale optimization feature map is input into the spatial attention subunit, where a two-dimensional spatial weight matrix is calculated by performing channel-wise pooling on the feature map, which represents the importance of each position in the image. Then, this spatial weight matrix is multiplied element by element with the original multi-scale optimization feature map to adjust the response intensity of different spatial positions and highlight regions containing pedestrians or other significant targets. This process helps the model better locate the exact position of the pedestrian, especially in complex backgrounds or under occlusion, improving the accuracy of detection.

[0075] In step S334, the spatially and channel-wise weighted feature maps are adaptively fused to obtain a spatial-channel feature map, which is input into a decoupled detection unit for processing by separate classification and regression branches to output a pedestrian detection result.

[0076] In step S334, the feature maps processed by channel attention and spatial attention are adaptively fused to form a spatial-channel feature map that integrates the advantages of both attention mechanisms. This fusion method not only preserves the sensitivity to key channels and spatial positions but also enhances the overall expressive ability of the model. Subsequently, the fused feature map is sent to a decoupled detection unit, where the classification branch is responsible for determining whether each candidate region contains a pedestrian, while the regression branch aims to predict the bounding box position of the pedestrian. By separating these two tasks, the mutual interference between them is reduced, allowing the model to focus more on its respective task and thus improving the accuracy and reliability of the final pedestrian detection result. This step ensures the optimization of the entire process from feature extraction to the final detection result, enhancing the performance of the intelligent driving system in complex environments.

[0077] In some embodiments of the present application, the channel attention subunit is divided into compression and excitation parts. The purpose of the compression part is to compress the global spatial information and then learn features in the channel dimension to form the importance of each channel. Finally, the excitation part assigns different weights to each channel. The main role of the compression operation is to enable the global information to be embedded in a low-dimensional model. The compression operation satisfies the following formula (2): (2); In formula (2), represents the compression output value of the i-th channel, and represents the average activation intensity of the channel in the global space. The role is to compress the two-dimensional spatial feature map into a scalar for generating a channel weight matrix later. represents the multi-scale optimized feature map corresponding to the i-th channel, and the shape is wherein represents the width of the feature map, represents the height of the feature map, is the output of a certain layer in the convolutional neural network, and the activation value of each position reflects the response intensity of the position to the i-th channel; represents a compression function, which is realized by global average pooling (GAP), that is, the mean value of all positions of each channel is taken to retain the global statistical information of the channel and suppress noise interference. ​​​​

[0078] The excitation part is implemented by using two fully connected layers, the first fully connected layer compresses the channels into channels to reduce the calculation amount, and the second fully connected layer restores the channels. is the compression ratio. The role of the fully connected layer is to train the channel weight matrix by using the correlation between the channels. The calculation of the excitation part satisfies the following formula (3): (3). In formula (3), is the compressed output value input into the excitation part, and the shape is , which is used to represent the global spatial statistical information of each channel; is the excitation weight matrix; represents the dimension reduction weight matrix, which is the weight matrix of the first fully connected layer, and the shape is , which is used to compress the channel number from to , reduce the calculation complexity, prevent overfitting, and in actual calculation, is equivalent to a linear transformation; represents the dimension increase weight matrix, which is the transpose of the weight matrix of the second fully connected layer, and the shape is , which is used to restore the channel number from to , reconstruct the channel dimension, and generate independent weight coefficients for each channel; represents a nonlinear activation function such as a ReLU function; represents a Sigmoid function; represents an excitation function; represents a channel weight matrix, and the shape is , each element , is larger, indicating that the feature of the th channel is more critical to the current task, and the response needs to be enhanced, is smaller, indicating that the feature of the th channel is redundant or seriously interfered by noise, and the contribution needs to be suppressed.

[0079] Then, the channel weight matrix is multiplied element by element with the multi-scale optimized feature map , and the spatial weighted feature map is output.

[0080] In some embodiments of the present application, when the compression ratio is 16, the effect of the excitation model is best.

[0081] In some embodiments of the present application, the spatial attention subunit dynamically learns the importance weights of different spatial positions in the feature map, so that the model focuses on the key areas related to the task and suppresses irrelevant background interference. Each spatial position of performs maximum pooling and average pooling along the channel dimension to aggregate channel information. The two pooling results are then concatenated and the spatial weight distribution is learned through a standard convolutional layer to obtain a spatial weight matrix that satisfies the following formula (4): (4); In formula (4), represents the spatial weight matrix, Represents multi-scale optimized feature map The result of maximum pooling is: Represents multi-scale optimized feature map The result of average pooling; Represents a splicing operation, Represents a convolutional layer.

[0082] Then, the spatial weight matrix and multi-scale optimized feature maps Multiply element by element and output spatial weighted feature map .

[0083] In some embodiments of the present application, the convolutional layer The shape is .

[0084] In some embodiments of the present application, a learnable parameter is introduced , the spatial weighted feature map and channel-weighted feature maps Perform adaptive fusion processing to obtain the spatial channel feature map The following formula (5) is satisfied: (5); In formula (5), the learnable parameters The following formula (6) is satisfied: (6); In formula (6), Represents the fully connected layer parameters, which are optimized through end-to-end training.

[0085] In some embodiments of the present application, the distance attention WIoU loss function adjusts the gradient of the IoU loss function by introducing a center distance dynamic weight coefficient.

[0086] The purpose of introducing the center distance dynamic weight coefficient is to dynamically adjust the gradient of the loss function according to the center point distance between the predicted bounding box and the real bounding box. Specifically, the center distance dynamic weight coefficient calculates the difference between the predicted pedestrian position bounding box center point coordinates and the real pedestrian position bounding box center point coordinates, and combines the normalized real bounding box size for weighted processing. In this way, the model can pay more attention to samples with larger center point distance, as these samples are usually more difficult to classify correctly and require more training to improve the performance of the model.

[0087] Based on the above center distance dynamic weight coefficient, the next step is to adjust the gradient of the IoU loss function. The IoU loss function is an important indicator for measuring the degree of overlap between the predicted bounding box and the real bounding box. By introducing the center distance dynamic weight coefficient, this step changes the sensitivity of the IoU loss function to different samples, especially giving greater weight to those with a larger center point distance, i.e., more difficult to accurately locate pedestrian targets. Such adjustment helps guide the model to learn more accurate bounding box positioning capabilities, especially in complex scenarios or when there are small targets.

[0088] In summary, by introducing the center distance dynamic weight coefficient to adjust the gradient of the IoU loss function, not only enhances the model's learning ability for difficult samples, but also improves the accuracy of the overall pedestrian detection task. This method is particularly suitable for applications such as autonomous driving, where high-precision pedestrian detection is crucial to ensuring road safety.

[0089] In some embodiments of the present application, the distance attention Wiou loss function satisfies the following formula (7): (7); In formula (7), denotes the distance attention Wiou loss function, denotes the center distance dynamic weight coefficient, denotes the IoU loss function. The center distance dynamic weight coefficient satisfies the following formula (8): (8); In formula (8), is the center point coordinate of the predicted pedestrian position bounding box. is the center point coordinate of the real pedestrian position bounding box. is the normalized width of the real pedestrian position bounding box. is the normalized height of the real pedestrian position bounding box.

[0090] In some embodiments of the present application, in step S400, according to the pedestrian detection result, an obstacle avoidance instruction is generated to control the vehicle to turn or brake, comprising: Step S410, when the confidence is greater than the preset steering confidence threshold, and the overlap area ratio of the pedestrian position bounding box and the vehicle driving trajectory is greater than the preset steering overlap area threshold, a steering instruction is generated to control the vehicle to steer.

[0091] In step S410, the potential interference degree of the pedestrian to the vehicle driving path is evaluated, and it is decided whether to avoid collision by steering. First, the system checks whether the confidence in the pedestrian detection result exceeds the preset steering confidence threshold to ensure the reliability of the detection. Then, the overlap area ratio between the pedestrian position bounding box and the vehicle predetermined driving trajectory is calculated, and if the ratio exceeds the preset steering overlap area threshold, it indicates that there is a risk of direct collision. In this case, the system will automatically generate a steering instruction to guide the vehicle to change direction and avoid the pedestrian, ensuring the safety of driving.

[0092] Step S420, when the confidence is greater than the preset braking confidence threshold, and the overlap area ratio of the pedestrian position bounding box and the vehicle driving trajectory is greater than the preset braking overlap area threshold, a braking instruction is generated to control the vehicle to brake.

[0093] In step S420, attention is focused on more urgent situations, i.e. the vehicle needs to be stopped immediately to prevent collision with the pedestrian. The system first confirms whether the confidence of the pedestrian detection result is higher than the preset braking confidence threshold to ensure the high accuracy of the detection. Then, the overlap area ratio between the pedestrian position bounding box and the vehicle driving trajectory is analyzed, and if the ratio exceeds the preset braking overlap area threshold, it indicates that the current situation is extremely dangerous and immediate action is required. At this time, the system will quickly generate a braking instruction to force the vehicle to slow down until it completely stops, maximizing the possibility of reducing accidents and ensuring the safety of pedestrians and passengers in the vehicle.

[0094] In some embodiments of the present application, the improved YOLOv8 model is trained using a small target replication and pasting enhancement method. The small target in the image to be trained is processed by randomly replicating and pasting the small target in the image to other positions, so as to improve the small target sample density and improve the training effect.

[0095] Secondly, the embodiment of the present application provides an intelligent driving pedestrian detection system for executing the intelligent driving pedestrian detection method described above.

[0096] In some embodiments of the present application, with reference to Figure 3 , the intelligent driving pedestrian detection system includes a data acquisition module 510, a data preprocessing module 520, a pedestrian detection module 530, and a vehicle obstacle avoidance module 540. The data acquisition module 510 is configured to acquire road environment images during intelligent driving.

[0097] The data preprocessing module 520 is used to perform size normalization preprocessing on the road environment image to obtain a normalized road image.

[0098] The pedestrian detection module 530 is used to input the normalized road image into the improved YOLOv8 model to obtain pedestrian detection results; wherein the pedestrian detection results include the pedestrian position bounding box and confidence level.

[0099] The vehicle obstacle avoidance module 540 is used to generate obstacle avoidance instructions based on pedestrian detection results and control vehicle steering or braking.

[0100] Furthermore, refer to Figure 4 An embodiment of the present application provides an intelligent driving pedestrian detection device, comprising: a processor 610, a memory 620, an on-board camera 630, and a vehicle control module 640. The on-board camera is used to obtain images of the road environment during intelligent driving. The memory is used to store programs. When the program is executed by the processor, the aforementioned intelligent driving pedestrian detection method is implemented, a pedestrian detection result is obtained, and an obstacle avoidance instruction is generated based on the pedestrian detection result. The vehicle control module is used to control the steering or braking of the vehicle based on the obstacle avoidance instruction.

[0101] In some embodiments of the present application, the vehicle-mounted camera is a multispectral camera that supports RGB and infrared image fusion input. By using a multispectral camera, the system can obtain richer environmental information under various lighting conditions. RGB cameras excel at capturing color images in daytime or good lighting conditions, while infrared cameras can provide clear thermal imaging in low-light or nighttime environments, helping to identify pedestrians or other heat-generating objects. This means that the system can maintain efficient pedestrian detection capabilities regardless of day or night.

[0102] In addition, an embodiment of the present application provides a vehicle that is integrated with the aforementioned intelligent driving pedestrian detection device.

[0103] In summary, the intelligent driving pedestrian detection method and its system, device and vehicle provided in the embodiments of the present application have the following technical effects.

[0104] The embodiment of the application first designs an improved YOLOv8 model, integrates a C2f residual unit, a mixed attention mechanism (including channel attention and spatial attention), and a decoupled detection structure, effectively enhancing the detection accuracy of small target pedestrians, occlusion interference, and complex lighting conditions. In particular, the distance attention Wiou loss function adjusts the gradient of the IoU loss function by introducing a center distance dynamic weight coefficient, further optimizing the positioning accuracy of the bounding box, so that the prediction box can more accurately approximate the real box. In addition, a multispectral camera that supports RGB and infrared image fusion input can maintain efficient pedestrian detection capability both during the day and at night, effectively improving the environmental adaptability and robustness of the system.

[0105] Based on the above technical advantages, the system can generate obstacle avoidance instructions to control the vehicle to turn or brake according to the pedestrian detection results, ensuring a quick and accurate response when encountering potential dangers, thereby greatly improving the safety and intelligence level of autonomous driving. Each link from data preprocessing to feature extraction to final pedestrian detection is carefully designed and optimized, solving the problem of high false negative rate and inaccurate positioning in the prior art. This method not only significantly improves the accuracy and reliability of pedestrian detection, but also provides strong technical support for achieving safer and more reliable intelligent driving. The overall optimization of the overall framework enables the intelligent driving system to operate stably in various complex environments, ensuring the safety of pedestrians and passengers.

[0106] It should be noted that in each specific embodiment of the present application, when relevant processing is required according to user information, user behavior data, user historical data, and user location information related to the user's identity or characteristics, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant national and regional laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain sensitive personal information of the user, it will obtain the user's separate permission or separate consent through a pop-up window or by jumping to a confirmation page, and after obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0107] In some alternative embodiments, the function / operations described in the block diagrams can not occur in the order described in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality / operations involved. Also, although the embodiments presented in the flow diagrams are shown as a sequence of operations, it is to be understood that the logical flow is merely illustrative of alternative embodiments. The disclosed methods are not limited to the order of operations presented herein. Alternative embodiments are contemplated in which the order of operations is changed, and in which sub-operations are performed in different orders or in parallel.

[0108] Further, although the present application has been described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also to be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation is within the routine of an engineer's knowledge given the property, functionality and internal relationships of the various functional modules disclosed in the devices shown herein. Accordingly, the present application is not limited to purely hardware implementations, but also encompasses software implementations and / or firmware implementations. It is also to be understood that the disclosed specific concepts are merely illustrative and not intended to limit the scope of the present application, which is defined by the appended claims and their equivalents.

[0109] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the technical solutions that make essential contributions to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of programs for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0110] The logic and / or steps represented in the flow diagrams or otherwise described herein, for example, can be considered as a sequence of executable program instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via an optical scanner, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0111] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via an optical scanner, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0112] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above described embodiments, the various steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies known in the art, or combinations thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0113] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "some embodiments", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in the embodiments or examples of the present specification. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0114] While the embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary and are not to be taken as limiting the scope of the application. The scope of the application is defined by the claims and their equivalents.

[0115] The above is the specific description of the preferred embodiment of the application, but the application is not limited to the embodiment, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the application.

Claims

1. An intelligent driving pedestrian detection method, characterized in that: The steps include: Acquire road environment images during intelligent driving; performing size normalization preprocessing on the road environment image to obtain a normalized road image; Inputting the normalized road image into the improved YOLOv8 model to obtain a pedestrian detection result; The pedestrian detection result includes a pedestrian position bounding box and a confidence level; Generate obstacle avoidance instructions based on the pedestrian detection results to control vehicle steering or braking; The improved YOLOv8 model includes a backbone network module, a feature pyramid fusion module and a detection head module; The detection head module includes a C2f residual unit, a hybrid attention mechanism unit and a decoupling detection unit; the hybrid attention mechanism unit is connected after the C2f residual unit; the hybrid attention mechanism unit is composed of a channel attention subunit and a spatial attention subunit connected in series; The improved YOLOv8 model is optimized by the distance attention WIoU loss function.

2. The intelligent driving pedestrian detection method according to claim 1, characterized in that: Inputting the normalized road image into the improved YOLOv8 model to obtain pedestrian detection results includes: Inputting the normalized road image into the backbone network module, and extracting a multi-scale feature map through the backbone network module; Inputting the multi-scale feature map into the feature pyramid fusion module, generating a high-resolution fused feature map through upsampling and cross-level connection; The fused feature map is input into the detection head module, processed in sequence by the C2f residual unit, the hybrid attention mechanism unit and the decoupling detection unit, and the pedestrian detection result is output.

3. The intelligent driving pedestrian detection method according to claim 2, characterized in that: The fused feature map is input into the detection head module, processed in sequence by the C2f residual unit, the hybrid attention mechanism unit and the decoupling detection unit, and the pedestrian detection result is output, including: Inputting the fused feature map into the C2f residual unit, fusing shallow detail features and deep semantic features through cross-stage partial connections to generate a multi-scale optimized feature map; Input the multi-scale optimized feature map into the channel attention subunit, generate a channel weight matrix, and multiply the multi-scale optimized feature map element by element, and output a channel weighted feature map; Input the multi-scale optimized feature map into the spatial attention subunit, generate a spatial weight matrix, and multiply the spatial weight matrix with the multi-scale optimized feature map element by element, and output a spatial weighted feature map; The spatial weighted feature map and the channel weighted feature map are adaptively fused to obtain a spatial channel feature map, which is input into the decoupled detection unit. The pedestrian detection result is output through separate classification branch and regression branch processing.

4. The intelligent driving pedestrian detection method according to claim 1, characterized in that: The road environment image is subjected to size normalization preprocessing to obtain a normalized road image, including Scaling the road environment image to a fixed resolution according to the original aspect ratio to obtain a scaled adapted image; Filling blank areas in the scaled adaptive image due to aspect ratio differences with a bilinear interpolation algorithm, with the filling pixel value being the mean of the edge pixels of the scaled adaptive image, to generate an edge mean-filled image; The pixel values ​​of the edge mean filled image are normalized, and the pixel values ​​are linearly mapped to the interval [0, 1] to generate the normalized road image.

5. The intelligent driving pedestrian detection method according to claim 1, characterized in that: The distance attention WIoU loss function adjusts the gradient of the IoU loss function by introducing a dynamic weight coefficient of the center distance; The distance attention WIoU loss function satisfies the following formula: ； in, Represents the distance attention WIoU loss function, represents the dynamic weight coefficient of the center distance, represents the IoU loss function; The center distance dynamic weight coefficient satisfies the following formula: ； in, The center point coordinates of the predicted pedestrian position bounding box; The coordinates of the center point of the actual pedestrian position bounding box; is the normalized width of the actual pedestrian position bounding box; is the normalized height of the bounding box of the actual pedestrian position.

6. The intelligent driving pedestrian detection method according to claim 1, characterized in that: The generating of obstacle avoidance instructions and controlling vehicle steering or braking according to the pedestrian detection result includes: When the confidence level is greater than a preset steering confidence level threshold, and the overlapping area ratio between the pedestrian position bounding box and the vehicle's driving trajectory is greater than a preset steering overlapping area threshold, a steering instruction is generated to control the vehicle steering; When the confidence is greater than a preset braking confidence threshold, and the overlapping area ratio between the pedestrian position boundary box and the vehicle driving trajectory is greater than a preset braking overlapping area threshold, a braking instruction is generated to control the vehicle braking.

7. An intelligent driving pedestrian detection system, characterized in that: Used to execute the intelligent driving pedestrian detection method as described in any one of claims 1-6.

8. An intelligent driving pedestrian detection device, characterized in that: include: Processors, memory, onboard cameras, and vehicle control modules; The vehicle-mounted camera is used to obtain road environment images during intelligent driving; The memory is used to store programs; When the program is executed by the processor, the intelligent driving pedestrian detection method according to any one of claims 1 to 6 is implemented, a pedestrian detection result is obtained, and an obstacle avoidance instruction is generated according to the pedestrian detection result; The vehicle control module is used to control the steering or braking of the vehicle according to the obstacle avoidance instruction.

9. The intelligent driving pedestrian detection device according to claim 8, characterized in that: The vehicle-mounted camera is a multispectral camera that supports RGB and infrared image fusion input.

10. A vehicle, characterized in that: The intelligent driving pedestrian detection device according to claim 8 or 9 is integrated.

Images