An automatic driving method, system and vehicle based on target pedestrian detection

Abstract

The application discloses an automatic driving method and system based on target pedestrian detection and a vehicle, and the method comprises the following steps: acquiring image data sets of a scene around an automatic driving vehicle; the image data sets contain multiple person images; inputting each person image in the image data sets into a trained pedestrian detection model; obtaining predicted position information and predicted attribute information of each person in the image data sets by using the pedestrian detection model; wherein the attribute information of the person is the type of the person; and controlling a driving route according to the predicted position information and the predicted attribute information. The application detects the position of a person around the vehicle and the type of the person, plans a driving route according to the position of the person and the type of the person, avoids the collision between the vehicle and the person, and improves the safety of automatic driving, thereby providing a guarantee for safe driving of the vehicle.

Description

Technical Field

[0001] This application relates to the field of automotive manufacturing technology, specifically to an autonomous driving method, system, and vehicle based on target pedestrian detection. Background Technology

[0002] Pedestrian detection is a benchmark problem in object detection and has great practical value in scenarios such as autonomous driving, intelligent transportation, passenger flow statistics and monitoring.

[0003] In autonomous driving scenarios, driving safety is a critical issue. The accuracy of vehicle and pedestrian location detection directly impacts driving safety; therefore, improving the accuracy of vehicle and pedestrian location prediction is of paramount importance.

[0004] In the existing technology, there are already public methods for predicting pedestrian targets using neural networks. However, these methods are often applied in scenarios such as shopping malls or fixed roadside monitoring equipment, which are generally detections in fixed scenarios. Furthermore, the detections in the existing technologies are generally single-aspect detections of pedestrian targets. Predicting different features of pedestrian targets requires a cascaded approach, using multiple prediction steps. Therefore, the prediction efficiency of pedestrian target features is low, resulting in low safety for autonomous vehicles that plan driving routes based on the prediction results of pedestrian target features.

[0005] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0006] This application provides an autonomous driving method, system, and vehicle based on target pedestrian detection to address the problem of low safety in autonomous vehicles caused by the low prediction efficiency and accuracy of existing neural network models for pedestrian location and attribute classification.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] The first aspect of this application provides an autonomous driving method based on target pedestrian detection, comprising the following steps:

[0009] Acquire an image dataset of the scene surrounding the autonomous vehicle; the image dataset contains multiple images of people;

[0010] Each person image in the image dataset is input into a trained pedestrian detection model. The pedestrian detection model is then used to obtain the predicted location information and predicted attribute information of each pedestrian in each person image in the image dataset. The attribute information of the pedestrian is the type to which the pedestrian belongs.

[0011] The driving route of the autonomous vehicle is controlled based on the predicted location information and the predicted attribute information.

[0012] Based on the above-mentioned technical means, this application embodiment detects the location and type of pedestrians around the vehicle, and plans the driving route according to the location and type of pedestrians, thereby avoiding collisions between vehicles and pedestrians and improving the safety of autonomous driving.

[0013] Optionally, in one embodiment of this application, before the step of inputting the image dataset into the trained pedestrian detection model, the method further includes:

[0014] Obtain a sample image set, which contains images of multiple pedestrian targets, and each image is labeled with the location and attribute information of the pedestrian target;

[0015] The preset neural network model is trained using a set of sample images to obtain a trained pedestrian detection model.

[0016] Based on the above technical means, this application embodiment uses pedestrian target images in a known sample dataset to train a preset neural network model to obtain a trained pedestrian detection model. Since the sample images are labeled with the location and attribute information of pedestrian targets, the trained pedestrian detection model can also achieve the purpose of detecting the location and attribute information of pedestrian targets in the input image.

[0017] Optionally, in one embodiment of this application, the preset neural network model includes: a backbone network, a neck network, and a head network connected in sequence;

[0018] The step of training a preset neural network model using a sample image set to obtain a trained pedestrian detection model includes:

[0019] The sample images in the sample image set are sequentially input into the backbone network to obtain the image feature information output by the backbone network;

[0020] The image feature information is input into the neck network to obtain the multi-scale fusion feature information output by the neck network;

[0021] The multi-scale fused feature information is input into the head network to obtain the predicted pedestrian location information and pedestrian attribute information output by the head network.

[0022] The parameters of the preset neural network model are adjusted based on the predicted pedestrian location information and pedestrian attribute information, as well as the true values ​​of pedestrian location information and pedestrian attribute information marked in the sample images in the sample image set, to obtain the basic model.

[0023] The basic model is pruned and compressed to obtain the pedestrian detection model.

[0024] According to the above technical means, the embodiments of this application use a sample image set to train a preset neural network model to obtain a basic model, and then obtain a pedestrian detection model based on the basic model. Since the pedestrian detection model provided in this embodiment is obtained by pruning and compressing the basic model, the pedestrian detection model provided in this application has a simple structure, high detection effect, and can simultaneously predict pedestrian position and pedestrian attribute information, thus achieving a better prediction effect.

[0025] Optionally, in one embodiment of this application, the constructed preset neural network model includes: each convolutional module in the backbone network, neck network and head network includes: a convolutional layer and a normalization layer;

[0026] The step of pruning and compressing the base model to obtain the pedestrian detection model includes:

[0027] The gamm parameter regularization loss constraint is used to train each normalization layer, and channels with gamm parameters below a preset threshold are pruned to obtain the pruned compressed model.

[0028] The parameters of the compressed model are fine-tuned to obtain the pedestrian detection model with fine-tuned parameters.

[0029] Based on the above technical means, in the embodiment of this application, when training the pedestrian detection model, the constraint training of the gamm parameter in the normalization layer is added, and channels with gamm parameters lower than the preset value are deleted, thereby achieving the purpose of simplifying the model structure and improving the prediction efficiency of the model.

[0030] Optionally, in one embodiment of this application, the step of fine-tuning the parameters of the compressed model to obtain a pedestrian detection model with fine-tuned parameters includes:

[0031] The trained model parameters are imported, and regularization constraints are applied to the normalization layer according to the preset learning rate. When the error value of the trained model output is within the preset range, the pedestrian detection model is obtained. The error value includes the error value between the predicted value and the true value of pedestrian location information, and the error value between the predicted value and the true value of pedestrian attribute information.

[0032] Based on the above technical means, the embodiments of this application can simplify the model structure while maintaining the prediction effect before model simplification, thereby effectively improving the prediction efficiency and maintaining the accuracy of the prediction data.

[0033] Optionally, in one embodiment of this application, the step of acquiring an image dataset of the scene surrounding the autonomous vehicle includes:

[0034] Collect scene images from various directions in front of the autonomous vehicle;

[0035] The collected scene images are preprocessed to filter out images of people that contain pedestrian targets and whose content similarity does not exceed a preset threshold.

[0036] The selected images of people form an image dataset.

[0037] Based on the aforementioned technical means, the method disclosed in this application embodiment acquires scene images around the autonomous vehicle and predicts whether there are pedestrians in the surrounding environment and their positions and attributes based on the scene images. Since the acquisition interval is directly related to the prediction of pedestrians in the scene, if the acquisition interval is long, it may cause a large deviation between the actual position and the predicted position of the pedestrian. If the acquisition time is short, it will not only increase the workload of image prediction but also lead to a large amount of repetitive image processing. Therefore, in this step, the acquired scene images are preprocessed first to filter out images containing pedestrian targets and whose image content is not repeated. The images containing pedestrians are then used to predict the pedestrian positions and attributes, thereby achieving accurate prediction of pedestrian information and improving efficiency.

[0038] Optionally, in one embodiment of this application, the pedestrian detection model includes: a feature extraction module, a feature fusion module, and a prediction output module;

[0039] The steps of inputting each person image in the image dataset into a trained pedestrian detection model, and using the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in each person image in the image dataset, include:

[0040] Each person's image is sequentially input into the feature extraction module to obtain multiple pedestrian feature information extracted by the feature extraction module;

[0041] The pedestrian feature information is sequentially input into the feature fusion module to obtain the pedestrian feature fusion information output by the feature fusion module;

[0042] The pedestrian feature fusion information is input into the prediction output module to obtain the predicted location information and predicted attribute information of the pedestrian output module.

[0043] Based on the above technical means, in specific applications, the embodiments of this application utilize the various modules in the pedestrian detection model to extract pedestrian features from the image of a person in turn, thereby extracting the accurate location and accurate attributes of the pedestrian in the image.

[0044] Optionally, in one embodiment of this application, the step of controlling the driving route based on the predicted location information and the predicted attribute information includes:

[0045] Based on the current location information of the autonomous vehicle, the current motion state of the autonomous vehicle, the predicted location information of the pedestrian, and the predicted attribute information of the pedestrian, a driving route map is constructed.

[0046] Control the vehicle to drive automatically according to the described driving route map.

[0047] Based on the above technical means, the embodiments of this application can construct a driving route map based on the vehicle's own location and driving speed to avoid collisions with surrounding pedestrians. Since the pedestrian's predicted location and attributes are accurate, the driving route map constructed can avoid collision events and improve the safety of autonomous vehicles.

[0048] A second aspect of this application provides an autonomous driving system based on target pedestrian detection, comprising:

[0049] The image acquisition module is used to acquire image datasets of the scene surrounding the autonomous vehicle;

[0050] The pedestrian detection module is used to input the images of each person in the image dataset into the trained pedestrian detection model, and use the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in the images of each person in the image dataset; wherein, the attribute information of the pedestrian is the type to which the pedestrian belongs.

[0051] A driving control module is used to control the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information.

[0052] A third aspect of this application provides a vehicle, including: a memory, a processor, and an autonomous driving program based on target pedestrian detection stored in the memory and executable on the processor. When the processor executes the autonomous driving program based on target pedestrian detection, it implements the steps of the autonomous driving method based on target pedestrian detection.

[0053] The beneficial effects of this application are:

[0054] This invention discloses an autonomous driving method based on target pedestrian detection. It acquires an image dataset of the scene surrounding the autonomous vehicle; the image dataset contains multiple images of people; each image of a person in the image dataset is input into a trained pedestrian detection model, which is used to obtain predicted location information and predicted attribute information of each pedestrian in each image of the image dataset; wherein the pedestrian attribute information is the type of pedestrian; the driving route is controlled based on the predicted location information and the predicted attribute information. This embodiment of the application detects the location and type of pedestrians around the vehicle, and plans the driving route based on the pedestrian's location and type, avoiding collisions between the vehicle and pedestrians, thereby improving the safety of autonomous driving and providing a guarantee for safe driving. Attached Figure Description

[0055] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0056] Figure 1 A flowchart illustrating the steps of the autonomous driving method based on target pedestrian detection provided in this application;

[0057] Figure 2 A schematic diagram of the overall structure of the pedestrian detection model provided in this application;

[0058] Figure 3 A schematic diagram of the backbone network structure provided for this application;

[0059] Figure 4 A schematic diagram of the bottleneck structure provided in this application;

[0060] Figure 5 A schematic diagram of the neck network structure provided in this application;

[0061] Figure 6 A schematic diagram of the header network structure provided in this application;

[0062] Figure 7 A schematic diagram of the model cutting provided in this application;

[0063] Figure 8 A schematic diagram of the principle structure of the autonomous driving system based on target pedestrian detection provided in this application;

[0064] Figure 9 This is a block diagram illustrating the internal structure of a vehicle as provided in an embodiment of this application. Detailed Implementation

[0065] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0066] Because the numerous weights to be trained within a neural network model often consume significant amounts of memory and storage bandwidth (e.g., AlexNet ~200MB, VGG-16 ~500MB), deploying the model on embedded platforms and mobile devices presents considerable challenges. Furthermore, the massive computational load is also extremely high, resulting in substantial costs. Therefore, model quantization and pruning become an essential step in deployment. The goal is to minimize model parameters while maintaining model accuracy (or with only acceptable accuracy loss), thereby improving the model's operational efficiency on edge devices and meeting the algorithm's real-time performance requirements.

[0067] In existing neural networks for predicting pedestrian targets, only one feature of the pedestrian target is typically extracted. However, to achieve accurate detection of pedestrian targets and better planning of vehicle driving routes, multiple features of the pedestrian target need to be extracted. Existing methods require feature extraction through multiple detection models or multiple steps to obtain multifaceted information, resulting in low detection efficiency and failing to meet the accuracy requirements of autonomous driving.

[0068] To address the aforementioned issues, this embodiment provides an autonomous driving method, system, and vehicle based on target pedestrian detection. The method involves acquiring an image dataset of the scene surrounding the autonomous vehicle; this image dataset contains multiple images of people. Each image of a person in the image dataset is input into a trained pedestrian detection model. The model then obtains predicted location information and predicted attribute information for each pedestrian in the image dataset. Based on the predicted location information and predicted attribute information, the driving route of the autonomous vehicle is controlled. Therefore, this embodiment can simultaneously provide multiple safety guarantees, improving the safety and reliability of the vehicle.

[0069] The method, system, and vehicle provided by the present invention will be further described in detail below with reference to the accompanying drawings.

[0070] The first aspect of this application provides an autonomous driving method based on target pedestrian detection, such as... Figure 1 As shown, it includes the following steps:

[0071] Step S1: Obtain an image dataset of the scene surrounding the autonomous vehicle; the image dataset contains multiple images of people.

[0072] When an autonomous vehicle is driving on the road, it collects images of the surrounding environment through multiple cameras installed on the vehicle, and constructs an image dataset based on the surrounding environment images.

[0073] Because road conditions are complex and constantly changing, and the road conditions encountered during driving are also changing in real time, accurately obtaining the precise location and attribute information of pedestrians on the road requires acquiring images of the surrounding environment at preset intervals and detecting pedestrian targets in these images. It's conceivable that the shorter the preset interval, the more accurate the pedestrian target localization; however, a short preset interval also increases the amount of feature extraction required for image processing, resulting in a large memory footprint. Conversely, a long preset interval reduces the accuracy of pedestrian target location and attribute detection, but it doesn't require processing a large number of images, thus not significantly increasing the data processing workload. Therefore, this step aims to adjust the preset interval as much as possible while ensuring detection accuracy, minimizing the amount of image data processed while maintaining the required accuracy.

[0074] This step includes: acquiring scene images from various directions in front of the autonomous vehicle; preprocessing the acquired scene images to select images of people that contain pedestrian targets and whose content similarity does not exceed a preset threshold; and constructing an image dataset from the selected images of people.

[0075] In this step, scene images around the autonomous vehicle are collected. Based on the scene images, the presence of pedestrians in the surrounding environment and their location and attributes are predicted. In order to reduce unnecessary data processing, the collected scene images are preprocessed in this step to filter out images containing pedestrian targets and whose content is not repeated. The pedestrian location and attributes are predicted for the images containing pedestrians, thereby achieving accurate prediction of pedestrian information and improving efficiency.

[0076] Step S2: Input the images of each person in the image dataset into the trained pedestrian detection model, and use the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in the images of each person in the image dataset; wherein, the attribute information of the pedestrian is the type to which the pedestrian belongs.

[0077] The selected images of people are input into the pre-trained pedestrian detection model. The pedestrian detection model extracts features from the received images of people to obtain the location and attribute information of the extracted images of people.

[0078] Specifically, the location information refers to the pedestrian's location when the image was captured. The autonomous vehicle's processing system transforms the coordinate system based on the pedestrian's position in the image to obtain the pedestrian's location information in the real scene. The attribute information includes: pedestrian age, pedestrian posture, and pedestrian occlusion information. In this step, a pedestrian detection model is used to simultaneously predict both the pedestrian's location and attribute information to obtain the final prediction result.

[0079] Furthermore, in one embodiment of this application, before the step of inputting the image dataset into the trained pedestrian detection model, the method further includes:

[0080] Obtain a sample image set, which contains images of multiple pedestrian targets, and each image is labeled with the location and attribute information of the pedestrian target;

[0081] The preset neural network model is trained using a set of sample images to obtain a trained pedestrian detection model.

[0082] This application embodiment uses pedestrian target images from a known sample dataset to train a preset neural network model, thereby obtaining a trained pedestrian detection model. Since the sample images are labeled with the location and attribute information of pedestrian targets, the trained pedestrian detection model can also achieve the purpose of detecting the location and attribute information of pedestrian targets in the input image.

[0083] like Figure 2 The diagram shows the overall structure of the pedestrian detection model, which includes a feature extraction module 10, a feature fusion module 20, and a prediction output module 30.

[0084] The steps of inputting each person image in the image dataset into a trained pedestrian detection model, and using the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in each person image in the image dataset, include:

[0085] Each person's image is sequentially input into the feature extraction module to obtain multiple pedestrian feature information extracted by the feature extraction module; the pedestrian feature information is sequentially input into the feature fusion module to obtain pedestrian feature fusion information output by the feature fusion module; the pedestrian feature fusion information is input into the prediction output module to obtain the predicted location information and predicted attribute information of the pedestrian output by the prediction output module.

[0086] Combination Figure 2 and 3As shown, the feature extraction module is the backbone network structure, containing multiple convolutional modules and multiple downsampling modules. Through these modules, the input image undergoes n1 convolutions in the first stage, n2 convolutions in the second stage, n3 convolutions in the third stage, n4 convolutions in the fourth stage, and n5 convolutions in the fifth stage, respectively, generating feature maps containing pedestrian feature information that are downsampled by 2x, 4x, 8x, 16x, and 32x. Here, n1, n2, n3, n4, and n5 are positive integers and can be customized as needed. Figure 4 As shown, the backbone network structure adopts a bottleneck structure to first reduce the dimensionality of the features, and then uses 3x3 convolution to extract image features. The bottleneck structure can perform data training and feature extraction more effectively and intuitively in low-dimensional space, and effectively reduce the number of parameters.

[0087] Combination Figure 5 As shown, the feature fusion module 20 receives feature maps downsampled by 8x, 16x, and 32x from the feature extraction module. First, it upsamples the deep feature map and concatenates it with the shallow feature map. Then, it downsamples the shallow feature map and concatenates it with the deep feature map. Finally, it outputs fused feature maps downsampled by 8x, 16x, and 32x.

[0088] Combination Figure 6 As shown, the prediction output module 30 receives the feature maps input from the feature fusion module 20 and outputs the predicted location information and attribute information of the pedestrian target. The attribute information includes: pedestrian age, pedestrian pose, and pedestrian occlusion attributes. The output of the prediction output module includes: pedestrian location (bbox_reg), pedestrian occlusion attributes (occ_cls), pedestrian target attributes (obj_cls), pedestrian age (age_cls), and pedestrian pose attributes (pose_cls).

[0089] Step S3: Control the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information.

[0090] Once the location and attribute information of pedestrians around the autonomous vehicle are predicted, the driving route of the autonomous vehicle is constructed based on the operating status of the autonomous vehicle itself, the location of each pedestrian, and the movement status of each pedestrian. The driving is controlled based on the constructed driving route. It can be imagined that the driving route of the vehicle is updated in real time according to the prediction results of different pedestrian locations and attributes to ensure the safety of the vehicle during driving.

[0091] In one embodiment of this application, the step of controlling the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information includes:

[0092] Based on the current location information of the autonomous vehicle, the current motion state of the autonomous vehicle, the predicted location information of the pedestrian, and the predicted attribute information of the pedestrian, a driving route map is constructed, and the vehicle is controlled to drive autonomously according to the driving route map.

[0093] The embodiments of this application can construct a driving route map based on the vehicle's own location and driving speed to avoid collisions with surrounding pedestrians. Since the pedestrian positions and pedestrian attributes predicted by this method are accurate, the driving route map that can be constructed can avoid collision events, thereby improving the safety of autonomous vehicles.

[0094] The method described in this application improves the safety of autonomous driving by detecting the location and type of pedestrians around the vehicle and planning the driving route based on the location and type of pedestrians.

[0095] In one embodiment, the pedestrian detection module proposed in this embodiment is trained based on a preset neural network model, which includes a backbone network, a neck network, and a head network connected in sequence.

[0096] Specifically, the step of training a preset neural network model using a sample image set to obtain a trained pedestrian detection model includes:

[0097] The sample images in the sample image set are sequentially input into the backbone network to obtain the image feature information output by the backbone network;

[0098] The image feature information is input into the neck network to obtain the multi-scale fusion feature information output by the neck network;

[0099] The multi-scale fused feature information is input into the head network to obtain the predicted pedestrian location information and pedestrian attribute information output by the head network.

[0100] The parameters of the preset neural network model are adjusted based on the predicted pedestrian location information and pedestrian attribute information, as well as the true values ​​of pedestrian location information and pedestrian attribute information marked in the sample images in the sample image set, to obtain the basic model.

[0101] The basic model is pruned and compressed to obtain the pedestrian detection model.

[0102] In a specific implementation, the steps for training the pedestrian detection model include:

[0103] Data collection: Human data is collected by a data collection vehicle under different road conditions, time periods, weather conditions, and scenarios to ensure full-scenario data coverage.

[0104] Data cleaning: Cleaning the collected data to avoid invalid or duplicate data.

[0105] Data annotation: The cleaned data is annotated with the target location and target attributes. In the image, the following are annotated for each pedestrian: Target location: The location of the pedestrian in the image is bounded by a rectangle (bbox); Target attributes: Pedestrian age: child, teenager, adult, elderly; Pedestrian posture: walking, sitting, squatting, lying down, cycling, running; Pedestrian occlusion: fully visible, partially occluded, severely occluded, completely invisible.

[0106] Data processing: Formatting the labeled human images and labeled data, as well as data transformation and enhancement before inputting them into the model.

[0107] Model Design: Design the backbone network, neck network, head network, and other modules of the model to achieve parallel target detection and attribute classification network structure.

[0108] Model training: Adjust the model training parameters to train a basic model that meets the desired effect, which will serve as a reference indicator for subsequent pruning of the model.

[0109] Model pruning: Regularization loss of the gamma parameter of the BN layer is added for constrained training, and gamma layers below the threshold are pruned to obtain a pruned compressed model.

[0110] Model fine-tuning: Since the model's performance deteriorates due to missing parameters after pruning and compression, it is necessary to fine-tune the pruned model to approximate the performance of the base model and finally obtain the trained pedestrian detection model.

[0111] As can be seen from the above steps, in order to achieve the effect of simple structure and high prediction accuracy of the pedestrian detection model proposed in this embodiment based on neural network model training, a basic model is first trained in this embodiment, and then the basic model is pruned and compressed to obtain the pedestrian detection model proposed in this application.

[0112] The principle of pruning and compressing the basic model in this embodiment is as follows:

[0113] Currently, mainstream convolutional neural networks typically consist of a Conv layer combined with a batchnorm layer. In each iteration, the batchnorm layer calculates the mean and variance of each channel in the current batch of data, then normalizes the input data based on the calculation results to approximate a normal distribution, thereby accelerating model convergence. Finally, it multiplies the data of each channel by gamma and adds beta to restore the distribution.

[0114]

[0115]

[0116]

[0117]

[0118] Here, gamma and beta are trainable parameters that act on each channel of the feature map.

[0119] The main idea behind the pruning in this embodiment is to add an L1 regularization term to the gamma parameter of the normalized layer during training. During training, the gamma parameter of unimportant channels gradually decreases. When it falls below a certain set threshold, the corresponding channel is pruned, thereby reducing model parameters and achieving a lightweight model. A schematic diagram of the pruning principle is shown below. Figure 7 As shown.

[0120] Specifically, the basic model includes a backbone network, a neck network, and a head network, and each convolutional module in the backbone network, neck network, and head network includes a convolutional layer and a normalization layer.

[0121] The step of pruning and compressing the base model to obtain the pedestrian detection model includes:

[0122] The gamm parameter regularization loss constraint is used to train each normalization layer, and channels with gamm parameters below a preset threshold are pruned to obtain the pruned compressed model.

[0123] The parameters of the compressed model are fine-tuned to obtain the pedestrian detection model with fine-tuned parameters.

[0124] Based on the above technical means, combined with Figure 7 As shown, in this embodiment of the application, when training the pedestrian detection model, the constraint training of the gamm parameter in the normalization layer is added, and channels with gamm parameters lower than the preset value are deleted, thereby achieving the purpose of simplifying the model structure and improving the prediction efficiency of the model in one embodiment of the application.

[0125] The steps for fine-tuning the parameters of the compressed model to obtain the pedestrian detection model with fine-tuned parameters include:

[0126] The trained model parameters are imported, and regularization constraints are applied to the normalization layer according to the preset learning rate. When the error value of the trained model output is within the preset range, the pedestrian detection model is obtained. The error value includes the error value between the predicted value and the true value of pedestrian location information, and the error value between the predicted value and the true value of pedestrian attribute information.

[0127] Specifically, the learning rate is set to 1 / 10 of the learning rate used when training the basic model. The normalization layer is then trained again with regularization constraints. When the prediction error of the trained model meets the preset requirements, the model parameters are fixed to obtain the pedestrian detection model proposed in this embodiment.

[0128] This application's embodiments can simplify the model structure while maintaining the predictive performance before simplification, thus effectively improving prediction efficiency and preserving the accuracy of the predicted data. Furthermore, the pedestrian detection model proposed in this embodiment differs from the cascaded approach of first outputting the target pedestrian's location using a detection model and then inputting the ROI region into an attribute classification model to predict the target's category. This patent's single-stage approach, which simultaneously predicts the target's location and attributes, avoids redundancy in the process and greatly improves the operational efficiency of the autonomous driving pedestrian detection and attribute classification model after deployment in real vehicles.

[0129] A second aspect of this application provides an autonomous driving system based on target pedestrian detection, such as... Figure 8 As shown, it includes:

[0130] The image acquisition module 100 is used to acquire image datasets of the scene around the autonomous vehicle; its function is as described in step S1.

[0131] The pedestrian detection module 200 is used to input the images of each person in the image dataset into the trained pedestrian detection model, and use the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in the images of each person in the image dataset; wherein, the attribute information of the pedestrian is the type to which the pedestrian belongs; its function is as described in step S2.

[0132] The driving control module 300 is used to control the driving route of the autonomous vehicle according to the predicted location information and the predicted attribute information, and its function is as described in step S3.

[0133] Figure 9 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0134] The memory 901, the processor 902, and the computer program stored on the memory 901 and capable of running on the processor 902.

[0135] When the processor 902 executes the program, it implements the steps of the autonomous driving method based on target pedestrian detection provided in the above embodiments.

[0136] Furthermore, the vehicle also includes:

[0137] Communication interface 903 is used for communication between memory 901 and processor 902.

[0138] The memory 901 is used to store computer programs that can run on the processor 902.

[0139] The memory 901 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0140] If the memory 901, processor 902, and communication interface 903 are implemented independently, then the communication interface 903, memory 901, and processor 902 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0141] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0142] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0143] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described autonomous driving method based on target pedestrian detection.

[0144] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0145] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0146] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0147] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can read and execute instructions from or in conjunction with such an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). In addition, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically by optically scanning paper or other media, then editing, interpreting or otherwise processing them as necessary, and then storing them in computer memory.

[0148] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0149] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0151] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An automatic driving method based on target pedestrian detection, characterized by, Includes the following steps: Acquire an image dataset of the scene surrounding the autonomous vehicle; the image dataset contains multiple images of people; Each person image in the image dataset is input into a trained pedestrian detection model. The pedestrian detection model is then used to obtain the predicted location information and predicted attribute information of each pedestrian in each person image in the image dataset. The attribute information of the pedestrian is the type to which the pedestrian belongs. The driving route of the autonomous vehicle is controlled based on the predicted location information and the predicted attribute information. The step of controlling the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information includes: Based on the current location information of the autonomous vehicle, the current motion state of the autonomous vehicle, and the predicted location information and predicted attribute information of the pedestrian, a driving route map is constructed; and the driving route is updated in real time based on the updates to the predicted location information and predicted attribute information of the pedestrian. The pedestrian detection model is obtained in the following way: Obtain a sample image set, which contains images of multiple pedestrian targets, and each image is labeled with the location and attribute information of the pedestrian target; A preset neural network model is trained using a sample image set to obtain a basic model; the basic model is pruned and compressed to obtain a compressed model; the parameters of the compressed model are fine-tuned to obtain a pedestrian detection model with fine-tuned parameters. The step of fine-tuning the parameters of the compressed model to obtain the pedestrian detection model after parameter fine-tuning includes: The trained model parameters are imported, and regularization constraints are applied to the normalization layer according to the preset learning rate. When the error value of the trained model output is within the preset range, the pedestrian detection model is obtained. The error value includes the error value between the predicted value and the true value of pedestrian location information, and the error value between the predicted value and the true value of pedestrian attribute information.

2. The autonomous driving method based on target pedestrian detection according to claim 1, characterized in that, The preset neural network model includes: a backbone network, a neck network, and a head network connected in sequence; The step of training a preset neural network model using a sample image set to obtain a trained pedestrian detection model includes: The sample images in the sample image set are sequentially input into the backbone network to obtain the image feature information output by the backbone network; The image feature information is input into the neck network to obtain the multi-scale fusion feature information output by the neck network; The multi-scale fused feature information is input into the head network to obtain the predicted pedestrian location information and pedestrian attribute information output by the head network. The parameters of the preset neural network model are adjusted based on the predicted pedestrian location information and pedestrian attribute information, as well as the true values ​​of pedestrian location information and pedestrian attribute information marked in the sample images in the sample image set, to obtain the basic model. The basic model is pruned and compressed to obtain the pedestrian detection model.

3. The autonomous driving method based on target pedestrian detection according to claim 2, characterized in that, Each convolutional module in the backbone network, neck network, and head network includes: a convolutional layer and a normalization layer; The step of pruning and compressing the base model to obtain the pedestrian detection model includes: The gamm parameter regularization loss constraint is used to train each normalization layer, and channels with gamm parameters below a preset threshold are pruned to obtain the pruned compressed model. The parameters of the compressed model are fine-tuned to obtain the pedestrian detection model with fine-tuned parameters.

4. The autonomous driving method based on target pedestrian detection according to claim 1, characterized in that, The step of acquiring an image dataset of the scene surrounding the autonomous vehicle includes: Collect scene images from various directions in front of the autonomous vehicle; The collected scene images are preprocessed to filter out images of people that contain pedestrian targets and whose content similarity does not exceed a preset threshold. The selected images of people form an image dataset.

5. The autonomous driving method based on target pedestrian detection according to claim 1, characterized in that, The pedestrian detection model includes: a feature extraction module, a feature fusion module, and a prediction output module; The steps of inputting each person image in the image dataset into a trained pedestrian detection model, and using the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in each person image in the image dataset, include: Each person's image is sequentially input into the feature extraction module to obtain multiple pedestrian feature information extracted by the feature extraction module; The pedestrian feature information is sequentially input into the feature fusion module to obtain the pedestrian feature fusion information output by the feature fusion module; The pedestrian feature fusion information is input into the prediction output module to obtain the predicted location information and predicted attribute information of the pedestrian output module.

6. An autonomous driving system based on target pedestrian detection, characterized in that, include: The image acquisition module is used to acquire image datasets of the scene surrounding the autonomous vehicle; The pedestrian detection module is used to input the images of each person in the image dataset into the trained pedestrian detection model, and use the pedestrian detection model to obtain the predicted location information and predicted attribute information of each pedestrian in the images of each person in the image dataset; wherein, the attribute information of the pedestrian is the type to which the pedestrian belongs. A driving control module is used to control the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information. The step of controlling the driving route of the autonomous vehicle based on the predicted location information and the predicted attribute information includes: Based on the current location information of the autonomous vehicle, the current motion state of the autonomous vehicle, and the predicted location information and predicted attribute information of the pedestrian, a driving route map is constructed; and the driving route is updated in real time based on the updates to the predicted location information and predicted attribute information of the pedestrian. The pedestrian detection model is obtained in the following way: Obtain a sample image set, which contains images of multiple pedestrian targets, and each image is labeled with the location and attribute information of the pedestrian target; A preset neural network model is trained using a sample image set to obtain a basic model; the basic model is pruned and compressed to obtain a compressed model; the parameters of the compressed model are fine-tuned to obtain a pedestrian detection model with fine-tuned parameters. The step of fine-tuning the parameters of the compressed model to obtain the pedestrian detection model after parameter fine-tuning includes: The trained model parameters are imported, and regularization constraints are applied to the normalization layer according to the preset learning rate. When the error value of the trained model output is within the preset range, the pedestrian detection model is obtained. The error value includes the error value between the predicted value and the true value of pedestrian location information, and the error value between the predicted value and the true value of pedestrian attribute information.

7. A vehicle, characterized in that, include: The system includes a memory, a processor, and an autonomous driving program based on target pedestrian detection stored in the memory and executable on the processor. When the processor executes the autonomous driving program based on target pedestrian detection, it implements the steps of the autonomous driving method based on target pedestrian detection as described in any one of claims 1-6.

Images