Autonomous driving vehicle control method, device, system and autonomous driving vehicle

Abstract

The present application discloses an autonomous driving vehicle control method, device, system and autonomous driving vehicle, belonging to the field of autonomous driving. Acquire perception data and motion scenes, determine the corresponding index based on the motion scenes, extract 3D feature data of the perception data through the 3D feature extraction network of the voxel prediction module; then input the 3D feature data and index into the voxel prediction network in the voxel prediction model, output the BEV grid and identify the position and category of the voxels in the BEV grid. In addition to the 3D feature data, the present application scheme also uses the index as the input of the voxel prediction network. Since the range and resolution of the output corresponding to the index standard motion scene, the range and resolution of the final output correspond to the index. In this way, when the same perception data is input, as long as different indexes are input at the same time, outputs of different ranges and resolutions can be obtained, so that one model can obtain outputs of different resolutions, meeting the diverse needs of users.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular, to an autonomous driving vehicle control method, device, system and autonomous driving vehicle. Background Art

[0002] The occupancy network model is a key method for autonomous vehicles to identify common obstacles and is used to provide accurate occupancy information of objects in three-dimensional space. Existing occupancy networks basically use semantic segmentation to classify voxels of predefined resolution in three-dimensional space, which can be simple occupied and non-occupied, or complex semantic information.

[0003] However, in related technologies, a model can only output one resolution, which cannot meet the diverse needs of users. Summary of the invention

[0004] In order to overcome the shortcomings of the related art, the present application provides an autonomous driving vehicle control method, device, system and autonomous driving vehicle to solve the problem in the related art that a model can only output one resolution and cannot meet the diverse needs of users.

[0005] The technical solution adopted by this application to solve its technical problem is:

[0006] In a first aspect, a method for controlling an autonomous driving vehicle is provided, comprising:

[0007] Acquire perception data and a motion scene, and determine a corresponding index according to the motion scene, wherein the index is used to characterize a range and a resolution of an output corresponding to the motion scene;

[0008] Inputting the perception data into a pre-trained voxel prediction model, and extracting corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model;

[0009] Input the 3D feature data and the index into a voxel prediction network in the voxel prediction model, output a BEV grid, and identify the position and category of the voxels in the BEV grid;

[0010] The autonomous vehicle is controlled based on the position and category of the voxels.

[0011] Further, the voxel prediction model includes a 3D feature extraction network and a voxel prediction network;

[0012] The 3D feature extraction network is used to extract 3D feature data from the perception data;

[0013] The voxel prediction network includes a conversion module and a prediction module. The conversion module is used to sample the 3D feature data according to the index and output a BEV grid of a target range and resolution corresponding to the index. The prediction module is used to identify the position and category of the voxels in the BEV grid.

[0014] Furthermore, the training step of the voxel prediction model includes:

[0015] Constructing a training sample set, wherein the training sample set includes a plurality of training samples, each training sample includes training perception data and M indexes;

[0016] The training samples in the training sample set are taken as input, and M BEV grids corresponding to the training perception data in the training samples and the positions and categories of the voxels in each BEV grid are taken as output, and the pre-constructed initial voxel prediction model is repeatedly trained; until the loss of the output BEV grid and the positions and categories of the voxels in each BEV grid and the positions and categories of the voxels in each BEV grid is less than a preset threshold, the training of the voxel prediction model is completed.

[0017] Further, the step of inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid comprises:

[0018] Mapping the 3D feature data to the BEV grid according to the index to obtain the position of each voxel; and obtaining a plurality of voxel sets based on the position of the voxel, each of the voxel sets being a set of all voxels in a three-dimensional figure with a base of a preset shape and a preset size and an infinite height; wherein the resolution of the BEV grid corresponds to the index;

[0019] Respectively obtaining at least two different initial category probability distributions for each of the voxel sets, each of the initial category probability distributions including N predicted probabilities, each predicted probability being a probability that the voxel set belongs to a preset category, wherein N is the number of preset categories;

[0020] Respectively obtaining a confidence weight of each voxel in the voxel set, wherein the confidence weight is used to characterize the confidence level of the voxel when the voxel adopts the initial category probability distribution;

[0021] Performing a weighted summation based on the initial category probability distribution and the confidence weight to obtain a target category probability distribution for each voxel;

[0022] The category with the largest probability value in the target category probability distribution is taken as the category of the voxel.

[0023] Further, the respectively obtaining the confidence weight of each voxel in the voxel set includes:

[0024] Obtain the candidate categories corresponding to the n predicted probabilities with the highest predicted probability values ​​in each initial category probability distribution, where n is a preset positive integer;

[0025] A target probability that the voxel belongs to the candidate category is obtained, and the target probability is used as a confidence weight for each voxel when the initial category probability distribution is adopted.

[0026] Further, obtaining the target probability that the voxel belongs to the candidate category includes:

[0027] respectively calculating the actual probability that the voxels belong to the candidate categories;

[0028] When all actual probabilities are 0, the target probability is 0; when there is any actual probability that is not 0, the target prediction probability of the candidate category with the highest actual probability is obtained, and the target probability is obtained using the following formula:

[0029] Target probability = target predicted probability / sum of the n predicted probabilities with the highest predicted probability values.

[0030] Furthermore, the obtaining of at least two different initial category probability distributions for each voxel set includes:

[0031] Using a probability prediction model to output the probability distribution of the voxel set in at least two different dimensions, the probability distribution of each dimension being used as an initial category probability distribution;

[0032] Alternatively, at least two different probability prediction models are used to output a probability distribution respectively, and the probability distribution output by each probability prediction model is used as an initial category probability distribution.

[0033] In a second aspect, an automatic driving vehicle control device is provided, comprising:

[0034] A data acquisition module, used to acquire perception data and motion scenes; determine a corresponding index according to the motion scene, wherein the index is used to characterize the range and resolution of the output corresponding to the motion scene;

[0035] A 3D feature extraction module, used for inputting the perception data into a pre-trained voxel prediction model, and extracting corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model;

[0036] A voxel prediction module, used for inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid;

[0037] A vehicle control module is used to control the autonomous driving vehicle based on the position and category of the voxel.

[0038] In a third aspect, an automatic driving vehicle control system is provided, comprising:

[0039] at least one processor and at least one memory;

[0040] The memory stores executable instructions of the processor;

[0041] The processor is configured to execute the autonomous driving vehicle control method provided by the technical solution of the first aspect.

[0042] In a fourth aspect, an autonomous driving vehicle is provided, which applies the autonomous driving vehicle control method provided by the technical solution of the first aspect.

[0043] Beneficial effects:

[0044] The present application provides an autonomous driving vehicle control method, device, system and autonomous driving vehicle, which obtains perception data and motion scenes, then determines the corresponding index based on the motion scenes, and extracts 3D feature data of the perception data through the 3D feature extraction network of the voxel prediction module; then the 3D feature data and index are input into the voxel prediction network in the voxel prediction model, and the BEV grid is output and the position and category of the voxels in the BEV grid are identified. In addition to the 3D feature data, the present application also uses the index as the input of the voxel prediction network. Since the range and resolution of the output corresponding to the index standard motion scene, the range and resolution of the final output correspond to the index. In this way, when the same perception data is input, as long as different indexes are input at the same time, outputs of different ranges and resolutions can be obtained, so that one model can obtain outputs of different resolutions, meeting the diverse needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of an autonomous driving vehicle control method provided in an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of a voxel prediction model structure provided in an embodiment of the present application;

[0048] Figure 3 This is a schematic diagram of a sensor installation provided in an embodiment of the present application;

[0049] Figure 4 is a schematic diagram of a specific voxel prediction model structure provided in an embodiment of the present application;

[0050] Figure 5 It is a 3D feature data mapping BEV grid schematic diagram provided in an embodiment of the present application;

[0051] Figure 6 This is a schematic diagram of lightweight semantic occupancy prediction provided by an embodiment of the present application;

[0052] Figure 7 It is a schematic diagram of the structure of an autonomous driving vehicle control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application is described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0054] For users, the higher the resolution, the better. However, when predicting voxels, if the resolution is higher, the number of voxels will increase, the amount of calculation will increase, and the real-time prediction will not be met. In actual use, for some sports scenes, such as high-speed driving, a high resolution is not actually required. If only a lower resolution is used, in other sports scenes, such as parking, it will cause inaccurate obstacle recognition. Therefore, for autonomous driving, it is very necessary to output in a form with multiple resolutions.

[0055] The solution to achieve this effect in related technologies is to train multiple models, each with an output of one resolution, and determine which model to call based on different motion scenes. However, the training and storage costs of multiple models are too high, and when multiple models are used, they need to be called. If the network is poor or disconnected, the call may not be realized, and the previously used model will still be used, resulting in the output resolution not meeting the requirements of the current motion scene.

[0056] To solve the above problems, refer to Figure 1 , the embodiment of the present application provides an automatic driving vehicle control method, comprising:

[0057] S11: Acquire perception data and a motion scene, and determine a corresponding index according to the motion scene, where the index is used to characterize a range and a resolution of an output corresponding to the motion scene.

[0058] Optionally, the perception data is data acquired by at least one of the following sensors:

[0059] Cameras, LiDAR, infrared and radar.

[0060] The motion scene is divided according to the range and resolution interval. By setting the corresponding relationship between the motion scene, the range and the resolution, the corresponding index can be determined by the motion scene.

[0061] Exemplarily, if the predefined range and resolution are divided into three types, all motion scenes are divided into three categories, and each category corresponds to an index.

[0062] Among them, the motion scene is determined based on vehicle information, and the vehicle information includes but is not limited to: the vehicle's current task (such as starting, driving, parking, reversing, overtaking) and / or vehicle status (such as vehicle speed).

[0063] S12: Input the perception data into a pre-trained voxel prediction model, and extract corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model.

[0064] S13: Input the 3D feature data and the index into the voxel prediction network in the voxel prediction model, output a BEV grid, and identify the position and category of the voxels in the BEV grid.

[0065] That is, the key to achieving a model with multiple ranges and resolutions in this application lies in the voxel prediction model, such as Figure 2 As shown, the structure of the voxel prediction model includes a 3D feature extraction network and a voxel prediction network;

[0066] The 3D feature extraction network is used to extract 3D feature data from the perception data; since the perception data is actually 2D data, the 3D feature extraction network is required to extract the 3D feature data therein for subsequent processing. Exemplarily, the 3D feature extraction network is a backbone network, or a backbone network + a pyramid network.

[0067] The voxel prediction network includes a conversion module and a prediction module. The conversion module is used to sample the 3D feature data according to the index and output a BEV grid of a target range and resolution corresponding to the index. The prediction module is used to identify the position and category of the voxels in the BEV grid. The input of the conversion module includes the index and the 3D feature data, and the output range and resolution correspond to the index. The principle is as follows: the conversion module determines the sampling range according to the range corresponding to the index, samples the 3D feature data based on the sampling range, and then maps the sampled features to the BEV grid based on the resolution corresponding to the index, so that the range and resolution of the output BEV grid correspond to the index.

[0068] When training a conventional model, since the input is only sensory data, if you want its output to have multiple ranges and resolutions, the model will eventually fail to converge. This application sets the input of the model to two, including not only sensory data but also indexes. The outputs obtained for the same sensory data based on different indexes have different ranges and resolutions. Among them, the training steps of the voxel prediction model include:

[0069] A training sample set is constructed, wherein the training sample set includes multiple training samples, each training sample includes training perception data and M indexes; that is, if M indexes are set, a training perception data is repeated M times, and each copy and an index are input into the initial model, where M is the number of index categories.

[0070] The training samples in the training sample set are used as input, and M BEV grids corresponding to the training perception data in the training samples and the positions and categories of the voxels in each BEV grid are used as output, and the pre-constructed initial voxel prediction model is repeatedly trained; until the loss of the output BEV grid and the positions and categories of the voxels in each BEV grid and the positions and categories of the voxels in each BEV grid is less than a preset threshold, and the training of the voxel prediction model is completed. Exemplarily, if index A represents the output range and resolution as a, and index B represents the output range and resolution as b, when a piece of training perception data is input into the model, the output range and resolution is a, if the input index is A, then the output at this time is judged to be a true value, and if the input index is B, then the output at this time is judged not to be a true value. In this way, the initial voxel prediction model is trained until the loss function meets the requirements.

[0071] In one embodiment, inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid comprises:

[0072] Mapping the 3D feature data to the BEV grid according to the index to obtain the position of each voxel; and obtaining a plurality of voxel sets based on the position of the voxel, each of the voxel sets being a set of all voxels in a three-dimensional figure with a base of a preset shape and a preset size and an infinite height; wherein the resolution of the BEV grid corresponds to the index;

[0073] The probability distribution of each voxel is calculated separately, and the category corresponding to the probability with the largest value in the output probability distribution is taken as the category of the voxel.

[0074] Taking a voxel set including d voxels and N predefined categories as an example, in this embodiment, the amount of calculation for a voxel set is d*N. If the output is high resolution, this amount of calculation is very large, and it is impossible to take into account both low cost and real-time performance of voxel prediction.

[0075] However, the inventors of the present application have found in practice that, although N types are defined, from the BEV perspective, the number of categories in a voxel set is limited. Therefore, there are many redundant calculations in the calculation method of the above-mentioned sampling embodiment.

[0076] Based on this, in another embodiment, inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid includes:

[0077] Mapping the 3D feature data to the BEV grid according to the index to obtain the position of each voxel; and obtaining a plurality of voxel sets based on the position of the voxel, each of the voxel sets being a set of all voxels in a three-dimensional figure with a base of a preset shape and a preset size and an infinite height; wherein the resolution of the BEV grid corresponds to the index;

[0078] At least two different initial category probability distributions are obtained for each voxel set respectively, each of the initial category probability distributions includes N predicted probabilities, each predicted probability is the probability that the voxel set belongs to a preset category, wherein N is the number of preset categories; that is, this embodiment first calculates the initial probability category distribution of each voxel set, rather than calculating the initial probability category distribution of each voxel, and in this step, the amount of calculation is 2*N.

[0079] The confidence weight of each voxel in the voxel set is obtained respectively, and the confidence weight is used to characterize the confidence of the voxel when it adopts the initial category probability distribution; in this step, the confidence of each voxel using one of the initial probability category distributions is further calculated. As a preferred implementation method of the embodiment of the present application, the corresponding model is directly trained to output the confidence weight, and the calculation amount is 2*d at this time.

[0080] Performing a weighted summation based on the initial category probability distribution and the confidence weight to obtain a target category probability distribution for each voxel;

[0081] The category with the largest probability value in the target category probability distribution is taken as the category of the voxel.

[0082] In the method provided in this embodiment, the amount of calculation is changed from d*N to 2N+2d. When the number of voxels d and the number of categories N are large, the amount of calculation can be greatly reduced.

[0083] As an optional implementation of the embodiment of the present application, respectively obtaining the confidence weight of each voxel in the voxel set includes:

[0084] Obtain the candidate categories corresponding to the n predicted probabilities with the highest predicted probability values ​​in each initial category probability distribution, where n is a preset positive integer; obtain the target probability that the voxel belongs to the candidate category, and use the target probability as the confidence weight for each voxel when adopting the initial category probability distribution.

[0085] Wherein, obtaining the target probability that the voxel belongs to the candidate category includes:

[0086] The actual probabilities of the voxels belonging to the candidate categories are calculated respectively; when all actual probabilities are 0, the target probability is 0; when there is any actual probability that is not 0, the target predicted probability of the candidate category with the highest actual probability is obtained.

[0087] The target probability is obtained using the following formula:

[0088] Target probability = target predicted probability / sum of the n predicted probabilities with the highest predicted probability values.

[0089] For example, when n=2, the candidate types in one of the initial probability distributions P are P1 and P2, and their corresponding predicted probabilities are p1 and p2 respectively. If one of the voxels is P1 and P2, the actual probabilities are a1 and a2 respectively.

[0090] If a1 and a2 are both 0, it means that the voxel is not accurate when using the initial probability distribution P. At this time, the target probability is 0. If one of a1 and a2 is not 0, and a1>a2, the target prediction probability is p1. At this time, the target probability = p1 / (p1+p2).

[0091] Optionally, the target predicted probability is directly used as the target probability, or the actual probability of the candidate class with the highest actual probability is directly used as the target probability.

[0092] In one embodiment, at least two different initial category probability distributions are obtained for each of the voxel sets, including: using a probability prediction model to output the probability distribution of the voxel set in at least two different dimensions, and the probability distribution of each dimension is used as an initial category probability distribution.

[0093] In another embodiment, at least two different initial category probability distributions are obtained for each of the voxel sets, including: using at least two different probability prediction models to output a probability distribution respectively, and the probability distribution output by each probability prediction model is used as an initial category probability distribution.

[0094] S14: Controlling the autonomous driving vehicle based on the position and category of the voxel.

[0095] After obtaining the voxel position and category, the position of the obstacle is identified and determined. Then the decision module generates control instructions for the autonomous driving vehicle. This application does not make any improvements to this and will not be described in detail here.

[0096] In order to more clearly illustrate the present application scheme, a specific implementation method is provided below:

[0097] The hardware structure is shown as follows Figure 3 As shown in the figure, three blind spot-filling low-beam laser radars are installed at the left front, right front and rear middle of the front of the vehicle to cover the 360° near area. This installation method ensures that there is no blind spot coverage around the vehicle.

[0098] With the above sensor configuration, we need to design the structure of the voxel prediction model. The structure of the voxel prediction model is as follows: Figure 4 First, the surround image acquired by the sensor passes through the backbone module and the fpn module to obtain 3D feature data, and then the 3D feature data and index are input to the zoom view transformer module. The output of the zoom view transformer module passes through the bev backbone module and the bev fpn module in turn, and finally the voxels are predicted through the OccPillars module.

[0099] Since voxels with different ranges and resolutions need to be output in different scenes, the network needs to make corresponding transformations when transforming from surround view image features to BEV features. This part corresponds to the zoom view transformer module in the figure. Since the BEV part of the network is fixed, the ratio of range and voxel resolution to zoom is the same. Through this module, the input of BEV can be adjusted according to the index when the network is inferred to the BEV part. This network design that flexibly adjusts the output according to the index can meet the perception needs of autonomous vehicles in different scenes. At the same time, the algorithm deployed on the vehicle side has very high requirements for real-time performance, and high-resolution semantic occupancy is a computationally intensive task. The final predicted parameter quantity is the number of voxels * the number of semantic categories. For existing methods, whether it is based on query-based transformer architecture or 3D or 2D convolution architecture, it is very time-consuming and unbearable for common vehicle-side computing platforms. In order to achieve a semantic occupancy prediction that can be inferred in real time, the OccPillars module remodels the occupancy prediction task into a task of predicting a probability distribution combination for each pillar and a weight combination for each voxel. This method can greatly reduce the amount of calculation and network latency.

[0100] In this embodiment, the zoom view transformer module can realize three zoom outputs, namely, a depth estimation interval of 0.8m in the range of 1-120m, a depth interval of 0.4m in the range of 1-60m, and a depth interval of 0.2m in the range of 1-30m. Figure 5 As shown, the sampled view frustum point cloud features are mapped into the BEV grid. Figure 5 In the figure, points of different colors represent sampling in different ranges. The input of the BEV module can be changed according to the index, thereby changing the resolution of the final output.

[0101] In order to be able to perceive the complete 3D space in real time on the vehicle side, a lightweight occupancy prediction head is required. For each semantically occupied voxel, the category probability distribution of the voxel needs to be obtained in order to train and obtain the final result. For common semantic segmentation-based methods, assuming that there are d voxels, the number of parameters that need to be predicted for the voxels in a pillar (i.e., a set of voxels with a bottom of a cylinder and an infinite height) for N categories is d*N. In the autonomous driving scenario, although there are many semantic categories, the number of semantic categories in a single pillar is limited from the perspective of BEV, which means that there are many redundant category predictions in a single pillar. In theory, a small number of semantic categories can be proposed first for the voxels in the pillar to make a choice. Based on this idea, two weights w1 and w2 can be predicted for each voxel, and two category probability distributions C1 and C2 can be predicted for each pillar. In this way, the number of predicted parameters is 2(d+N). Finally, the category probability distribution of each voxel is w1*C1+w2*C2. As Figure 6 As shown in the figure, the left side shows the number of parameters required for dense semantic prediction, and the right side shows the number of parameters required for OccPillars module prediction. From the visualization results, the advantage of the occupancy prediction head will increase significantly with the increase of the number of semantic categories and resolution.

[0102] It should be emphasized that since the network needs to learn the occupancy prediction ability at three scales at the same time, it is necessary to make sure that the network is evenly exposed to all samples at three scales during network training to prevent the network from jittering back and forth and being difficult to converge during gradient descent due to the offset problem caused by data distribution. The specific method is to repeat the sampling of each batch of samples three times during the data loading iteration phase, and let the network continuously learn the true value of a sample at three different scales during the model training phase. The true value must also be adjusted according to the range and resolution required for model training.

[0103] The specific autonomous driving vehicle control method provided in the embodiment of the present application can, first of all, reduce the computational complexity of occupancy prediction from d*N to 2(d+N), greatly reducing the pressure of deploying high-resolution semantic occupancy prediction on the vehicle side. Secondly, it is possible to dynamically adjust the perception range and voxel resolution according to different scenarios and business needs, which increases the flexibility of the network, eliminates the need to switch models, and improves the robustness of the system. Finally, the proposed training method can align the results of network training with the result indicators of separate training, effectively utilizing the capacity of the BEV paradigm.

[0104] Based on the same inventive concept, Figure 7 As shown, the present application provides an automatic driving vehicle control device 70, comprising:

[0105] The data acquisition module 71 is used to acquire the perception data and the motion scene; determine the corresponding index according to the motion scene, and the index is used to characterize the range and resolution of the output corresponding to the motion scene;

[0106] A 3D feature extraction module 72, configured to input the perception data into a pre-trained voxel prediction model, and extract corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model;

[0107] The voxel prediction module 73 is used to input the 3D feature data and the index into the voxel prediction network in the voxel prediction model, output a BEV grid, and identify the position and category of the voxels in the BEV grid.

[0108] Among them, the voxel prediction model includes a 3D feature extraction network and a voxel prediction network;

[0109] The 3D feature extraction network is used to extract 3D feature data from the perception data; the voxel prediction network includes a conversion module and a prediction module, the conversion module is used to sample the 3D feature data according to the index and output a BEV grid of a target range and resolution corresponding to the index, and the prediction module is used to identify the position and category of the voxels in the BEV grid.

[0110] Furthermore, the training step of the voxel prediction model includes:

[0111] A training sample set is constructed, wherein the training sample set includes a plurality of training samples, each training sample includes training perception data and M indexes; the training samples in the training sample set are taken as input, and M BEV grids corresponding to the training perception data in the training samples and the positions and categories of the voxels in each BEV grid are taken as output, and the pre-constructed initial voxel prediction model is repeatedly trained; until the loss of the output BEV grid and the positions and categories of the voxels in each BEV grid and the positions and categories of the voxels in each BEV grid is less than a preset threshold, the training of the voxel prediction model is completed.

[0112] As a preferred implementation method of the embodiment of the present application, the step of inputting the 3D feature data and the index into the voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid includes:

[0113] According to the index, the 3D feature data is mapped to the BEV grid to obtain the position of each voxel; and based on the position of the voxel, multiple voxel sets are obtained, each of which is a set of all voxels in a three-dimensional figure with a base of a preset shape and a preset size and an infinite height; wherein the resolution of the BEV grid corresponds to the index; at least two different initial category probability distributions are obtained for each voxel set, each of which includes N predicted probabilities, each predicted probability is the probability that the voxel set belongs to a preset category, wherein N is the number of preset categories; confidence weights of each voxel in the voxel set are obtained, and the confidence weights are used to characterize the confidence of the voxel when the initial category probability distribution is adopted; weighted summation is performed according to the initial category probability distribution and the confidence weights to obtain the target category probability distribution of each voxel; and the category with the largest probability value in the target category probability distribution is taken as the category of the voxel.

[0114] The step of respectively obtaining the confidence weight of each voxel in the voxel set comprises:

[0115] Obtain the candidate categories corresponding to the n predicted probabilities with the highest predicted probability values ​​in each initial category probability distribution, where n is a preset positive integer; obtain the target probability that the voxel belongs to the candidate category, and use the target probability as the confidence weight for each voxel when adopting the initial category probability distribution.

[0116] Preferably, obtaining the target probability that the voxel belongs to the candidate category comprises:

[0117] The actual probability that the voxel belongs to the candidate category is calculated respectively; when all actual probabilities are 0, the target probability is 0; when there is any actual probability that is not 0, the target predicted probability of the candidate category with the highest actual probability is obtained, and the target probability is obtained using the following formula: target probability = target predicted probability / sum of the n predicted probabilities with the highest predicted probability values.

[0118] In one embodiment, obtaining at least two different initial category probability distributions for each voxel set includes:

[0119] A probability prediction model is used to output the probability distribution of the voxel set in at least two different dimensions, and the probability distribution of each dimension is used as an initial category probability distribution.

[0120] In another embodiment, obtaining at least two different initial category probability distributions for each voxel set respectively includes:

[0121] At least two different probability prediction models are used to output a probability distribution respectively, and the probability distribution output by each probability prediction model is used as an initial category probability distribution.

[0122] The vehicle control module 74 is used to control the autonomous driving vehicle based on the position and category of the voxel.

[0123] The autonomous driving vehicle control device provided in the embodiment of the present application first obtains the perception data and the motion scene, then determines the corresponding index based on the motion scene, and extracts the 3D feature data of the perception data through the 3D feature extraction network of the voxel prediction module; then the 3D feature data and the index are input into the voxel prediction network in the voxel prediction model, the BEV grid is output and the position and category of the voxels in the BEV grid are identified. In addition to the 3D feature data, the present application scheme also uses the index as the input of the voxel prediction network. Since the range and resolution of the output corresponding to the index standard motion scene, the range and resolution of the output finally obtained correspond to the index. In this way, when the same perception data is input, as long as different indexes are input at the same time, outputs of different ranges and resolutions can be obtained, so that one model can obtain outputs of different resolutions, meeting the diverse needs of users.

[0124] Based on the same inventive concept, the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the steps of the autonomous driving vehicle control method provided in any of the above embodiments are implemented.

[0125] Based on the same inventive concept, the present application provides an autonomous driving vehicle control system, comprising:

[0126] at least one processor and at least one memory;

[0127] The memory stores executable instructions of the processor;

[0128] The processor is configured to execute the autonomous driving vehicle control method provided by any of the above embodiments.

[0129] Based on the same inventive concept, the present application provides an autonomous driving vehicle, applying the autonomous driving vehicle control method provided by any of the above embodiments.

[0130] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0131] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0133] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0134] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0136] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0137] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0138] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for controlling an automatic driving vehicle, characterized in that: include: Acquire the perception data and the motion scene, and determine a corresponding index according to the motion scene, wherein the index is used to characterize the range and resolution of the output corresponding to the motion scene, wherein the motion scene is determined according to the vehicle speed; Inputting the perception data into a pre-trained voxel prediction model, and extracting corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model; Inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid; Inputting the 3D feature data and the index into a voxel prediction network in the voxel prediction model, outputting a BEV grid, and identifying the position and category of the voxels in the BEV grid comprises: According to the index, the 3D feature data is mapped to the BEV grid to obtain the position of each voxel; and based on the position of the voxel, multiple voxel sets are obtained, each of which is a set of all voxels in a three-dimensional figure with a preset shape and preset size as the base and infinite height; wherein the resolution of the BEV grid corresponds to the index; at least two different initial category probability distributions are obtained for each voxel set, each of which includes N predicted probabilities, each predicted probability is the probability that the voxel set belongs to a preset category, wherein N is the number of preset categories; confidence weights of each voxel in the voxel set are obtained, respectively, and the confidence weights are used to characterize the confidence of the voxel when the initial category probability distribution is adopted; weighted summation is performed according to the initial category probability distribution and the confidence weights to obtain the target category probability distribution of each voxel; the category with the largest probability value in the target category probability distribution is taken as the category of the voxel; The autonomous vehicle is controlled based on the position and category of the voxels.

2. The method according to claim 1, characterized in that The voxel prediction model includes a 3D feature extraction network and a voxel prediction network; The 3D feature extraction network is used to extract 3D feature data from the perception data; The voxel prediction network includes a conversion module and a prediction module. The conversion module is used to sample the 3D feature data according to the index and output a BEV grid of a target range and resolution corresponding to the index. The prediction module is used to identify the position and category of the voxels in the BEV grid.

3. The method according to claim 2, characterized in that The training steps of the voxel prediction model include: Constructing a training sample set, wherein the training sample set includes a plurality of training samples, each training sample includes training perception data and M indexes; The training samples in the training sample set are taken as input, and M BEV grids corresponding to the training perception data in the training samples and the positions and categories of the voxels in each BEV grid are taken as output, and the pre-constructed initial voxel prediction model is repeatedly trained; until the loss of the output BEV grid and the positions and categories of the voxels in each BEV grid and the positions and categories of the voxels in each BEV grid is less than a preset threshold, the training of the voxel prediction model is completed.

4. The method according to claim 1, characterized in that: The step of respectively obtaining the confidence weight of each voxel in the voxel set comprises: Obtain the candidate categories corresponding to the n predicted probabilities with the highest predicted probability values ​​in each initial category probability distribution, where n is a preset positive integer; A target probability that the voxel belongs to the candidate category is obtained, and the target probability is used as a confidence weight for each voxel when the initial category probability distribution is adopted.

5. The method according to claim 4, characterized in that The acquiring the target probability that the voxel belongs to the candidate category comprises: respectively calculating the actual probability that the voxels belong to the candidate categories; When all actual probabilities are 0, the target probability is 0; when there is any actual probability that is not 0, the target prediction probability of the candidate category with the highest actual probability is obtained, and the target probability is obtained using the following formula: Target probability = target predicted probability / sum of the n predicted probabilities with the highest predicted probability values.

6. The method according to claim 1, characterized in that The step of respectively obtaining at least two different initial category probability distributions for each of the voxel sets comprises: Using a probability prediction model to output the probability distribution of the voxel set in at least two different dimensions, the probability distribution of each dimension being used as an initial category probability distribution; Alternatively, at least two different probability prediction models are used to output a probability distribution respectively, and the probability distribution output by each probability prediction model is used as an initial category probability distribution.

7. An automatic driving vehicle control device, characterized in that: include: A data acquisition module, used to acquire perception data and motion scenes; determine a corresponding index according to the motion scene, wherein the index is used to characterize the range and resolution of the output corresponding to the motion scene, wherein the motion scene is determined according to the vehicle speed; A 3D feature extraction module, used for inputting the perception data into a pre-trained voxel prediction model, and extracting corresponding 3D feature data through a 3D feature extraction network of the voxel prediction model; A voxel prediction module is used to input the 3D feature data and the index into a voxel prediction network in the voxel prediction model, output a BEV grid, and identify the position and category of the voxels in the BEV grid; the inputting the 3D feature data and the index into the voxel prediction network in the voxel prediction model, outputting the BEV grid, and identifying the position and category of the voxels in the BEV grid includes: According to the index, the 3D feature data is mapped to the BEV grid to obtain the position of each voxel; and based on the position of the voxel, multiple voxel sets are obtained, each of which is a set of all voxels in a three-dimensional figure with a preset shape and preset size as the base and infinite height; wherein the resolution of the BEV grid corresponds to the index; at least two different initial category probability distributions are obtained for each voxel set, each of which includes N predicted probabilities, each predicted probability is the probability that the voxel set belongs to a preset category, wherein N is the number of preset categories; confidence weights of each voxel in the voxel set are obtained, respectively, and the confidence weights are used to characterize the confidence of the voxel when the initial category probability distribution is adopted; weighted summation is performed according to the initial category probability distribution and the confidence weights to obtain the target category probability distribution of each voxel; the category with the largest probability value in the target category probability distribution is taken as the category of the voxel; A vehicle control module is used to control the autonomous driving vehicle based on the position and category of the voxel.

8. An automatic driving vehicle control system, characterized in that: include: at least one processor and at least one memory; The memory stores executable instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 6.

9. An autonomous driving vehicle, characterized in that: The method according to any one of claims 1 to 6 is used.

Images