Model training method, path planning method, device, equipment and medium

Abstract

The present application discloses a model training method, a path planning method, an apparatus, a device and a medium, and relates to the field of computer technology. The method comprises: obtaining sample data, wherein the sample data comprises sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data; using a pre-trained trajectory planning model of a joint path planning model to be trained, performing trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information; using a pre-trained control planning model of a joint path planning model to be trained, performing control planning processing on the fusion feature decoding information to obtain control planning information; based on the sample data, the trajectory planning information and the control planning information, performing iterative training processing on the joint path planning model to be trained to obtain a trained joint path planning model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to technical fields such as intelligent transportation and autonomous driving, and in particular to a model training method, a path planning method, device, equipment and medium. Background Art

[0002] Generally, the path planning and control modules are the core systems of autonomous vehicles. The main task of the path planning module is to generate a safe, efficient and comfortable driving route for the vehicle based on perception prediction and high-precision map information. The main task of the control module is to output vehicle actuator-related instructions, such as brake / accelerator pedal opening and steering wheel angle, based on the trajectory of the path planning module and combined with the vehicle's kinematic and dynamic models.

[0003] At present, the planning module and control module in related technologies are independent of each other. The path planning module is usually divided into three sub-modules: behavior decision, path planning and motion planning, and combines some sub-modules such as initialization, scene management, task management, horizontal and vertical separation, trajectory post-processing, etc. to plan the path output. The control module is usually divided into lateral control and longitudinal control, and the vehicle driving control information is solved through optimization methods. Summary of the invention

[0004] The present application provides a model training method, a path planning method, an apparatus, a device and a medium, which solve the problems such as poor performance of the path planning model. The technical solution is as follows:

[0005] In a first aspect, a model training method is provided, wherein the joint path planning model to be trained includes a pre-trained trajectory planning model and a pre-trained control planning model, and the method includes:

[0006] Acquire sample data, the sample data including sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data;

[0007] Using the pre-trained trajectory planning model, performing trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information;

[0008] Using the pre-trained control planning model, performing control planning processing on the fused feature decoding information to obtain control planning information;

[0009] Based on the sample data, the trajectory planning information, and the control planning information, the joint path planning model to be trained is iteratively trained to obtain a trained joint path planning model.

[0010] In a possible implementation, the pre-trained trajectory planning model to be trained includes a first encoder, a second encoder and a decoder. Before performing trajectory planning processing on the sample data using the pre-trained trajectory planning model, the method includes:

[0011] Using the first encoder, encoding the sample bird's-eye view feature image to obtain bird's-eye view feature image encoding information;

[0012] Using the second encoder, encoding the sample vehicle driving data to obtain vehicle driving feature encoding information;

[0013] Performing feature splicing processing on the bird's-eye view feature image coding information and the vehicle driving feature coding information to obtain fused feature coding information;

[0014] Using the decoder, decoding the fused feature encoding information to obtain a trajectory planning result;

[0015] Based on the sample data and the trajectory planning result, the trajectory planning model to be trained is iteratively trained to obtain the pre-trained trajectory planning model.

[0016] In a possible implementation, the decoder includes a decoding network, a classification network, and a regression network. The decoder is used to decode the fusion feature encoding information to obtain a trajectory planning result, including:

[0017] Using the decoding network, the fused feature encoding information is decoded to obtain fused feature decoding information;

[0018] Inputting the fused feature decoding information into the classification network to obtain the driving direction category information of the trajectory;

[0019] Inputting the fused feature decoding information into the regression network to obtain the position information of the trajectory;

[0020] The trajectory planning result is obtained based on the driving direction category information and the position information.

[0021] In a possible implementation, before using the pre-trained control planning model to perform control planning processing on the fused feature decoding information, the method includes:

[0022] Inputting the sample data into a pre-trained control planning model to be trained to obtain an output control planning result;

[0023] Based on the sample data and the control planning result, the control planning model to be trained is iteratively trained to obtain the pre-trained control planning model that has been trained.

[0024] In a second aspect, a path planning method is provided, the method comprising:

[0025] Acquire vehicle driving data to be processed and a bird's-eye view feature map corresponding to the vehicle driving data;

[0026] Inputting the vehicle driving data and the bird's-eye view feature map into a joint path planning model to obtain trajectory planning information and control planning information output by the joint path planning model;

[0027] Based on the trajectory planning information and the control planning information, obtaining a joint path planning result;

[0028] The joint path planning model is determined by adopting the above-mentioned aspects and any possible implementation method, and the joint path planning model includes a trajectory planning model and a control planning model.

[0029] In a third aspect, a model training device is provided, wherein the joint path planning model to be trained includes a pre-trained trajectory planning model and a pre-trained control planning model, and the device includes:

[0030] A first acquisition unit is used to acquire sample data, wherein the sample data includes sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data;

[0031] A trajectory planning unit, used to perform trajectory planning processing on the sample data using the pre-trained trajectory planning model to obtain trajectory planning information and fusion feature decoding information;

[0032] A control planning unit, configured to perform control planning processing on the fused feature decoding information using the pre-trained control planning model to obtain control planning information;

[0033] A model training unit is used to iteratively train the joint path planning model to be trained based on the sample data, the trajectory planning information, and the control planning information to obtain a trained joint path planning model.

[0034] In a fourth aspect, a path planning device is provided, the device comprising:

[0035] An acquisition unit, used to acquire vehicle driving data to be processed and a bird's-eye view feature map corresponding to the vehicle driving data;

[0036] A planning unit, used for inputting the vehicle driving data and the bird's-eye view feature map into a joint path planning model to obtain trajectory planning information and control planning information output by the joint path planning model;

[0037] An obtaining unit, configured to obtain a joint path planning result based on the trajectory planning information and the control planning information;

[0038] Wherein, the joint path planning model is determined by using the device as described in the third aspect, and the joint path planning model includes a trajectory planning model and a control planning model.

[0039] In a fifth aspect, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the method of the above-mentioned aspects and any possible implementation manner.

[0040] In a sixth aspect, an electronic device is provided, including:

[0041] at least one processor; and

[0042] a memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0044] In a seventh aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and any possible implementation method.

[0045] In an eighth aspect, an autonomous driving vehicle is provided, comprising the electronic device as described above.

[0046] The beneficial effects of the technical solution provided by this application include at least:

[0047] It can be seen from the above technical scheme that, on the one hand, the embodiment of the present application can obtain sample data including sample vehicle driving data and a sample bird's-eye view feature map of the perception information corresponding to the sample vehicle driving data, and use the pre-trained trajectory planning model of the joint path planning model to be trained to perform trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information, and use the pre-trained control planning model to perform control planning processing on the fusion feature decoding information to obtain control planning information, and based on the sample data, the trajectory planning information, and the control planning information, perform iterative training processing on the joint path planning model to be trained to obtain a trained joint path planning model. Since the joint path planning model mainly composed of a pre-trained trajectory planning model and a pre-trained control planning model is trained based on the sample vehicle driving data and the sample bird's-eye view feature map of the perception information corresponding to the sample vehicle driving data, a more accurate and effective end-to-end joint path planning model can be obtained, which ensures the planning performance and effectiveness of the model, thereby ensuring the reliability of the path planning and control planning of the autonomous driving vehicle.

[0048] It can be seen from the above technical solution that, on the other hand, the embodiment of the present application can obtain the vehicle driving data to be processed and the bird's-eye feature map corresponding to the vehicle driving data, and input the vehicle driving data and the bird's-eye feature map into the joint path planning model to obtain the joint path planning result. The joint path planning model includes a trajectory planning model and a control planning model. Since the joint path planning model has better end-to-end planning performance, the end-to-end joint path planning model is used to perform path and control planning on the acquired vehicle driving data and the bird's-eye feature map corresponding to the vehicle driving data. While ensuring the accuracy of the joint path planning result, the corresponding joint path planning result can be directly output based on the end-to-end planning model, which reduces the computing power overhead of the planning, thereby improving the reliability and efficiency of the path and control planning of the autonomous driving vehicle.

[0049] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1It is a flowchart of a model training method provided by an embodiment of the present application;

[0052] Figure 2 is a flowchart of a path planning method provided by another embodiment of the present application;

[0053] Figure 3 is a flowchart of a path planning method provided by another embodiment of the present application;

[0054] Figure 4 It is a schematic diagram of the architecture of an application scenario of a path planning method provided by another embodiment of the present application;

[0055] Figure 5 is a structural block diagram of a model training device provided in another embodiment of the present application;

[0056] Figure 6 is a structural block diagram of a path planning device provided by another embodiment of the present application;

[0057] Figure 7 It is a block diagram of an electronic device used to implement the path planning method and model training method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0058] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0059] Obviously, the described embodiments are only part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0060] It should be noted that the terminal devices involved in the embodiments of the present application may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0061] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0062] At present, in the planning scheme of related technologies, the planning module and the control module are independent of each other, and this scheme has multiple problems: First, the entire path planning and control system is divided into several subsystems and multi-functional modules, which are difficult to develop and optimize. Secondly, due to the limited computing power on the vehicle side, the computing power allocated to each module is insufficient, resulting in a significant impact on performance. Thirdly, the path planning and control system has poor generalization. When encountering long-tail scenarios, the traditional modular-based system needs to add corresponding rules and codes, which makes the system's scalability worse and the maintenance cost increases significantly.

[0063] Therefore, there is an urgent need for a model training method that can improve the performance of joint path and control planning, thereby ensuring the reliability of autonomous driving vehicle planning.

[0064] Please refer to Figure 1 , which shows a flow chart of a method for model training provided by an embodiment of the present application. The joint path planning model to be trained may include a pre-trained trajectory planning model and a pre-trained control planning model. The method for model training may specifically include:

[0065] Step 101: Acquire sample data, where the sample data includes sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data.

[0066] Step 102: Using the pre-trained trajectory planning model, perform trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information.

[0067] Step 103: Use the pre-trained control planning model to perform control planning processing on the fused feature decoding information to obtain control planning information.

[0068] Step 104: Based on the sample data, the trajectory planning information, and the control planning information, the joint path planning model to be trained is iteratively trained to obtain a trained joint path planning model.

[0069] It should be noted that the sample vehicle driving data may include but is not limited to the position, speed, acceleration, direction, etc. of the autonomous driving vehicle.

[0070] It should be noted that the sample bird's-eye feature map can be a bird's-eye feature map of the perception information corresponding to the sample vehicle driving data. The sample bird's-eye feature map can be a bird's-eye feature map determined based on the perception information collected by the vehicle. The sample bird's-eye feature map can be rasterized perception information generated based on the bird's-eye (BEV) spatial representation. The sample bird's-eye feature map can be directly obtained from the upstream perception module. The sample bird's-eye feature map may include but is not limited to drivable area information, route information, and occupancy information.

[0071] It should be noted that the pre-trained trajectory planning model and the pre-trained control planning model may be models determined by separate training.

[0072] It should be noted that the sample data used to train the trajectory planning model, the sample data used to train the control planning model, and the sample data used to train the joint path planning model can be the same training sample data set.

[0073] In this way, the joint path planning model including the pre-trained trajectory planning model and the pre-trained control planning model can be trained and processed through a sample bird's-eye view feature map based on the sample vehicle driving data and the perception information corresponding to the sample vehicle driving data, so that the path planning and control planning tasks can learn from each other and improve together, and a more accurate and effective end-to-end joint path planning model can be obtained, which ensures the planning performance and effectiveness of the model, thereby ensuring the reliability of the path planning and control planning of the autonomous driving vehicle.

[0074] Optionally, in a possible implementation of this embodiment, before step 102, the pre-trained trajectory planning model to be trained is iteratively trained based on the sample data to obtain a pre-trained trajectory planning model that has been trained.

[0075] In this implementation, the pre-trained trajectory planning model to be trained may be a trajectory planning model to be trained. The trained pre-trained trajectory planning model may be used for joint path planning model training processing.

[0076] Optionally, in a possible implementation of this embodiment, the pre-trained trajectory planning model to be trained includes a first encoder, a second encoder, and a decoder. Before step 102, first, the sample bird's-eye view feature map can be encoded using the first encoder to obtain bird's-eye view feature map encoding information. Secondly, the sample vehicle driving data is encoded using the second encoder to obtain vehicle driving feature encoding information. Thirdly, the bird's-eye view feature map encoding information and the vehicle driving feature encoding information are feature spliced ​​to obtain fused feature encoding information. Thirdly, the fused feature encoding information is decoded using the decoder to obtain a trajectory planning result. Thirdly, based on the sample data and the trajectory planning result, the trajectory planning model to be trained is iteratively trained to obtain the pre-trained trajectory planning model that has been trained.

[0077] In a specific implementation process of the implementation method, in response to the iterative training process reaching a preset first convergence condition, a pre-trained trajectory planning model that has been trained and initial values ​​of model parameters of the pre-trained trajectory planning model in the joint path planning model are obtained. The initial values ​​of model parameters of the pre-trained trajectory planning model can be used for subsequent training processing of the joint path planning model.

[0078] In this implementation, the pre-trained trajectory planning model may also include a decoding network, a classification network, and a regression network.

[0079] In another specific implementation process of this implementation method, first, the decoding network can be used to decode the fused feature encoding information to obtain fused feature decoding information. Then, the fused feature decoding information can be input into the classification network to obtain the driving direction category information of the trajectory. Then, the fused feature decoding information can be input into the regression network to obtain the location information of the trajectory. Then, based on the driving direction category information and the location information, the trajectory planning result can be obtained.

[0080] In this implementation, the decoding network may be a decoding network based on a multi-layer transform structure. Preferably, the decoding network may be a decoding network based on a 4-layer transform structure.

[0081] In this implementation, the driving direction category information of the track may include straight ahead, left turn, right turn, etc. The position information of the track may include the three-dimensional coordinate information and speed of the track points obtained by planning.

[0082] In one case of the specific implementation process, the driving direction category information and the position information of the trajectory corresponding to the driving direction category information may be used as the trajectory planning result.

[0083] For example, for one planning cycle, the trajectory planning result of the planning cycle may include going straight and a series of trajectory points corresponding to going straight. For another planning cycle, the trajectory planning result of the planning cycle may include turning left and a series of trajectory points corresponding to turning left.

[0084] In this implementation, the classification network may be a classification head network. The classification head network may be mainly composed of MLP, and the loss function of the classification head network is cross entropy. The classification network may decode the direction category of the trajectory, for example, straight ahead, left turn, right turn, etc.

[0085] Here, the regression network may be an Ego Traj Head network of the trajectory. The Ego Traj Head network may be mainly composed of MLP, and the Ego Traj Head network may plan the trajectory position information, for example, the three-dimensional position and speed of the vehicle in the next few seconds. The loss function of the Ego Traj Head network is the L2 distance.

[0086] In this way, the fused feature encoding information can be decoded by using a decoding network, a classification network, and a regression network to obtain the corresponding trajectory planning results including the driving direction category information and the trajectory position information, which can further improve the accuracy of the model output results.

[0087] In this implementation, the first encoder may include a spatial feature encoder and a first position encoder.

[0088] In another specific implementation process of this implementation, first, the spatial feature encoder can be used to perform spatial feature encoding processing on the sample bird's-eye view feature map to obtain spatial feature encoding information of the sample bird's-eye view feature map. Secondly, the first position encoder can be used to perform position encoding processing on the sample bird's-eye view feature map to obtain position encoding information of the sample bird's-eye view feature map. Thirdly, the spatial feature encoding information of the sample bird's-eye view feature map and the position encoding information of the sample bird's-eye view feature map are added to obtain the bird's-eye view feature map encoding information.

[0089] In this specific implementation process, the spatial feature encoder can be an encoder based on a convolutional neural network (CNN) structure. The spatial feature encoder can be used to process the spatial feature information in the sample bird's-eye view feature map. The spatial feature information in the sample bird's-eye view feature map may include edge information and shape information of the relevant area.

[0090] In one case of the specific implementation process, the sample bird's-eye view feature map can be input into a CNN-based encoder to output spatial feature encoding information of the sample bird's-eye view feature map.

[0091] In this specific implementation process, the first position encoder can be an encoder based on a multilayer perceptron (MLP) structure. The first position encoder can be used to process the position information in the sample bird's-eye view feature map. Here, the position information in the sample bird's-eye view feature map may include coordinate information, etc.

[0092] In another specific implementation process of this implementation method, the sample bird's-eye view feature map can be input into a position encoder based on a multi-layer perceptron, and the position encoding information of the sample bird's-eye view feature map can be output.

[0093] In this way, the input sample bird's-eye view feature map can be encoded by using the spatial feature encoder and the first position encoder respectively, thereby improving the accuracy of the results of the spatial feature encoding processing and the results of the position encoding processing, thereby further improving the accuracy of the results of the encoding processing of the sample bird's-eye view feature map.

[0094] In this implementation, the second encoder may include a driving state characteristic encoder and a second position encoder.

[0095] In another specific implementation process of this implementation mode, first, the sample vehicle driving data can be input into the driving state feature encoder to obtain the state feature encoding information of the output sample vehicle driving data. Secondly, the sample vehicle driving data can be input into the second position encoder to obtain the position encoding information of the output sample vehicle driving data. Thirdly, the state feature encoding information and the position encoding information can be added to obtain the vehicle driving feature encoding information.

[0096] In this implementation, the driving state feature encoder may be an encoder based on an MLP structure. The driving state feature encoder may be used to process driving state information in the sample vehicle driving data. The driving state information of the autonomous driving vehicle may include speed, acceleration, direction, etc.

[0097] In this implementation, the second position encoder may also be an encoder based on an MLP structure. The second position encoder may be used to process position information in vehicle driving data of an autonomous driving vehicle. The position information of the autonomous driving vehicle may include coordinate information, etc., and the coordinate information may include two-dimensional coordinate information and three-dimensional coordinate information.

[0098] In this way, the input sample vehicle driving data can be encoded by using the driving state feature encoder and the second position encoder respectively, thereby improving the accuracy of the driving state feature encoding processing results and the position encoding processing results, thereby further improving the accuracy of the obtained vehicle driving feature encoding information.

[0099] In this implementation, the pre-trained trajectory planning model to be trained may also include a splicing network.

[0100] In another specific implementation process of this implementation method, the stitching network can be used to perform feature stitching processing on the bird's-eye view feature map coding information and the vehicle driving feature coding information to obtain a result of the feature stitching processing, and then based on the result of the feature stitching processing, fused feature coding information can be obtained.

[0101] In this implementation, the concatenated network may be a concatenated layer (Concat) network.

[0102] It should be noted that the various specific implementation processes provided in this implementation method can be combined with each other to implement the model training method of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned various specific implementation processes, which will not be repeated here.

[0103] Optionally, in a possible implementation of this embodiment, before step 103, first, the sample data may be input into a pre-trained control planning model to be trained to obtain an output control planning result. Secondly, based on the sample data and the control planning result, the control planning model to be trained may be iteratively trained to obtain the pre-trained control planning model that has been trained.

[0104] In this implementation, the pre-trained control planning model to be trained may be a control planning model to be trained. The trained pre-trained control planning model may be used for joint path planning model training processing, that is, may be a model used for joint path planning model training.

[0105] In a specific implementation process of this implementation method, in response to the iterative training process reaching a preset second convergence condition, a trained pre-trained control planning model and initial values ​​of model parameters of the pre-trained control planning model in the joint path planning model are obtained.

[0106] Here, the initial values ​​of the model parameters of the pre-trained control planning model can be used for the subsequent training process of the joint path planning model.

[0107] In this implementation, the pre-trained control planning model can control the network. The control network can be a control output head network, i.e., a control head network. The control network can be mainly composed of MLP. The loss function of the control network can be L2 distance. The control network can decode the control planning result of the vehicle.

[0108] Here, the control planning result may be control planning information, which may include but is not limited to a steering wheel angle, an accelerator pedal opening, a brake pedal opening, and the like.

[0109] It should be noted that the specific implementation process provided in this implementation can be combined with the various specific implementation processes provided in the aforementioned implementation to implement the model training method of this embodiment. For a detailed description, please refer to the relevant content in the aforementioned implementation, which will not be repeated here.

[0110] Figure 2 is a flow chart of a path planning method provided by another embodiment of the present application, such as Figure 2 shown.

[0111] Step 201: Obtain vehicle driving data to be processed and a bird's-eye view feature map corresponding to the vehicle driving data.

[0112] Step 202: Input the vehicle driving data and the bird's-eye view feature map into a joint path planning model to obtain trajectory planning information and control planning information output by the joint path planning model.

[0113] Step 203: Obtain a joint path planning result based on the trajectory planning information and the control planning information, wherein the joint path planning model includes a trajectory planning model and a control planning model.

[0114] It should be noted that the joint path planning model is pre-trained using the model training method in the above-mentioned embodiment. The joint path planning model can be a model based on a neural network.

[0115] It should be noted that vehicle driving data may include but is not limited to the position, speed, acceleration, direction, etc. of the autonomous driving vehicle.

[0116] It should be noted that the bird's-eye view feature map can be determined based on perception information. The bird's-eye view feature map can be based on rasterized perception information generated under the BEV spatial representation. The bird's-eye view feature map can be directly obtained from the upstream perception module. The bird's-eye view feature map may include but is not limited to drivable area information, route information, and occupancy information.

[0117] In this way, by obtaining the vehicle driving data to be processed and the bird's-eye view feature map corresponding to the vehicle driving data, the vehicle driving data and the bird's-eye view feature map are input into the joint path planning model to obtain a joint path planning result. The joint path planning model includes a trajectory planning model and a control planning model. Since the joint path planning model has better end-to-end planning performance, the end-to-end joint path planning model is used to perform path and control planning on the acquired vehicle driving data and the bird's-eye view feature map corresponding to the vehicle driving data. While ensuring the accuracy of the joint path planning result, the corresponding joint path planning result can be directly output based on the end-to-end planning model, which reduces the computing power overhead of planning, thereby improving the reliability and efficiency of path planning of autonomous driving vehicles.

[0118] Optionally, in a possible implementation of this embodiment, in step 202, first, the trajectory planning model is used to perform trajectory planning processing on the vehicle driving data and the bird's-eye view feature map to obtain trajectory planning information and fusion feature decoding information. Secondly, the control planning model is used to perform control planning processing on the fusion feature decoding information to obtain control planning information.

[0119] In this implementation, the trajectory planning model includes a first encoder, a second encoder and a decoder.

[0120] In a specific implementation process of this implementation method, first, the first encoder can be used to encode the bird's-eye view feature map to obtain bird's-eye view feature map encoding information. Secondly, the second encoder is used to encode the vehicle driving data to obtain vehicle driving feature encoding information. Thirdly, the bird's-eye view feature map encoding information and the vehicle driving feature encoding information are subjected to feature splicing processing to obtain fused feature encoding information. Thirdly, the decoder is used to decode the fused feature encoding information to obtain trajectory planning information.

[0121] In this implementation, the decoder may include a decoding network, a classification network, and a regression network.

[0122] In another specific implementation process of this implementation method, first, the decoding network can be used to decode the fused feature encoding information to obtain fused feature decoding information. Secondly, the fused feature decoding information is input into the classification network to obtain the driving direction category information of the trajectory. Thirdly, the fused feature decoding information can be input into the regression network to obtain the location information of the trajectory. Thirdly, the driving direction category information and the location information can be used as trajectory planning information.

[0123] In another specific implementation process of this implementation, first, the decoding network can be used to decode the fused feature encoding information to obtain fused feature decoding information. Secondly, the fused feature decoding information is input into the control planning model to obtain control planning information.

[0124] In this implementation, the joint path planning model also includes a splicing network.

[0125] In another specific implementation process of this implementation method, the stitching network can be used to perform feature stitching processing on the bird's-eye view feature map coding information and the vehicle driving feature coding information to obtain a result of the feature stitching processing, and then based on the result of the feature stitching processing, fused feature coding information can be obtained.

[0126] Optionally, in a possible implementation of this embodiment, in step 203, the trajectory planning information and the control planning information may be used as a joint path planning result. The joint path planning result may include the driving direction category information of the trajectory, the location information of the trajectory, and the control planning information of the trajectory.

[0127] Here, the control planning information may correspond to the trajectory planning information.

[0128] Exemplarily, the joint path planning result may include driving direction category information, location information, and control planning information corresponding to each trajectory point of a planned series of trajectory points.

[0129] It should be noted that the various specific implementation processes provided in this implementation can be combined with each other to implement the path planning method of this embodiment. For detailed description, please refer to the relevant content in the above implementation, which will not be repeated here.

[0130] In order to better understand the method of the embodiment of the present application, the method of the embodiment of the present application is described below with reference to the accompanying drawings and specific application scenarios.

[0131] Figure 3 is a flow chart of a path planning method provided by another embodiment of the present application, such as Figure 3 In this embodiment, the path planning method may include:

[0132] Step 301: Obtain sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data.

[0133] In this implementation, the sample vehicle driving data (ego information) may include but is not limited to the position, speed, acceleration, direction, etc. of the autonomous driving vehicle.

[0134] In this embodiment, the sample bird's-eye view feature map can be a sample bird's-eye view feature map determined based on perception information. The sample bird's-eye view feature map can be directly obtained from the upstream perception module. The sample bird's-eye view feature map can be rasterized perception information generated based on BEV spatial representation. The sample bird's-eye view feature map can include but is not limited to drivable area information (Drivable Area), route information (Route), and occupancy information (Occupancy), etc. The route information can be vehicle navigation route information determined based on navigation information.

[0135] Here, the drivable area information, route information, and occupancy information can all be represented as a feature map of height (H) × width (W) × number of channels (C), that is, a bird's-eye view feature map.

[0136] In this embodiment, the sample bird's-eye view feature map may be a sample bird's-eye view feature map of the perception information at the time corresponding to the sample vehicle driving data, or a sample bird's-eye view feature map of the perception information of the area corresponding to the sample vehicle driving data.

[0137] Step 302: Based on the sample data, the pre-trained trajectory planning model to be trained is iteratively trained to obtain a pre-trained trajectory planning model that has been trained.

[0138] In an embodiment, based on the sample data, the pre-trained trajectory planning model to be trained is iteratively trained until the preset first convergence condition is met to obtain the pre-trained trajectory planning model.

[0139] Step 303: Based on the sample data, iterative training is performed on the pre-trained control planning model to be trained to obtain a pre-trained control planning model that has been trained.

[0140] In an embodiment, based on the sample data, the pre-trained control planning model to be trained is iteratively trained until a preset second convergence condition is met, thereby obtaining a trained pre-trained control planning model.

[0141] In this embodiment, the pre-trained trajectory planning model to be trained and the pre-trained control planning model to be trained can be trained separately according to the sample data to obtain a trained pre-trained trajectory planning model and a trained pre-trained control planning model.

[0142] In this embodiment, the model parameters of the trained pre-trained trajectory planning model and the model parameters of the trained pre-trained control planning model are used as the initial values ​​of the model parameters of the joint path planning model to be trained, and the joint path planning model to be trained is jointly trained based on sample data.

[0143] Step 304: Use the pre-trained trajectory planning model to perform trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information.

[0144] Step 305: Use the pre-trained control planning model to perform control planning processing on the fused feature decoding information to obtain control planning information.

[0145] Step 306: Based on the sample data, trajectory planning information, and control planning information, the joint path planning model to be trained is iteratively trained to obtain a trained joint path planning model.

[0146] Step 307: Obtain the vehicle driving data to be processed and the bird's-eye view feature map corresponding to the vehicle driving data.

[0147] In this embodiment, the bird's-eye view feature map may include drivable area information, route information, and occupancy information.

[0148] In this embodiment, the vehicle driving data may include the position, speed, acceleration, direction, etc. of the vehicle in the current frame and historical frames.

[0149] Step 308: Input the vehicle driving data and the bird's-eye view feature map into the joint path planning model, and use the trajectory planning model of the joint path planning model to process the vehicle driving data and the bird's-eye view feature map to obtain trajectory planning information and fusion feature decoding information.

[0150] In this embodiment, the trajectory planning model includes a first encoder, a second encoder and a decoder.

[0151] Step 309: Use the control planning model of the joint path planning model to process the fused feature decoding information to obtain control planning information.

[0152] Step 310: Use the trajectory planning information and the control planning information as a joint path planning result.

[0153] In this embodiment, the control planning model may include a control head network, ie, a control network.

[0154] In this embodiment, Figure 4 FIG. 1 is a schematic diagram of an application scenario of a path planning method provided by another embodiment of the present application. Figure 4As shown, in this application scenario, the joint path planning model may include a trajectory planning model and a control planning model. The trajectory planning model may include a spatial feature encoder, a first position encoder, a driving state feature encoder, a second position encoder, a splicing network, a decoding network, a classification network, and a regression network. First, the vehicle driving data and the bird's-eye view feature map can be obtained, and the bird's-eye view feature map can include drivable area information, route information, and occupancy information. Secondly, the vehicle driving data and the bird's-eye view feature map are input into the joint path planning model, and the model can output the joint path planning result. The joint path planning result includes the predicted vehicle driving direction category information, trajectory position information, and control information.

[0155] Here, if Figure 4 As shown, first, the vehicle driving data and the bird's-eye view feature map corresponding to the vehicle driving data can be input into the joint path planning model. The joint path planning model may include a trajectory planning model and a control planning model. The trajectory planning model may include a spatial feature encoder, a first position encoder, a driving state feature encoder, a second position encoder, a splicing network, a decoding network, a classification network, and a regression network. The bird's-eye view feature map includes drivable area information, route information, and occupancy information. Secondly, the spatial feature encoder and the first position encoder are used to process the bird's-eye view feature map respectively, and the two processing results are added to obtain the bird's-eye view feature map encoding information. Thirdly, the driving state feature encoder and the second position encoder are used to process the vehicle driving data respectively, and the two processing results are added to obtain the vehicle driving feature encoding information. Thirdly, the splicing network is used to splice the bird's-eye view feature map encoding information and the vehicle driving feature encoding information to obtain the fused feature encoding information. Thirdly, the decoding network is used to process the fused feature encoding information to obtain the fused feature decoding information. Again, the fused feature decoding information is input into the classification network (CLSHead), regression network (Ego Traj Head), and control planning model (Control Head) to obtain the predicted vehicle's driving direction category information, trajectory position information, and control information. The vehicle's driving direction category information, trajectory position information, and control information are used as the joint path planning result.

[0156] In this embodiment, the input end of the joint path planning model can directly use vehicle driving data and rasterized perception information generated based on BEV spatial representation, that is, a bird's-eye view feature map. The bird's-eye view feature map can include drivable area information, route information, and occupancy information. All information can be generated in real time. Route information can be obtained from navigation information. Vehicle driving data, that is, vehicle information, can include the current and historical position, speed, acceleration, direction, etc. of the vehicle. Secondly, the bird's-eye view feature map and vehicle driving data are respectively passed through the encoder, and the embedded features (Embedding, Emb) of the position encoder are added to obtain two-way encoding features. The two-way encoding features are spliced ​​and input into the four-layer Transformer decoding network for decoding. Finally, the decoded feature information output by the decoding network is decoded through the classification network, regression network and control network to decode the direction category of the trajectory of the joint path planning and the corresponding three-dimensional position information and control information of the trajectory, and obtain the final joint path planning result.

[0157] In this embodiment, the trajectory planning model may include two output heads, one is a classification network (CLS Head) for trajectory classification, and the other is a regression network (Ego Traj Head) for trajectory regression. The classification network and the regression network can respectively decode the driving direction category of the trajectory, for example, straight, left turn, right turn, etc., and the trajectory position information, for example, the three-dimensional position of the vehicle in the next few seconds. The two output heads of the trajectory planning model are composed of MLP. The loss function of trajectory classification can be cross entropy. The loss function of trajectory regression can be L2 distance.

[0158] Here, the control planning model can be a control network, which includes a control output head (ControlHead), or can be composed of MLP, which can decode the control instructions of the vehicle, that is, the control planning information. The control planning information can include the steering wheel angle, the accelerator pedal opening, the brake pedal opening, etc. The loss function of the control network can be the L2 distance.

[0159] In this embodiment, it can be understood that the training strategy of the entire joint path planning model can be to first pre-train the trajectory planning model to be trained based on the sample data of the training data set, that is, including all encoders, decoding networks, and planning trajectory output head networks. This allows the entire trajectory planning model to converge to a relatively good initial value of the model parameters, so as to facilitate the subsequent joint training of the entire joint path planning model. At the same time, based on the encoder and decoding network of the trajectory planning model pre-trained in the first step, the corresponding parameters can be frozen, and only the sample data can be used to fine-tune the control output head network, that is, the control planning model to be trained. This allows the output of the control planning model to reach a better initial value of the model parameters. Finally, all model parameters are turned on for joint training of trajectory planning + control planning. In this way, the performance of the entire joint path planning model can be further improved, and the planning and control tasks can learn from each other to jointly improve the reliability of the model planning results.

[0160] In this embodiment, the solution in this embodiment can be used to build an end-to-end autonomous driving joint path planning and control planning model. Millions of data can be used for training, and ultimately the entire model can achieve excellent performance.

[0161] In addition, by adopting the solution in this embodiment, the complex traditional planning and control modules can be replaced by an entire neural network through an end-to-end autonomous driving joint path planning and control model, and the information of both can be output in parallel at the same time. The entire end-to-end model has a simple structure, a high performance ceiling, simple research and development and maintenance, and strong generalization. In the commercial vehicle industry, it can greatly reduce the cost of autonomous driving products, laying the foundation for the large-scale promotion of high-level autonomous driving products.

[0162] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Figure 5 A structural block diagram of a model training device provided by an embodiment of the present application is shown. Figure 5As shown. The model training device 500 of this embodiment may include a first acquisition unit 501, a trajectory planning unit 502, a control planning unit 503 and a model training unit 504. Among them, the joint path planning model to be trained includes a pre-trained trajectory planning model and a pre-trained control planning model. The first acquisition unit 501 is used to acquire sample data, and the sample data includes sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data; the trajectory planning unit 502 is used to use the pre-trained trajectory planning model to perform trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information; the control planning unit 503 is used to use the pre-trained control planning model to perform control planning processing on the fusion feature decoding information to obtain control planning information; the model training unit 504 is used to iteratively train the joint path planning model to be trained based on the sample data, the trajectory planning information, and the control planning information to obtain a trained joint path planning model.

[0165] Optionally, in a possible implementation of this embodiment, the pre-trained trajectory planning model to be trained includes a first encoder, a second encoder and a decoder, and the trajectory planning unit 502 can be used to use the first encoder to encode the sample bird's-eye view feature map to obtain bird's-eye view feature map encoding information; use the second encoder to encode the sample vehicle driving data to obtain vehicle driving feature encoding information; perform feature splicing processing on the bird's-eye view feature map encoding information and the vehicle driving feature encoding information to obtain fused feature encoding information; use the decoder to decode the fused feature encoding information to obtain a trajectory planning result; based on the sample data and the trajectory planning result, iteratively train the trajectory planning model to be trained to obtain the pre-trained trajectory planning model that has been trained.

[0166] Optionally, in a possible implementation of this embodiment, the decoder includes a decoding network, a classification network, and a regression network, and the trajectory planning unit 502 can be used to use the decoding network to decode the fused feature encoding information to obtain fused feature decoding information; input the fused feature decoding information into the classification network to obtain the driving direction category information of the trajectory; input the fused feature decoding information into the regression network to obtain the location information of the trajectory; and obtain the trajectory planning result based on the driving direction category information and the location information.

[0167] Optionally, in a possible implementation of this embodiment, the control planning unit 503 can be used to input the sample data into a pre-trained control planning model to be trained to obtain an output control planning result; based on the sample data and the control planning result, the control planning model to be trained is iteratively trained to obtain the pre-trained control planning model that has been trained.

[0168] In this embodiment, sample data can be acquired by a first acquisition unit, and the sample data includes sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data. Then, the trajectory planning unit can use the pre-trained trajectory planning model of the joint path planning model to be trained to perform trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information. The control planning unit can use the pre-trained control planning model of the joint path planning model to be trained to perform control planning processing on the fusion feature decoding information to obtain control planning information, so that the model training unit can iteratively train the joint path planning model to be trained based on the sample data, the trajectory planning information, and the control planning information to obtain a trained joint path planning model. Since the joint path planning model mainly composed of the pre-trained trajectory planning model and the pre-trained control planning model is trained based on the sample vehicle driving data and the sample bird's-eye view feature map of the perception information corresponding to the sample vehicle driving data, a more accurate and effective end-to-end joint path planning model can be obtained, which ensures the planning performance and effectiveness of the model, thereby ensuring the reliability of the path planning and control planning of the autonomous driving vehicle.

[0169] Figure 6 A structural block diagram of a path planning device provided by an embodiment of the present application is shown as follows: Figure 6 As shown. The path planning device 600 of this embodiment may include an acquisition unit 601, a planning unit 602 and an acquisition unit 603. The acquisition unit 601 is used to acquire the vehicle driving data to be processed and the bird's-eye view feature map corresponding to the vehicle driving data; the planning unit 602 is used to input the vehicle driving data and the bird's-eye view feature map into the joint path planning model to obtain the trajectory planning information and control planning information output by the joint path planning model; the acquisition unit 603 is used to obtain the joint path planning result based on the trajectory planning information and the control planning information; wherein the joint path planning model is determined by a device using model training, and the joint path planning model includes a trajectory planning model and a control planning model.

[0170] Optionally, in a possible implementation of this embodiment, the planning unit 602 can be used to use the trajectory planning model to perform trajectory planning processing on the vehicle driving data and the bird's-eye view feature map to obtain trajectory planning information and fused feature decoding information, and use the control planning model to perform control planning processing on the fused feature decoding information to obtain control planning information.

[0171] In this embodiment, the vehicle driving data to be processed and the bird's-eye view feature map corresponding to the vehicle driving data can be obtained by the acquisition unit, and then the vehicle driving data and the bird's-eye view feature map can be input into the joint path planning model by the planning unit to obtain the trajectory planning information and control planning information output by the joint path planning model. The acquisition unit obtains the joint path planning result based on the trajectory planning information and the control planning information. The joint path planning model includes a trajectory planning model and a control planning model. Since the joint path planning model has better end-to-end planning performance, the end-to-end joint path planning model is used to perform path and control planning on the acquired vehicle driving data and the bird's-eye view feature map corresponding to the vehicle driving data. While ensuring the accuracy of the joint path planning result, the corresponding joint path planning result can be directly output based on the end-to-end planning model, thereby reducing the computing power overhead of planning, thereby improving the reliability and efficiency of path planning of the autonomous driving vehicle.

[0172] In the technical solution of this application, the user personal information involved, such as the collection, storage, use, processing, transmission, provision and disclosure of user images and attribute data, etc., complies with the provisions of relevant laws and regulations and does not violate public order and good morals.

[0173] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0174] According to an embodiment of the present application, further, an autonomous driving vehicle including the provided electronic device is provided, and the autonomous driving vehicle may include an unmanned vehicle of level L2 or above, for example, an unmanned commercial vehicle, an unmanned logistics vehicle, an autonomous driving logistics vehicle, an unmanned electric vehicle, etc.

[0175] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0176] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0177] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0178] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the method of path planning and the method of model training. For example, in some embodiments, the method of path planning and the method of model training may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the method of path planning and the method of model training described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the path planning method and the model training method in any other appropriate manner (for example, by means of firmware).

[0179] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0180] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0181] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0183] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0184] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0185] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0186] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A model training method, characterized in that: The joint path planning model to be trained includes a pre-trained trajectory planning model and a pre-trained control planning model, and the method includes: Acquire sample data, the sample data including sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data; Using the pre-trained trajectory planning model, performing trajectory planning processing on the sample data to obtain trajectory planning information and fusion feature decoding information; Using the pre-trained control planning model, performing control planning processing on the fused feature decoding information to obtain control planning information; Based on the sample data, the trajectory planning information, and the control planning information, an iterative training process is performed on the joint path planning model to be trained to obtain a trained joint path planning model.

2. The method according to claim 1, characterized in that The pre-trained trajectory planning model to be trained includes a first encoder, a second encoder and a decoder. Before performing trajectory planning processing on the sample data using the pre-trained trajectory planning model, the method includes: Using the first encoder, encoding the sample bird's-eye view feature image to obtain bird's-eye view feature image encoding information; Using the second encoder, encoding the sample vehicle driving data to obtain vehicle driving feature encoding information; Performing feature splicing processing on the bird's-eye view feature image coding information and the vehicle driving feature coding information to obtain fused feature coding information; Using the decoder, decoding the fused feature encoding information to obtain a trajectory planning result; Based on the sample data and the trajectory planning result, the trajectory planning model to be trained is iteratively trained to obtain the pre-trained trajectory planning model.

3. The method according to claim 2, characterized in that The decoder includes a decoding network, a classification network, and a regression network. The decoder is used to decode the fusion feature encoding information to obtain a trajectory planning result, including: Using the decoding network, the fused feature encoding information is decoded to obtain fused feature decoding information; Inputting the fused feature decoding information into the classification network to obtain the driving direction category information of the trajectory; Inputting the fused feature decoding information into the regression network to obtain the position information of the trajectory; The trajectory planning result is obtained based on the driving direction category information and the position information.

4. The method according to claim 1, characterized in that: Before the control planning process is performed on the fused feature decoding information by using the pre-trained control planning model, the method includes: Inputting the sample data into a pre-trained control planning model to be trained to obtain an output control planning result; Based on the sample data and the control planning result, the control planning model to be trained is iteratively trained to obtain the pre-trained control planning model that has been trained.

5. A path planning method, characterized in that: The method comprises: Acquire vehicle driving data to be processed and a bird's-eye view feature map corresponding to the vehicle driving data; Inputting the vehicle driving data and the bird's-eye view feature map into a joint path planning model to obtain trajectory planning information and control planning information output by the joint path planning model; Based on the trajectory planning information and the control planning information, obtaining a joint path planning result; The joint path planning model is determined by the method according to any one of claims 1 to 4, and the joint path planning model includes a trajectory planning model and a control planning model.

6. A model training device, characterized in that: The joint path planning model to be trained includes a pre-trained trajectory planning model and a pre-trained control planning model, and the device includes: A first acquisition unit is used to acquire sample data, wherein the sample data includes sample vehicle driving data and a sample bird's-eye view feature map corresponding to the sample vehicle driving data; A trajectory planning unit, used to perform trajectory planning processing on the sample data using the pre-trained trajectory planning model to obtain trajectory planning information and fusion feature decoding information; A control planning unit, configured to perform control planning processing on the fused feature decoding information using the pre-trained control planning model to obtain control planning information; A model training unit is used to iteratively train the joint path planning model to be trained based on the sample data, the trajectory planning information, and the control planning information to obtain a trained joint path planning model.

7. A path planning device, characterized in that: The device comprises: An acquisition unit, used to acquire vehicle driving data to be processed and a bird's-eye view feature map corresponding to the vehicle driving data; A planning unit, used for inputting the vehicle driving data and the bird's-eye view feature map into a joint path planning model to obtain trajectory planning information and control planning information output by the joint path planning model; An obtaining unit, configured to obtain a joint path planning result based on the trajectory planning information and the control planning information; Wherein, the joint path planning model is determined by the device as described in claim 6, and the joint path planning model includes a trajectory planning model and a control planning model.

8. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

10. An autonomous driving vehicle comprising the electronic device as claimed in claim 8.

Images