Method and apparatus for trajectory planning

Abstract

According to embodiments of this disclosure, a method, apparatus, device, storage medium, and program product for trajectory planning are provided. The method includes: generating a first trajectory and target environment features corresponding to the target environment information based on target environment information of an autonomous vehicle using a first planning model; and generating a target trajectory of the autonomous vehicle based on the first trajectory and the target environment features using a second planning model, wherein the second planning model is trained based on a reinforcement learning process, and reward information associated with the reinforcement learning process is determined based on executing a set of actions generated by the second planning model in a simulator. Based on this approach, embodiments of this disclosure can improve the quality of the generated trajectory.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for trajectory planning. Background Technology

[0002] Autonomous driving is a technology that uses computers to replace or assist human drivers in perceiving the vehicle's surroundings, planning the vehicle's trajectory, and controlling the vehicle to reach a designated destination.

[0003] Autonomous vehicles can use perception technology to identify their surroundings and employ machine learning algorithms to optimize routes and make decisions. However, existing trajectory planning capabilities still face challenges, especially when dealing with unfamiliar or extreme traffic scenarios; the system's generalization ability and adaptability need further improvement. Summary of the Invention

[0004] In a first aspect of this disclosure, a trajectory planning method is provided. The method includes: generating a first trajectory and target environment features corresponding to the target environment information based on target environment information of an autonomous vehicle using a first planning model; and generating a target trajectory of the autonomous vehicle based on the first trajectory and the target environment features using a second planning model, wherein the second planning model is trained based on a reinforcement learning process, and reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

[0005] In a second aspect of this disclosure, an apparatus for trajectory planning is provided. The apparatus includes: a first generation module configured to generate a first trajectory and target environment features corresponding to the target environment information based on target environment information of an autonomous vehicle using a first planning model; and a second generation module configured to generate a target trajectory of the autonomous vehicle based on the first trajectory and the target environment features using a second planning model, wherein the second planning model is trained based on a reinforcement learning process, and reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0009] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 An example of trajectory planning based on imitation learning is shown;

[0012] Figure 2 Example systems according to some embodiments of this disclosure are shown;

[0013] Figure 3 Training diagrams according to some embodiments of this disclosure are shown;

[0014] Figure 4 A schematic diagram illustrating an example process of trajectory planning according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A schematic structural block diagram of an example device for trajectory planning according to certain embodiments of the present disclosure is shown; and

[0016] Figure 6 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0021] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0022] As briefly mentioned earlier, existing trajectory planning capabilities still face challenges, especially when dealing with unseen or extreme traffic scenarios, where the system's generalization ability and adaptability need improvement. Furthermore, despite continuous technological advancements, relying solely on data-driven methods can lead to overfitting. The system may perform well in scenarios covered by the training data, but its performance may degrade in complex or rare, unseen scenarios.

[0023] For example, Figure 1 An example trajectory planning process based on imitation learning (IL) is shown. Figure 1Examples a and b in the text show that at a certain intersection, the most common behavior is to go straight.

[0024] Figure 1 Example c illustrates the performance of the imitation learning strategy in a scenario similar to the training data. In this scenario, because the strategy is trained based on common behaviors (such as moving straight), imitation learning can generate a plausible trajectory when the target is similar to the target in the training data.

[0025] Figure 1 Example d in the diagram illustrates a situation where the imitation learning strategy might generate an unreasonable trajectory when typical goals or behaviors are unavailable. For instance, if the goal at an intersection is to turn rather than go straight, the imitation learning strategy, having primarily learned the behavior of going straight, may fail to generate a suitable turning trajectory.

[0026] As can be seen, imitation learning IL may encounter the "Copycat problem" in autonomous driving trajectory planning, that is, the policy may simply extrapolate from historical states without truly understanding the basic principles of driving, which will lead to a decline in performance in rare or unseen situations.

[0027] Embodiments of this disclosure propose a trajectory planning scheme. According to various embodiments of this disclosure, a first trajectory and target environment features corresponding to the target environment information are generated using a first planning model based on target environment information of an autonomous vehicle; and a target trajectory of the autonomous vehicle is generated using a second planning model based on the first trajectory and target environment features, wherein the second planning model is trained based on a reinforcement learning process, and the reward information associated with the reinforcement learning process is determined based on executing a set of actions generated by the second planning model in a simulator.

[0028] In this way, embodiments of the present disclosure can utilize imitation learning to capture and learn behavioral patterns in common driving scenarios, ensuring the generation of stable and reliable trajectory predictions in these scenarios. Furthermore, by introducing reinforcement learning components, embodiments of the present disclosure can further optimize trajectories, enabling them to exhibit better generalization capabilities when facing rare or unknown scenarios.

[0029] This two-stage approach not only improves the performance of autonomous driving systems in known challenging scenarios but also enhances their ability to cope with unexpected situations in unknown environments. By deeply integrating imitation learning and reinforcement learning, this research effectively mitigates the problem of over-reliance on training data, improves the rationality of decision-making, and enhances the model's adaptability to new situations, thereby improving the overall reliability and safety of autonomous driving systems.

[0030] Example System

[0031] Figure 2A schematic diagram of an example trajectory planning system 200 according to some embodiments of the present disclosure is shown. When the trajectory planning system 200 is applied, the system 200 can be deployed at an autonomous vehicle to perform the trajectory planning process for the autonomous vehicle. Furthermore, when the trajectory planning system 200 is trained, the system 200 can be trained, for example, by a suitable training device (e.g., a server).

[0032] like Figure 2 As shown, system 200 may include two planning models, namely, a first planning model 220 and a second planning model 250. In some embodiments, the first planning model 220 may be trained based on imitation learning (IL), and the second planning model 250 may be trained based on reinforcement learning (RL).

[0033] like Figure 2 As shown, in the application phase (also known as the reasoning phase) of system 200, system 200 can acquire environmental information 210 associated with autonomous vehicles and can process environmental information 210 using the first planning model 220.

[0034] As an example, environmental information 210 may include detection boxes of objects associated with the autonomous vehicle, which may be represented, for example, as 3D boxes. Furthermore, environmental information 210 may also include, for example, map data (e.g., high-precision map data) that may represent map elements in the traffic scene associated with the autonomous vehicle.

[0035] like Figure 2 As shown, the first planning model 220 can generate a first trajectory 240 based on environmental information 210. Furthermore, the first planning model 220 can also generate environmental features 230. As an example, environmental features 230 can be hidden features corresponding to the feature layer preceding the motion prediction head of the first planning model 220.

[0036] As an example, the first planning model 220 can use a multi-layer fully connected neural network to encode detection boxes and map information. Furthermore, the first planning model 220 can also fuse information from the two sources. For instance, the first planning model 220 can employ a cross-attention algorithm to fuse map information and detection box information.

[0037] Furthermore, the first planning model 220 can utilize a multi-layer fully connected network to process the fused features to predict the future trajectory of the autonomous vehicle. As an example, the first planning model 220 can generate multiple predicted trajectories for the autonomous vehicle. Different trajectories may, for example, have corresponding confidence levels. As an example, environmental features 230 may correspond to features fused from multi-source information.

[0038] Furthermore, environmental features 230 and the first trajectory 240 can be further provided to a second planning model 250 trained based on reinforcement learning to generate the target trajectory 260 of the autonomous vehicle.

[0039] As an example, environmental features 230 may include features fused from multiple sources, and the first trajectory 240 may indicate the motion information of the autonomous vehicle at different times, such as acceleration and steering angle.

[0040] Furthermore, the second planning model 250 can observe the environmental features 230 and the first trajectory 240 as environmental variables, and can generate the next action based on the current environmental features, thereby determining the target trajectory 260.

[0041] As an example, the first trajectory 240 and / or the target trajectory 260 can indicate the pose and motion information of the autonomous vehicle at different times, so as to control the autonomous vehicle to drive according to the predicted motion information.

[0042] In this way, embodiments of the present disclosure can utilize imitation learning to capture and learn behavioral patterns in common driving scenarios, ensuring the generation of stable and reliable trajectory predictions in these scenarios. Furthermore, by introducing reinforcement learning components, embodiments of the present disclosure can further optimize trajectories, enabling them to exhibit better generalization capabilities when facing rare or unknown scenarios.

[0043] The following will be further combined Figure 3 To describe the training process of System 200. For example... Figure 3 As shown, the training system 300 may include a simulation learning unit 320, a reinforcement learning unit 330, and a simulator 310.

[0044] like Figure 3 As shown, during the training process, the simulation learning unit 320 can acquire environmental information 320 (also known as training environment information) and use the policyr 322 (i.e., the first planning model) to generate environmental features 324 (also known as training environment features) and a set of simulation learning actions 326 (also known as the second trajectory).

[0045] Furthermore, the reinforcement learning unit 330 can acquire input information 332, which may include environmental features 324 generated by the simulation learning phase, a set of simulation learning actions 326, and corresponding rewards.

[0046] Furthermore, based on the reinforcement learning process, the reinforcement learning unit 330 can determine the value 334 (also known as the Q value) corresponding to the action and the value 336 (also known as the V value) corresponding to the state.

[0047] Furthermore, the reinforcement learning unit 330 can control the policyr 314 to generate a corresponding set of reinforcement learning actions based on the loss of the value 334. Furthermore, the simulator 310 can execute reinforcement learning actions based on environmental information 312 to determine the reward information corresponding to the reinforcement learning. As an example, the simulator 310 can be used to perform closed-loop environment simulation.

[0048] In some embodiments, the reward information may include a first reward component, also known as a collision component. In some embodiments, the minimum distance between the vehicle (i.e., the self-vehicle) and the obstacle during the simulation process may be determined based on the simulation results of the simulator 330, and the first reward component may be determined based on the first minimum distance.

[0049] As an example, if the vehicle collides with an obstacle during the simulation, its first reward component can be set to a negative value, for example.

[0050] In some embodiments, the reward information may include a second reward component, also known as a road deviation component. As an example, the minimum distance between the vehicle (i.e., the driver) and the road edge during the simulation can be determined based on the simulation results of the simulator 330. Further, the second reward component can be determined based on the minimum distance to the road edge.

[0051] As an example, the minimum distance to the road edge when a vehicle drives off the road can be a negative value. Accordingly, the second reward component can also be set to a negative value, for example.

[0052] In some embodiments, the reward information may include a third reward component, also known as a completion component. As an example, it can be determined whether the vehicle (i.e., the driver) successfully reached its destination during the simulation based on the simulation results of simulator 310. In response to the vehicle successfully reaching its destination, the third reward component can be set to a preset value. Conversely, if the vehicle does not reach its destination, the third reward component can also be set to zero, for example.

[0053] In some embodiments, the reward information may include a fourth reward component, also known as a progress component. As an example, based on the simulation results of simulator 310, a first distance from the vehicle (i.e., the self-vehicle) to the destination at a first time t1 and a second distance from the destination at a second time t2 can be determined, where the first time t1 and the second time t2 are adjacent times. Further, the fourth reward component can be determined based on the difference between the first distance and the second distance.

[0054] For example, if t2 is the next moment after t1, the fourth reward component can be set to a positive value if the vehicle is closer to its destination. Conversely, the fourth reward component can be set to a negative value.

[0055] In some embodiments, the reward information includes a fifth reward component, also known as a smoothness component. For example, two consecutive actions of the vehicle (i.e., the self-driving vehicle) during the simulation process can be determined based on the simulation results of the simulator 310. Further, the fifth reward component can be determined based on the smoothness of the two actions.

[0056] As an example, the smoothness level is associated with the difference in acceleration and / or orientation corresponding to two actions. For instance, the smoothness level can be determined based on a comparison of the acceleration difference and a first threshold. Alternatively, the smoothness level can be determined based on a comparison of the orientation difference and a second threshold.

[0057] Therefore, in the reinforcement learning process, if the value network and value-action network of reinforcement learning are unstable, the output of the action network of reinforcement learning can be close to the output of the imitation learning process.

[0058] This two-stage approach not only improves the performance of autonomous driving systems in known challenging scenarios but also enhances their ability to cope with unexpected situations in unknown environments. By deeply integrating imitation learning and reinforcement learning, this research effectively mitigates the problem of over-reliance on training data, improves the rationality of decision-making, and enhances the model's adaptability to new situations, thereby improving the overall reliability and safety of autonomous driving systems.

[0059] Example process

[0060] Figure 4 A flowchart of an example process 400 for trajectory planning according to some embodiments of the present disclosure is shown. Process 400 can be implemented at system 200. References are made below. Figure 1 Describe the process 400.

[0061] like Figure 4 As shown in box 410, system 200 uses a first planning model to generate a first trajectory and target environment features corresponding to the target environment information based on the target environment information of the autonomous vehicle.

[0062] In box 420, system 200 uses a second planning model to generate a target trajectory for the autonomous vehicle based on the first trajectory and target environment features. The second planning model is trained based on a reinforcement learning process, and the reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

[0063] In some embodiments, the reward information includes a first reward component, and the first reward component is determined based on the following process: determining a first minimum distance between the vehicle and the obstacle during the simulation based on the simulation results of the simulator; and determining the first reward component based on the first minimum distance.

[0064] In some embodiments, the reward information includes a second reward component, and the second reward component is determined based on the following process: determining a second minimum distance between the vehicle and the road edge during the simulation based on the simulation results of the simulator; and determining the second reward component based on the second minimum distance.

[0065] In some embodiments, the reward information includes a third reward component, and the third reward component is determined based on the following process: determining whether the vehicle successfully reached its destination during the simulation based on the simulation results of the simulator; and setting the third reward component to a preset value in response to the vehicle successfully reaching its destination.

[0066] In some embodiments, the reward information includes a fourth reward component, and the fourth reward component is determined based on the following process: determining a first distance from the vehicle to the destination at a first time and a second distance from the vehicle to the destination at a second time, the first time and the second time being adjacent times, based on the simulation results of the simulator; and determining the fourth reward component based on the difference between the first distance and the second distance.

[0067] In some embodiments, the reward information includes a fifth reward component, and the fifth reward component is determined based on the following process: determining two consecutive actions of the vehicle during the simulation process based on the simulation results of the simulator; and determining the fifth reward component based on the smoothness of the two actions.

[0068] In some embodiments, the smoothness is associated with the difference in acceleration and / or the difference in orientation between the two actions.

[0069] In some embodiments, a set of motion indicators provides acceleration and orientation information at multiple moments.

[0070] In some embodiments, a set of actions is generated by a second planning model based on a second trajectory and training environment features, which are generated by a first planning model based on training environment information.

[0071] Example devices and equipment

[0072] Figure 5 A schematic structural block diagram of a trajectory planning device 500 according to certain embodiments of the present disclosure is shown. The device 500 may be implemented as or included in an autonomous vehicle. The various modules / components in the device 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0073] As shown in the figure, the device 500 includes: a first generation module 510 configured to generate a first trajectory and target environment features corresponding to the target environment information based on the target environment information of the autonomous vehicle using a first planning model; and a second generation module 520 configured to generate a target trajectory of the autonomous vehicle based on the first trajectory and the target environment features using a second planning model, wherein the second planning model is trained based on a reinforcement learning process, and the reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

[0074] In some embodiments, the reward information includes a first reward component, and the first reward component is determined based on the following process: determining a first minimum distance between the vehicle and the obstacle during the simulation based on the simulation results of the simulator; and determining the first reward component based on the first minimum distance.

[0075] In some embodiments, the reward information includes a second reward component, and the second reward component is determined based on the following process: determining a second minimum distance between the vehicle and the road edge during the simulation based on the simulation results of the simulator; and determining the second reward component based on the second minimum distance.

[0076] In some embodiments, the reward information includes a third reward component, and the third reward component is determined based on the following process: determining whether the vehicle successfully reached its destination during the simulation based on the simulation results of the simulator; and setting the third reward component to a preset value in response to the vehicle successfully reaching its destination.

[0077] In some embodiments, the reward information includes a fourth reward component, and the fourth reward component is determined based on the following process: determining a first distance from the vehicle to the destination at a first time and a second distance from the vehicle to the destination at a second time, the first time and the second time being adjacent times, based on the simulation results of the simulator; and determining the fourth reward component based on the difference between the first distance and the second distance.

[0078] In some embodiments, the reward information includes a fifth reward component, and the fifth reward component is determined based on the following process: determining two consecutive actions of the vehicle during the simulation process based on the simulation results of the simulator; and determining the fifth reward component based on the smoothness of the two actions.

[0079] In some embodiments, the smoothness is associated with the difference in acceleration and / or the difference in orientation between the two actions.

[0080] In some embodiments, a set of motion indicators provides acceleration and orientation information at multiple moments.

[0081] In some embodiments, a set of actions is generated by a second planning model based on a second trajectory and training environment features, which are generated by a first planning model based on training environment information.

[0082] The units included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 500 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0083] Figure 6 A block diagram is shown illustrating a computing device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 6 The computing device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The computing device 600 shown can be used to achieve, for example Figure 2 The system 200 shown.

[0084] like Figure 6 As shown, computing device 600 is in the form of a general-purpose computing device. Components of computing device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage devices 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 600.

[0085] Computing device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 600.

[0086] The computing device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0087] The communication unit 640 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 600 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 600 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0088] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 600 can also communicate as needed with one or more external devices (not shown) via communication unit 640. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 600, or with any device (e.g., network card, modem, etc.) that enables computing device 600 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0089] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0090] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0091] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0092] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0094] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A trajectory planning method, comprising: Using the first planning model based on the target environment information of the autonomous vehicle, a first trajectory and target environment features corresponding to the target environment information are generated. as well as The autonomous vehicle generates a target trajectory using a second planning model based on the first trajectory and the target environment features. The second planning model is trained based on a reinforcement learning process, and the reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

2. The method of claim 1, wherein the reward information includes a first reward component, and the first reward component is determined based on the following process: Based on the simulation results of the simulator, determine the first minimum distance between the vehicle and the obstacle during the simulation process; and The first reward component is determined based on the first minimum distance.

3. The method of claim 1, wherein the reward information includes a second reward component, and the second reward component is determined based on the following process: Based on the simulation results of the simulator, determine the second minimum distance between the vehicle and the road edge during the simulation process; and The second reward component is determined based on the second minimum distance.

4. The method of claim 1, wherein the reward information includes a third reward component, and the third reward component is determined based on the following process: Based on the simulation results of the simulator, determine whether the vehicle successfully reached its destination during the simulation process; and In response to the vehicle successfully reaching its destination, the third reward component is set to a preset value.

5. The method of claim 1, wherein the reward information includes a fourth reward component, and the fourth reward component is determined based on the following process: Based on the simulation results of the simulator, a first distance from the vehicle to the destination at a first time point and a second distance from the vehicle to the destination at a second time point are determined, wherein the first time point and the second time point are adjacent times; and The fourth reward component is determined based on the difference between the first distance and the second distance.

6. The method of claim 1, wherein the reward information includes a fifth reward component, and the fifth reward component is determined based on the following process: Based on the simulation results of the simulator, two consecutive actions of the vehicle during the simulation process are determined; and The fifth reward component is determined based on the smoothness of the two actions.

7. The method of claim 6, wherein the smoothness is associated with the acceleration difference and / or orientation difference corresponding to the two actions.

8. The method of claim 1, wherein the set of actions indicates acceleration information and orientation information at multiple moments.

9. The method of claim 1, wherein the set of actions is generated by the second planning model based on the second trajectory and training environment features, and the second trajectory and the training environment features are generated by the first planning model based on training environment information.

10. A trajectory planning apparatus, comprising: The first generation module is configured to use a first planning model to generate a first trajectory and target environment features corresponding to the target environment information based on the target environment information of the autonomous vehicle. as well as The second generation module is configured to generate a target trajectory for the autonomous vehicle based on the first trajectory and the target environment features using a second planning model, wherein the second planning model is trained based on a reinforcement learning process, and the reward information associated with the reinforcement learning process is determined based on performing a set of actions generated by the second planning model in a simulator.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images