Trajectory Planning Method, Trajectory Display Method, Device, Vehicle, Medium and Chip

Abstract

Embodiments of the present disclosure provide a trajectory planning method, a trajectory display method, a device, a vehicle, a medium, and a chip, which relate to the field of autonomous driving. Among them, the trajectory planning method includes: obtaining map element information, the current position information, and the historical position information of a target object in a target area according to the environmental perception data of the target area; inputting the current position information, the historical position information, and the map element information into a feature extraction and fusion model to obtain a first fusion feature output by the feature extraction and fusion model; and generating a target planning trajectory of a target vehicle according to the first fusion feature. Among them, the above-mentioned target area is an area where the target vehicle including the to-be-planned trajectory is located, the above-mentioned target object includes the target vehicle and obstacles, and the above-mentioned first fusion feature includes interaction features representing the mutual influence between different target objects and between the target object and the map element.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and more particularly, to a trajectory planning method, a trajectory display method, a device, a vehicle, a medium, and a chip. Background Art

[0002] With the increasing application of autonomous driving technology, the environment faced by autonomous vehicles during driving is becoming more and more complex. In order to safely and reliably control the operation of autonomous vehicles, it is necessary to plan the trajectory of the vehicle. However, the trajectory planned by the trajectory planning method in the related art has poor robustness in a complex environment, and how to improve the robustness of vehicle trajectory planning has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, embodiments of the present disclosure propose a new technical solution for trajectory planning.

[0004] According to a first aspect of the embodiments of the present disclosure, there is provided a trajectory planning method, the method comprising:

[0005] Obtaining map element information, current position information, and historical position information of a target object in a target area according to environmental perception data of the target area; wherein, the target area is an area where a target vehicle located at a position where a trajectory to be planned is located, and the target object includes the target vehicle and an obstacle;

[0006] Inputting the current position information, the historical position information, and the map element information into a feature extraction and fusion model to obtain a first fusion feature output by the feature extraction and fusion model; wherein, the first fusion feature includes interaction features characterizing the mutual influence between different target objects and between the target object and the map element;

[0007] Generating a target planning trajectory of the target vehicle according to the first fusion feature.

[0008] Optionally, the feature extraction and fusion model includes a target encoder and a target decoder; the feature extraction and fusion model is used to generate the first fusion feature in the following manner:

[0009] Performing feature extraction on the map element information, the current position information, and the historical position information based on the target encoder to obtain map information features, historical trajectory features of the target object, and first predicted trajectory features;

[0010] Performing feature fusion on the map information features, the historical trajectory features of the target object, and the first predicted trajectory features based on the target decoder to obtain the first fusion feature.

[0011] Optionally, the target decoder is a decoder based on a cross-attention mechanism; the decoder based on the target fuses the map information features, the predicted trajectory features of the target object, and the historical trajectory features to obtain the first fused feature, including:

[0012] Fuse the first predicted trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain a second predicted trajectory feature;

[0013] Fuse the second predicted trajectory feature and the map information feature based on the cross-attention mechanism to obtain a third predicted trajectory feature;

[0014] Fuse the third predicted trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain the first fused feature.

[0015] Optionally, the target decoder includes a first cross-attention unit; fusing the first predicted trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain a second predicted trajectory feature, including:

[0016] Generate a first query feature according to the first predicted trajectory feature and the relative time dimension interaction feature of the target object; the relative time dimension interaction feature of the target object is an interaction feature used to represent the target object in the time dimension obtained by embedding coding and fusing the current position information and historical position information of the target object;

[0017] Generate a first key feature and a first value feature according to the historical trajectory feature and the relative time dimension interaction feature of the target object;

[0018] Input the first query feature, the first key feature, and the first value feature into the first cross-attention unit to generate the second predicted trajectory feature.

[0019] Optionally, the target decoder further includes a first cross-attention unit; fusing the second predicted trajectory feature and the map information feature based on the cross-attention mechanism to obtain a third predicted trajectory feature, including:

[0020] Generate a second query feature according to the second predicted trajectory feature and the relative position interaction feature of the target object; the relative position interaction feature of the target object is a feature used to represent the mutual influence between target objects obtained by embedding coding and fusing the current position information and historical position information of the target object;

[0021] Generate a second key feature and a second value feature according to the map information feature and the relative position interaction feature of the target object;

[0022] Input the second query feature, the second key feature, and the second value feature into the second cross-attention unit to generate the third predicted trajectory feature.

[0023] Optionally, the fusing of the third predicted trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain the first fused feature includes:

[0024] Generate a third query feature according to the third predicted trajectory feature and the relative map interaction feature of the target object; the relative map interaction feature of the target object is a feature obtained by embedding encoding and fusing according to the current position information, historical position information, and map element information of the target object and is used to characterize the mutual influence between the target object and the map element;

[0025] Generate a third key feature and a third value feature according to the historical trajectory feature and the relative map interaction feature of the target object;

[0026] Input the third query feature, the third key feature, and the third value feature into the third cross-attention unit to generate the first fused feature.

[0027] Optionally, the generating of the target planning trajectory of the target vehicle according to the first fused feature includes:

[0028] Input the first fused feature into a pre-generated trajectory prediction and planning model to obtain the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle output by the trajectory prediction and planning model;

[0029] Determine the target planning trajectory of the target vehicle according to the initial planning trajectory and the obstacle prediction trajectory.

[0030] Optionally, the trajectory prediction and planning model includes a self-vehicle trajectory planning module and an obstacle trajectory prediction module; the trajectory prediction and planning model generates the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle in the following manner:

[0031] The self-vehicle trajectory planning module generates the initial planning trajectory of the target vehicle according to the first fused feature;

[0032] The obstacle trajectory prediction module generates the obstacle prediction trajectory of the obstacle according to the first fused feature, the initial planning trajectory, and the current position information of the obstacle.

[0033] Optionally, the self-vehicle trajectory planning module includes a first self-attention unit and a first neural network unit; the self-vehicle trajectory planning module generates the initial planning trajectory of the target vehicle according to the first fused feature, including:

[0034] Extract the first trajectory feature of the target vehicle from the first fusion feature based on the first self-attention unit;

[0035] Input the first trajectory feature and the current state information of the target vehicle into the first neural network unit to obtain the initial planned trajectory of the target vehicle.

[0036] Optionally, the obstacle trajectory prediction module includes a fourth cross-attention unit, a second self-attention unit, and a second neural network unit; the obstacle trajectory prediction module generates the obstacle prediction trajectory of the obstacle according to the first fusion feature, the initial planned trajectory, and the current position information of the obstacle, including:

[0037] Fuse the first fusion feature, the initial planned trajectory, and the current position information of the obstacle based on the fourth cross-attention unit to obtain a second fusion feature;

[0038] Extract the trajectory feature of the obstacle from the second fusion feature based on the second self-attention unit;

[0039] Input the trajectory feature into the second neural network unit to obtain the obstacle prediction trajectory of the obstacle.

[0040] Optionally, the step of fusing the first fusion feature, the initial planned trajectory, and the current position information of the obstacle based on the fourth cross-attention unit to obtain a second fusion feature includes:

[0041] After performing embedding encoding and fusion on the current position information of the obstacle and the initial planned trajectory of the target vehicle, obtain the interactive feature of the obstacle relative to the target vehicle;

[0042] Generate a fourth query feature according to the first fusion feature and the interactive feature of the obstacle relative to the target vehicle;

[0043] Generate a fourth key feature and a fourth value feature according to the initial planned trajectory and the interactive feature of the obstacle relative to the target vehicle;

[0044] Input the fourth query feature, the fourth key feature, and the fourth value feature into the fourth cross-attention unit to generate the second fusion feature.

[0045] Optionally, the step of determining the target planned trajectory of the target vehicle according to the initial planned trajectory and the obstacle prediction trajectory includes:

[0046] Predict the collision probability of the target vehicle colliding with the obstacle according to the initial planned trajectory and the obstacle prediction trajectory;

[0047] When the collision probability is greater than or equal to a preset collision probability threshold, update the initial planned trajectory according to the predicted trajectory of the obstacle, and use the updated initial planned trajectory as the target planned trajectory to reduce the collision probability;

[0048] When the collision probability is less than the preset collision probability threshold, use the initial planned trajectory as the target planned trajectory.

[0049] According to a second aspect of the present disclosure, there is provided a trajectory display method, the method comprising:

[0050] Determine a target planned trajectory of a target vehicle according to environmental perception data of a target area;

[0051] Display the target planned trajectory;

[0052] Wherein, the target planned trajectory is a trajectory generated based on a first fusion feature output by a feature extraction and fusion model, the environmental perception data is used to obtain map element information in the target area and current position information and historical position information of a target object, the target area is an area including the position where the target vehicle is located, the target object includes the target vehicle and an obstacle, and the feature extraction and fusion model is used to output the first fusion feature according to the input current position information, historical position information and map element information.

[0053] According to a third aspect of the present disclosure, there is provided an electronic device, comprising a memory and a processor, the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of the first aspect and / or the second aspect.

[0054] According to a fourth aspect of the present disclosure, there is provided a vehicle, comprising a memory and a processor, the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of the first aspect and / or the second aspect.

[0055] According to a fifth aspect of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and the computer program realizes the method according to any one of the first aspect and / or the second aspect when executed by a processor.

[0056] According to a sixth aspect of the present disclosure, there is provided a chip, comprising a processing unit, and the processing unit is configured to execute the method according to any one of the first aspect and / or the second aspect.

[0057] Based on the trajectory planning method provided by the embodiments of the present disclosure, a first fusion feature representing the mutual influence between different target objects and between the target object and the map elements is generated, and a target planning trajectory of the target vehicle is generated according to the first fusion feature, realizing the interactive feedback among the target vehicle, the obstacle, and the map elements, and capable of achieving efficient and robust trajectory planning in a dynamic environment, thereby improving the safety, smoothness, and driving efficiency of the autonomous vehicle.

[0058] Through the following detailed description of the exemplary embodiments of the present disclosure with reference to the accompanying drawings, other features and advantages of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] Figure 1 is a schematic diagram of an intelligent networked system to which the method provided by the embodiments of the present disclosure can be applied.

[0061] Figure 2 is according to Figure 1 a schematic diagram of a vehicle provided by the embodiment shown.

[0062] Figure 3 is a schematic flowchart of a trajectory planning method provided by the embodiments of the present disclosure.

[0063] Figure 5 is a schematic structural diagram of a feature extraction and fusion model and a trajectory prediction and planning model provided by the embodiments of the present disclosure.

[0064] Figure 4 is a schematic structural diagram of another feature extraction and fusion model and a trajectory prediction and planning model provided by the embodiments of the present disclosure.

[0065] Figure 6 is a schematic diagram of an encoding and fusion process provided by the embodiments of the present disclosure.

[0066] Figure 7 is a schematic flowchart of a trajectory display method provided by the embodiments of the present disclosure.

[0067] Figure 8 is a schematic diagram of displaying a target planning trajectory provided by the embodiments of the present disclosure.

[0068] Figure 9 is a schematic structural diagram of an electronic device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure.

[0070] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way a limitation on the present disclosure, its application, or its use.

[0071] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the above techniques, methods, and devices should be regarded as part of the specification.

[0072] In all the examples shown and discussed herein, any specific values should be construed as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

[0073] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, further discussion thereof is not required in subsequent drawings.

[0074] First, the application scenarios of the embodiments of the present disclosure will be described.

[0075] Figure 1 is a schematic diagram of an intelligent connected system 100 to which the method provided by the embodiments of the present disclosure can be applied. As Figure 1 shown, the intelligent connected system 100 may include: a vehicle 101, a server 102, and a user terminal 103.

[0076] In some examples, the vehicle 101 may be a vehicle with an autonomous driving function. Among them, autonomous driving is also known as driverless or intelligent driving. A vehicle with an autonomous driving function can perform driving tasks such as environmental perception, decision-making and planning, and control execution. The levels of autonomous driving can refer to the automotive intelligence grading standard formulated by the Society of Automotive Engineers (SAE). For example, the L0 level is manual driving, L1 is assisted driving, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly autonomous driving, and L5 is fully autonomous driving. The above classification method for the levels of autonomous driving is only for example, and the present disclosure embodiments do not limit the classification criteria and levels of autonomous driving.

[0077] In some examples, the server 102 can be a single server or a distributed server cluster composed of multiple servers, and its deployment method can include a local server or a cloud server. The server 102 can communicate with the vehicle 101 and / or the user terminal 103 based on a communication network, and provide various services for the vehicle 101 and / or the user terminal 103. For example, the server can receive the perception data sent by the vehicle and provide services such as high-precision maps, data analysis, and decision-making planning for the vehicle. Another example is that the server can receive query instructions or control instructions sent by the user terminal and provide corresponding services for the user.

[0078] In some examples, the user terminal 103 can be any form of electronic device that provides services for users, such as a personal computer, a laptop, a smart tablet, a smart phone, a smart wearable device, etc. The user can interact with the vehicle or the server through the human-machine interaction terminal configured on the vehicle 101, or can also interact with the vehicle or the server through the user terminal 103. For example, the user can query the status and / or parameters of the vehicle through the user terminal, or control the vehicle to execute set tasks and / or modify configuration parameters, etc.; among them, the user terminal runs an application program based on the intelligent connected system to achieve interaction with the vehicle or the server. The application program can be a local application, a web application or a small program, etc., which is not limited here.

[0079] In some examples, the above application program running on the user terminal can provide authentication or authorization services for users. Users who have successfully authenticated and been granted corresponding permissions can query and / or control the vehicle within the granted permissions.

[0080] The vehicle 101, the server 102, and the user terminal 103 can communicate through the communication link provided by the communication network 104. The communication network 104 can include one or more networks of any type. For example, the communication network 104 can include the Internet, a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a public switched telephone network (PSTN), a satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC, etc. networks that provide communication, or a combination of the above multiple networks. The communication networks between the vehicle 101 and the server 102, between the user terminal 103 and the server 102, and between the user terminal 103 and the vehicle 101 can be the same or different.

[0081] It should be noted that Figure 1The structure of the intelligent networked system 100 shown is only schematic. The intelligent networked system in the embodiments of the present disclosure is not limited to the above structure and may include more or fewer devices as needed, or the devices may be combined or split. For example, the intelligent networked system may not include a user terminal and / or a server; again, for example, the user terminal and the server may be combined and deployed.

[0082] Figure 2 is provided according to Figure 1 the schematic diagram of a vehicle 101 shown in the embodiment. As Figure 2 shown, the vehicle 101 may include a sensing component 1011, a computing platform 1012, an execution component 1013, etc. Among them, the sensing component 1011, the computing platform 1012, and the execution component 1013 may be connected by a bus or other means.

[0083] In some examples, the sensing component 1011 may be used to collect information about the vehicle itself or the outside. The sensing component 1011 may include at least one of a vision sensing unit, a radar, a positioning and navigation unit, an inertial measurement unit (IMU), or other sensing units. Among them, the vision sensor unit may include one or more cameras, the radar may include at least one of a lidar, a millimeter-wave radar, an ultrasonic radar, or other radars, and the positioning and navigation unit may include at least one of a GPS system, a Beidou system, or other global positioning systems.

[0084] In some examples, the computing platform 1012 may include a device with computing capabilities for processing the sensed information collected by the sensing component 1011 to obtain control information and sending corresponding control instructions to the execution component 1013, so that the execution component 1013 performs corresponding actions, thereby achieving the control of the vehicle 101. Exemplarily, the computing platform 1012 may perform actions such as simultaneous localization and mapping (SLAM), path planning, and behavior decision-making on the vehicle, thereby achieving autonomous control of the vehicle. The computing platform 1012 may include at least one processor and at least one memory. Each processor may execute the instructions stored in the memory alone or jointly to implement the method provided by the embodiments of the present disclosure. The processor in the embodiments of the present disclosure may include at least one of a central processing unit (CPU), a graphic processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), a data processing unit (DPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), a microcontroller unit (MCU), or other processors. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk. In addition to storing instructions, the memory may also store data, such as high-precision maps, path information, the position, direction, speed, etc. of the vehicle. The data stored in the memory may be acquired and used by the processor.

[0085] In some examples, the computing platform of the vehicle may perform computing tasks independently or communicate with a server to complete computing tasks. For example, the computing platform of the vehicle may cooperate with the server to complete corresponding computing tasks.

[0086] The computing platform 1012 may be disposed in the vehicle 101, and part or all of the computing platform 1012 may also be disposed in the server corresponding to the vehicle. For example, functions with relatively high real-time requirements in the computing platform 1012 are disposed in the vehicle, and functions with relatively low real-time requirements are disposed in the server corresponding to the vehicle.

[0087] In some examples, the execution component 1013 is configured to perform corresponding actions based on the control of the computing platform 1012, so that the vehicle 101 completes a movement task. The execution component 1013 may include, for example, a power component, a braking component, a transmission component, a steering component, etc.

[0088] It should be noted that Figure 2 The structure of the vehicle 101 shown in is only schematic. The vehicle in the embodiments of the present disclosure is not limited to the above structure, and may include more or fewer components according to needs, and the devices may also be combined or split. For example, the vehicle may not include the above computing platform. For another example, the vehicle may further include a communication component, an interface component, a multimedia component, an input component, an output component, etc.

[0089] The embodiments of the present disclosure may be applied to the scenario of ego-vehicle trajectory planning of autonomous vehicles. With the increasing application of autonomous driving technology, the environment faced by autonomous vehicles during driving is becoming more and more complex. In order to safely and reliably control the operation of autonomous vehicles, on the basis of global path planning, it is also necessary to perform short-term trajectory planning on the vehicle to determine the best driving route of the vehicle itself.

[0090] In the related art, the obstacle trajectory prediction module and the ego-vehicle trajectory planning module may be processed as independent modules. The trajectory prediction module may predict the future motion states of the obstacles around the vehicle according to the perception information, and the ego-vehicle trajectory planning module then performs trajectory planning on the ego-vehicle according to the predicted future motion states of the obstacles and the global path planning to determine the best driving route of the ego-vehicle. However, this processing method of separating prediction and planning, when facing a complex dynamic environment, due to the independence of trajectory prediction and trajectory planning and the lack of real-time interaction feedback, may lead to inconsistencies between the prediction results and the planning decisions, thereby affecting the overall reaction speed and decision-making accuracy of the system. Secondly, most of the existing trajectory planning methods are based on static or semi-static environments, and it is difficult to effectively cope with the behavior changes of other dynamic traffic participants, and problems such as inaccurate obstacle avoidance, uneven routes, or unstable decisions may occur. In addition, with the increasing complexity of urban road scenarios and highway scenarios, the traditional separation processing method shows certain limitations in terms of computing resources and time costs, and it is difficult to meet the real-time requirements of autonomous driving systems. Therefore, there is an urgent need for a new trajectory planning method that can achieve efficient and robust trajectory planning in a dynamic environment.

[0091] Figure 3 is a flowchart showing a trajectory planning method provided by an embodiment of the present disclosure. This trajectory planning method can be executed by Figure 1 the vehicle and / or server shown. As Figure 3 shown, the trajectory planning method of this embodiment may include the following steps S310 to S330.

[0092] Step S310: Obtain map element information within the target area, as well as the current position information and historical position information of the target object, based on the environmental perception data of the target area.

[0093] Among them, the target area may be an area where the target vehicle located with the trajectory to be planned is located. For example, the target area may be the maximum area detected by the sensors of the target vehicle, or the target area may be an area within a preset range around the position of the target vehicle as the center. The target objects within the target area may include the target vehicle and obstacles, and the obstacles may be one or more. Optionally, the obstacles may be dynamic obstacles, such as other vehicles or pedestrians, etc., or may be static obstacles, such as signboards, cone barrels, etc.

[0094] The map element information may be the representation of the static element information within the target area in a high-precision map. For example, the map element information may include road information, lane information, intersection information, traffic sign information, marking line information, arrow information, etc. in the high-precision map.

[0095] The current position information of the target object may be the position information at the current moment, for example, it may be the position information determined based on the environmental perception data at the current moment.

[0096] The historical position information of the target object may include multiple position information determined based on the environmental perception data at multiple historical moments. For example, it may be the movement trajectory of the target object within a preset time (such as 3 seconds) before. For a static obstacle, the movement trajectory may be a position point, and for a dynamic obstacle, the movement trajectory may include a straight line or curve composed of multiple position points.

[0097] In some examples, the above environmental perception data may be data obtained by detecting the surrounding environment of the vehicle through sensors during vehicle driving. For example, the environmental perception data may include image data collected based on a vision sensor and / or point cloud data collected based on a radar. The environmental perception data can be input into a pre-generated neural network model to obtain the map element information within the target area, as well as the current position information and historical position information of the target object output by the neural network model.

[0098] Step S320: Input the current location information, historical location information, and map element information into the feature extraction and fusion model to obtain the first fusion feature output by the feature extraction and fusion model.

[0099] Among them, the first fusion feature may include interaction features representing the mutual influence between different target objects and between the target object and the map element.

[0100] Exemplarily, the feature extraction and fusion model may perform feature extraction on the input current location information, historical location information, and map element information of target objects including the target vehicle and obstacles, and output the first fusion feature that fuses the information related to the target vehicle, obstacles, and map elements. The interaction features in the first fusion feature show the mutual influence between different target objects and between the target object and the map element, so as to form an overall understanding of the dynamic environment based on the first fusion feature. Optionally, the first fusion feature may be a high-dimensional fusion feature, such as 128-dimensional or 256-dimensional.

[0101] Step S330: Generate the target planning trajectory of the target vehicle according to the first fusion feature.

[0102] Exemplarily, the first fusion feature may be input into a pre-generated trajectory prediction and planning model to obtain the target planning model.

[0103] Using the method of the above steps S310 to S330, generate the first fusion feature representing the mutual influence between different target objects and between the target object and the map element, and generate the target planning trajectory of the target vehicle according to the first fusion feature, realizing the interactive feedback among the target vehicle, obstacles, and map elements, and being able to achieve efficient and robust trajectory planning in a dynamic environment, thereby improving the safety, stability, and driving efficiency of autonomous vehicles.

[0104] In some embodiments of the present disclosure, step S330 may include the following steps S331 and S332:

[0105] Step S331: Input the first fusion feature into a pre-generated trajectory prediction and planning model to obtain the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle output by the trajectory prediction and planning model.

[0106] In some examples, the trajectory prediction and planning model may include a network model using a multi-layer perceptron (MLP, Multilayer Perceptron). Through the MLP, the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle can be predicted from the first fusion feature. The obstacle prediction trajectory includes the predicted position of the obstacle at a future moment, which can provide an input basis for the further planning and obstacle avoidance decision of the target vehicle.

[0107] Step S332: Determine the target planned trajectory of the target vehicle according to the initial planned trajectory and the obstacle prediction trajectory.

[0108] Among them, the target planned trajectory and the initial planned trajectory may be the same or different. For example, the initial planned trajectory can be used as the target planned trajectory, or the target planned trajectory can be obtained by adjusting the initial planned trajectory according to the obstacle prediction trajectory.

[0109] In some examples, the collision probability of the target vehicle colliding with the obstacle can be predicted according to the initial planned trajectory and the obstacle prediction trajectory; when the collision probability is greater than or equal to the preset collision probability threshold, update the initial planned trajectory according to the obstacle prediction trajectory, and use the updated initial planned trajectory as the target planned trajectory to reduce the collision probability; or, when the collision probability is less than the preset collision probability threshold, use the initial planned trajectory as the target planned trajectory.

[0110] Among them, the collision probability can be used to indicate the probability of the target vehicle and the obstacle colliding at a future moment. The preset collision probability threshold can be any preset value.

[0111] In other examples, the safety of the target planned trajectory can also be evaluated. The evaluation indicators can include collision probability, trajectory curvature, trajectory comfort, etc. When the evaluation result meets the preset conditions, the target vehicle can be controlled to drive according to the target planned trajectory.

[0112] In this way, the interactive feedback between the ego-vehicle trajectory planning and the obstacle trajectory prediction can be realized, and efficient and robust trajectory planning can be achieved in a dynamic environment, thereby improving the safety, smoothness and driving efficiency of autonomous vehicles.

[0113] Figure 4 It is a schematic structural diagram of a feature extraction and fusion model and a trajectory prediction and planning model provided by an embodiment of the present disclosure. As Figure 4 shown, the feature extraction and fusion model 41 may include a target encoder 411 and a target decoder 412; the trajectory prediction and planning model 42 may include an ego-vehicle trajectory planning head 421 and an obstacle trajectory prediction head 422. The ego-vehicle trajectory planning head may also be referred to as an ego-vehicle trajectory planning module, and the obstacle trajectory prediction head may also be referred to as an obstacle trajectory prediction module.

[0114] In some examples, the feature extraction and fusion model can be used to generate a first fusion feature in the following manner: based on a target encoder, feature extraction is performed on map element information, current location information, and historical location information to obtain map information features, historical trajectory features of the target object, and first predicted trajectory features; based on a target decoder, feature fusion is performed on the map information features, historical trajectory features of the target object, and first predicted trajectory features to obtain a first fusion feature.

[0115] In some examples, the target encoder 411 can be any pre-trained encoder. For example, it can be a PnP Encoder (Prediction and Planning Encoder). Optionally, the target encoder can include a convolutional unit and an attention mechanism unit to perform feature extraction on the input map element information, current location information, and historical location information to obtain high-dimensional (e.g., 128-dimensional) map information features, historical trajectory features of the target object, and first predicted trajectory features.

[0116] Exemplarily, the above-mentioned map element information input to the target encoder can be represented as (B, M, P, 2), where B represents the data batch, M represents the number of map elements, P represents the number of key points of the map element, and 2 represents that each key point coordinate is represented in the form of two-dimensional coordinates. The above-mentioned current location information of the target object can be represented as (B, N, 1, 2), where B represents the data batch, N represents the number of target objects, 1 represents the current frame or current moment, and 2 represents that the current location coordinate of the target object is represented in the form of two-dimensional coordinates. The above-mentioned historical location information of the target object can be represented as (B, N, HT, 2), where B represents the data batch, N represents the number of target objects, HT represents the number of historical moments or historical frames, and 2 represents that the historical location coordinate of the target object is represented in the form of two-dimensional coordinates. It should be noted that the key point coordinates of the above-mentioned map elements, the current location coordinates of the target object, and the historical location coordinates can also be represented in the form of three-dimensional coordinates.

[0117] The above-mentioned map information features output by the target encoder can be represented as (B*M*K, 1, M, 128), where B represents the data batch, M represents the number of map elements, K represents the maximum number of preset prediction trajectories, such as 6 or 8, and 128 represents the encoded dimension of 128 dimensions. The first prediction trajectory feature of the above-mentioned target object can be represented as (B, K, N, 128), where B represents the data batch, K represents the maximum number of preset prediction trajectories, N represents the number of target objects, and 128 represents the encoded dimension of 128 dimensions. The historical trajectory feature of the above-mentioned target object can be represented as (B, N, HT, 128), where B represents the data batch, N represents the number of target objects, HT represents the number of historical moments or historical frames, and 128 represents the encoded dimension of 128 dimensions.

[0118] In this way, through the target encoder, high-dimensional feature representations of map information, the current position information, and the historical position information of the target object can be obtained, and more abundant features can be extracted to improve the accuracy of trajectory prediction and trajectory planning.

[0119] In some embodiments, the target decoder 412 may be a decoder based on a cross-attention mechanism. Refer to Figure 4 , the target decoder 412 may include a first cross-attention unit 4121, a second cross-attention unit 4122, and a third cross-attention unit 4123; the method of obtaining the first fusion feature by fusing the map information features, the prediction trajectory features, and the historical trajectory features of the target object by the target decoder may include the following steps S11 to step S13:

[0120] Step S11, fusing the first prediction trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain a second prediction trajectory feature.

[0121] In one implementation, the first prediction trajectory feature may be used as the first query feature (Qurey), and the historical trajectory feature may be used as the first key feature (Key) and the first value feature (Value); the first query feature, the first key feature, and the first value feature are input into the first cross-attention unit to generate a second prediction trajectory feature.

[0122] In another implementation, the first query feature may be generated according to the first prediction trajectory feature and the relative time dimension interaction feature of the target object; the first key feature and the first value feature may be generated according to the historical trajectory feature and the relative time dimension interaction feature of the target object; the first query feature, the first key feature, and the first value feature are input into the first cross-attention unit to generate a second prediction trajectory feature.

[0123] Among them, the interaction feature of the target object with respect to the time dimension can be obtained by performing embedding encoding and fusion on the current position information and historical position information of the target object, and is used to represent the interaction feature of the target object in the time dimension. As Figure 7 shown, the current position information and historical position information of the target object can be respectively subjected to Fourier embedding encoding, and then the encoded vectors are interactively fused based on a pre-trained second network to obtain the interaction feature of the target object with respect to the time dimension. Among them, the second network may include a linear transformation layer (linear). In this way, the interaction influence of the historical moment of the target object on the current moment can be represented by the interaction feature of the target object with respect to the time dimension, so as to improve the accuracy of trajectory planning and trajectory prediction in complex scenarios.

[0124] Step S12: Based on the cross-attention mechanism, fuse the second predicted trajectory feature and the map information feature to obtain a third predicted trajectory feature.

[0125] In one implementation, the second predicted trajectory feature can be used as the second query feature (Qurey), and the map information feature can be used as the second key feature (Key) and the second value feature (Value); the second query feature, the second key feature, and the second value feature are input into the second cross-attention unit to generate a third predicted trajectory feature.

[0126] In another implementation, the second query feature can be generated according to the second predicted trajectory feature and the relative position interaction feature of the target object; the second key feature and the second value feature can be generated according to the map information feature and the relative position interaction feature of the target object; the second query feature, the second key feature, and the second value feature are input into the second cross-attention unit to generate a third predicted trajectory feature.

[0127] Among them, the relative position interaction feature of the target object can be obtained by performing embedding encoding and fusion on the current position information and historical position information of the target object, and is used to represent the mutual influence between target objects. Continuing as Figure 7 shown, the current position information and historical position information of the target object can be respectively subjected to Fourier embedding encoding, and then the encoded vectors are interactively fused based on a pre-trained third network to obtain the relative position interaction feature of the target object. Among them, the third network may include a linear transformation layer (linear). In this way, the interaction influence between multiple target objects can be represented by the relative position interaction feature of the target object, so as to improve the accuracy of trajectory planning and trajectory prediction in complex scenarios.

[0128] Step S13: Based on the cross-attention mechanism, fuse the third predicted trajectory feature and the historical trajectory feature to obtain a first fusion feature.

[0129] In one implementation, the fourth predicted trajectory feature can be used as the third query feature (Query), and the map information feature can be used as the third key feature (Key) and the third value feature (Value); the third query feature, the third key feature, and the third value feature are input into the third cross-attention unit to generate the fourth predicted trajectory feature.

[0130] In another implementation, the third query feature can be generated based on the third predicted trajectory feature and the relative map interaction feature of the target object; the third key feature and the third value feature can be generated based on the historical trajectory feature and the relative map interaction feature of the target object; the third query feature, the third key feature, and the third value feature are input into the third cross-attention unit to generate the first fusion feature.

[0131] Among them, the relative map interaction feature of the target object is a feature obtained by embedding encoding and fusing based on the current position information, historical position information, and map element information of the target object, and is used to represent the mutual influence between the target object and the map element.

[0132] Continue as Figure 7 shown, the Fourier embedding encoding can be performed on the current position information, historical position information, and map element information of the target object respectively, and then the encoded vectors are interactively fused based on the pre-trained fourth network to obtain the relative position interaction feature of the target object. Among them, the fourth network can include a linear transformation layer (linear). In this way, the interaction influence between the target object and the map element can be represented by the relative map interaction feature of the target object, so as to improve the accuracy of trajectory planning and trajectory prediction in complex scenarios.

[0133] By adopting the above method, the target decoder performs feature fusion on the map information feature, the predicted trajectory feature of the target object, and the historical trajectory feature based on three sequentially connected cross-attention units. The first fusion feature obtained contains the mutual influence between the target vehicle, the obstacle, and the map element, and also contains the interaction influence between the target vehicle and the obstacle in the time dimension. Therefore, the overall understanding of the dynamic environment can be formed through the first fusion feature, and the accuracy of trajectory planning and trajectory prediction in complex scenarios is improved.

[0134] Figure 5 is a schematic structural diagram of another feature extraction and fusion model and trajectory prediction and planning model provided by an embodiment of the present disclosure. Refer to Figure 5, the trajectory prediction and planning model 42 may include a self-vehicle trajectory planning module 421 and an obstacle trajectory prediction module 422; the trajectory prediction and planning model may generate an initial planned trajectory of the target vehicle and an obstacle prediction trajectory of the obstacle in the following manner: the self-vehicle trajectory planning module may generate the initial planned trajectory of the target vehicle according to the first fused feature; the obstacle trajectory prediction module may generate the obstacle prediction trajectory of the obstacle according to the first fused feature, the initial planned trajectory, and the current position information of the obstacle.

[0135] In some examples, the first fused feature may be input into the self-vehicle trajectory planning module, and the self-vehicle trajectory planning module may perform trajectory planning according to the first fused feature to generate the initial planned trajectory of the target vehicle.

[0136] Continue to refer to Figure 5 , the self-vehicle trajectory planning module 421 may include a first self-attention unit 4211 and a first neural network unit 4212, and the first neural network unit may include a multi-layer perceptron MLP. The first trajectory feature of the target vehicle may be extracted from the first fused feature based on the first self-attention unit; the first trajectory feature is input into the first neural network unit to obtain the initial planned trajectory of the target vehicle.

[0137] In some examples, the input of the first neural network unit may further include the current state information of the target vehicle, such as the current position, speed, acceleration, direction angle, etc. of the target vehicle. In this way, based on the first fused feature and the current state information of the target vehicle, an initial planned trajectory that meets the requirements of obstacle avoidance, safety, and path smoothness is generated, providing an optimal driving path for the target vehicle to avoid collisions with surrounding obstacles, while meeting the requirements of path smoothness and driving efficiency.

[0138] In some examples, the first fused feature, the initial planned trajectory, and the current position information of the obstacle may be input into the obstacle trajectory prediction module, and the obstacle trajectory prediction module performs trajectory prediction according to the first fused feature, the initial planned trajectory, and the current position information of the obstacle to generate the obstacle prediction trajectory of the obstacle.

[0139] Continue to refer to Figure 5 , the obstacle trajectory prediction module 422 may include a fourth cross-attention unit 4221, a second self-attention unit 4222, and a second neural network unit 4223, and the first neural network unit may include a multi-layer perceptron MLP.

[0140] In this example, the first fusion feature, the initial planned trajectory, and the current position information of the obstacle can be fused based on the fourth cross-attention unit to obtain a second fusion feature; the trajectory feature of the obstacle can be extracted from the second fusion feature based on the second self-attention unit; the trajectory feature is input into the second neural network unit to obtain the predicted trajectory of the obstacle.

[0141] In one implementation, the first fusion feature can be used as the fourth query feature; the initial planned trajectory can be used as the fourth key feature and the fourth value feature; the fourth query feature, the fourth key feature, and the fourth value feature are input into the fourth cross-attention unit to generate a second fusion feature.

[0142] In another implementation, the method for fusing the first fusion feature, the initial planned trajectory, and the current position information of the obstacle based on the fourth cross-attention unit to obtain a second fusion feature may include: after performing embedding encoding and fusion on the current position information of the obstacle and the initial planned trajectory, an interaction feature of the obstacle relative to the target vehicle is obtained; according to the first fusion feature and the interaction feature of the obstacle relative to the target vehicle, a fourth query feature is generated; according to the initial planned trajectory and the interaction feature of the obstacle relative to the target vehicle, a fourth key feature and a fourth value feature are generated; the fourth query feature, the fourth key feature, and the fourth value feature are input into the fourth cross-attention unit to generate a second fusion feature.

[0143] Exemplarily, Fourier Embedding encoding can be performed on the current position information of the obstacle and the initial planned trajectory respectively, and then the encoded vectors are interactively fused based on a pre-trained first network to obtain an interaction feature of the obstacle relative to the target vehicle. Among them, the first network may include a linear transformation layer. In this way, the fusion extraction of the interaction features between the obstacle and the initial planned trajectory of the target vehicle can be realized, so as to perform interference analysis and dynamic behavior prediction on the obstacle in combination with the future driving trajectory of the target vehicle, and obtain a more accurate predicted trajectory of the obstacle.

[0144] In some embodiments of the present disclosure, the target planned trajectory generated by using the trajectory planning method in the above embodiments can be used to control a vehicle. For example, the vehicle can be controlled to travel according to the generated target planned trajectory. According to the actual vehicle test data, the passing rate at intersections is greater than 90%, and the takeover rate is greater than 15 kilometers per takeover. Especially for scenarios such as passing through intersections and roads bifurcating into two, the problem that the ego vehicle makes unreasonable lane selections resulting in serpentine driving, that is, the vehicle sways left and right during driving and cannot maintain a straight line, is significantly improved. The data-driven trajectory planning method can provide a smoother planned trajectory and avoid the problem of serpentine driving. In addition, for the abnormal degradation of the autonomous driving function caused by the short perception lane in front of an intersection, the missing part can be complemented based on the above method to avoid the abnormal degradation or exit of the autonomous driving function.

[0145] Figure 7 is a schematic flowchart of a trajectory display method provided by an embodiment of the present disclosure. This trajectory display method can be executed by Figure 1 the vehicle and / or server shown. As Figure 7 shown, the trajectory planning method of this embodiment can include:

[0146] Step S710, determining the target planned trajectory of the target vehicle according to the environmental perception data of the target area.

[0147] Step S720, displaying the target planned trajectory.

[0148] Exemplarily, the target planned trajectory can be displayed through the display device of the vehicle and / or server. For example, the target planned trajectory can be displayed on the in-vehicle display device at the vehicle end, or can also be displayed through devices such as a client (such as a mobile phone).

[0149] Wherein, the target planned trajectory is a trajectory generated based on the first fusion feature output by the feature extraction and fusion model. The environmental perception data is used to obtain the map element information in the target area and the current position information and historical position information of the target object. The target area is an area including the position where the target vehicle is located. The target object includes the target vehicle and obstacles. The feature extraction and fusion model is used to output the first fusion feature according to the input current position information, historical position information, and map element information.

[0150] The generation method of this target planned trajectory can refer to the description in the foregoing embodiments of the present disclosure and will not be elaborated here.

[0151] In this way, the interactive feedback among the target vehicle, obstacles, and map elements can be realized, and efficient and robust trajectory planning can be achieved in a dynamic environment, generating a target planned trajectory with higher robustness and presenting the target planned trajectory to the user, facilitating the user to obtain the future driving direction of the vehicle.

[0152] In some embodiments, the obstacle prediction trajectory of the obstacle and / or the relevant information of the map element may also be displayed.

[0153] Figure 8 This is a schematic diagram showing a target planned trajectory provided by an embodiment of the present disclosure. As Figure 8 shown, in a coordinate system with the vehicle center as the coordinate origin and the current driving direction of the vehicle as the x-axis direction (which may also be referred to as the view coordinate system, View Coordinate System), the target area may exemplarily be the rectangular area in the figure, that is, the four corner coordinates of the target area are respectively (100, 40), (100, -40), (-40, -40), (-40, 40), and the coordinate unit is meters. Taking the target vehicle as the ego vehicle, as shown in the legend, the blue line is the target planned trajectory of the ego vehicle, for example, the target planned trajectory within the next 6 seconds; the red line is the historical trajectory of the ego vehicle, for example, the historical trajectory of the ego vehicle within the previous 2 seconds; the green line is the historical trajectory of the obstacle, for example, the historical trajectory of the obstacle within the previous 2 seconds; the brown line is the road boundary line output by environmental perception; the black line is the left and right lane lines output by environmental perception; the sky-blue line is other lines output by environmental perception, such as the diversion line, etc. Among them, the diversion line usually appears at intersections that are too wide, irregular, or have complex driving conditions, ramps of cloverleaf interchanges, or other special locations.

[0154] It should be noted that in this embodiment, one or more of the above lines may be displayed, and other information may also be displayed. For example, only the target planned trajectory of the ego vehicle may be displayed, or the target planned trajectory and the historical trajectory of the ego vehicle may be displayed, or the obstacle prediction trajectory and the relevant information of other map elements may be displayed.

[0155] Figure 9 This is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As Figure 9 shown, the electronic device 1000 may include a memory 1010 and a processor 1020. The memory 1010 may be used to store computer instructions, and the processor 1020 may be used to call the computer instructions from the memory 1010 to execute all or part of the steps of any method in the foregoing embodiments of the present disclosure. Among them, the processor may be one or more, and the one or more processors may execute the instructions alone or jointly. The memory may also be one or more, and the one or more memories may store the above computer instructions alone or jointly. Optionally, the electronic device may be Figure 1 the server and / or the vehicle in

[0156] An embodiment of the present disclosure also provides a vehicle, which may include a memory and a processor. The memory may be used to store computer instructions, and the processor may be used to call the computer instructions from the memory to execute all or part of the steps of any of the methods in the foregoing embodiments of the present disclosure. Wherein, the processor may be one or more, and the one or more processors may execute the instructions alone or jointly. The memory may also be one or more, and the one or more memories may store the foregoing computer instructions alone or jointly.

[0157] The vehicle in the foregoing embodiments of the present disclosure may be an electric vehicle, a hybrid vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle may be an autonomous vehicle or a non-autonomous vehicle. Exemplarily, the vehicle provided in this embodiment may be Figure 1 or Figure 2 the vehicle shown.

[0158] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any of the methods in the foregoing embodiments of the present disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and it may also be a transitory storage medium.

[0159] An embodiment of the present disclosure also provides a chip, which may include a processing unit, and the processing unit may be used to execute all or part of the steps of any of the methods in the foregoing embodiments of the present disclosure. The chip may be in the form of an application-specific integrated circuit (ASIC), a system-on-chip (SOC), a field-programmable gate array (FPGA), etc. This embodiment does not limit this. Optionally, the chip may further include a storage unit, and the storage unit may be used to store computer instructions. The processing unit may be used to call the computer instructions from the storage unit to execute all or part of the steps of any of the methods in the foregoing embodiments of the present disclosure.

[0160] An embodiment of the present disclosure also provides a computer program product, which may include a computer program. When the computer program is executed by a processor, it may implement any of the methods in the foregoing embodiments of the present disclosure.

[0161] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium, on which computer-readable program instructions for causing a processor to implement any of the methods in the foregoing embodiments of the present disclosure are uploaded.

[0162] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example--but not limited to--an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0163] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0164] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, which may include object - oriented programming languages - such as Smalltalk, C++, etc., and conventional procedural programming languages - such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0165] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0166] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine, such that when these instructions are executed by the processor of the computer or other programmable data - processing apparatus, a device is produced that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0168] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions. It should be noted that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

[0169] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary skill in the technical field to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A trajectory planning method, characterized in that, The method includes: Obtaining map element information, the current position information, and the historical position information of a target object within the target area according to the environmental perception data of the target area; wherein, the target area is an area containing the position of a target vehicle for which a trajectory is to be planned, and the target object includes the target vehicle and obstacles; Performing feature extraction on the current position information, the historical position information, and the map element information based on a target encoder in a feature extraction and fusion model to obtain map information features, historical trajectory features of the target object, and first predicted trajectory features; Performing feature fusion on the map information features, the historical trajectory features of the target object, and the first predicted trajectory features based on a target decoder in the feature extraction and fusion model to obtain first fusion features output by the feature extraction and fusion model; wherein, the first fusion features include interaction features characterizing the mutual influence between different target objects and between the target object and map elements; Generating a target planning trajectory for the target vehicle according to the first fusion features.

2. The method according to claim 1, characterized in that The target decoder is a decoder based on a cross-attention mechanism; the performing feature fusion on the map information features, the historical trajectory features of the target object, and the first predicted trajectory features based on the target decoder in the feature extraction and fusion model to obtain first fusion features output by the feature extraction and fusion model includes: Fusing the first predicted trajectory features and the historical trajectory features based on the cross-attention mechanism to obtain second predicted trajectory features; Fusing the second predicted trajectory features and the map information features based on the cross-attention mechanism to obtain third predicted trajectory features; Fusing the third predicted trajectory features and the historical trajectory features based on the cross-attention mechanism to obtain the first fusion features.

3. The method according to claim 2, wherein The target decoder includes a first cross-attention unit; the fusing the first predicted trajectory features and the historical trajectory features based on the cross-attention mechanism to obtain second predicted trajectory features includes: Generating a first query feature according to the first predicted trajectory features and the relative time dimension interaction features of the target object; the relative time dimension interaction features of the target object are obtained by embedding encoding and fusing the current position information and the historical position information of the target object and are used to characterize the interaction features of the target object in the time dimension; Generating a first key feature and a first value feature according to the historical trajectory features and the relative time dimension interaction features of the target object; Inputting the first query feature, the first key feature, and the first value feature into the first cross-attention unit to generate the second predicted trajectory features.

4. The method according to claim 2, wherein The target decoder includes a second cross-attention unit; The fusing the second predicted trajectory features and the map information features based on the cross-attention mechanism to obtain third predicted trajectory features includes: Generate a second query feature according to the second predicted trajectory feature and the relative position interaction feature of the target object; the relative position interaction feature of the target object is a feature obtained by embedding encoding and fusing the current position information and historical position information of the target object, and is used to characterize the mutual influence between the target objects. Generate a second key feature and a second value feature according to the map information feature and the relative position interaction feature of the target object. Input the second query feature, the second key feature, and the second value feature into a second cross-attention unit to generate the third predicted trajectory feature.

5. The method according to claim 2, wherein The target decoder includes a third cross-attention unit; the fusion of the third predicted trajectory feature and the historical trajectory feature based on the cross-attention mechanism to obtain the first fusion feature includes: Generate a third query feature according to the third predicted trajectory feature and the relative map interaction feature of the target object; the relative map interaction feature of the target object is a feature obtained by embedding encoding and fusing the current position information, historical position information, and map element information of the target object, and is used to characterize the mutual influence between the target object and the map element. Generate a third key feature and a third value feature according to the historical trajectory feature and the relative map interaction feature of the target object. Input the third query feature, the third key feature, and the third value feature into a third cross-attention unit to generate the first fusion feature.

6. The method according to any one of claims 1 to 5, characterized in that, The generation of the target planning trajectory of the target vehicle according to the first fusion feature includes: Input the first fusion feature into a pre-generated trajectory prediction and planning model to obtain the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle output by the trajectory prediction and planning model. Determine the target planning trajectory of the target vehicle according to the initial planning trajectory and the obstacle prediction trajectory.

7. The method according to claim 6, wherein The trajectory prediction and planning model includes a self-vehicle trajectory planning module and an obstacle trajectory prediction module; the trajectory prediction and planning model generates the initial planning trajectory of the target vehicle and the obstacle prediction trajectory of the obstacle in the following manner: The self-vehicle trajectory planning module generates the initial planning trajectory of the target vehicle according to the first fusion feature. The obstacle trajectory prediction module generates the obstacle prediction trajectory of the obstacle according to the first fusion feature, the initial planning trajectory, and the current position information of the obstacle.

8. The method according to claim 7, wherein The self-vehicle trajectory planning module includes a first self-attention unit and a first neural network unit. The self-vehicle trajectory planning module generates the initial planning trajectory of the target vehicle according to the first fusion feature, including: Extract the first trajectory feature of the target vehicle from the first fusion feature based on the first self-attention unit. Input the first trajectory feature and the current state information of the target vehicle into the first neural network unit to obtain the initial planning trajectory of the target vehicle.

9. The method according to claim 7, characterized in that, The obstacle trajectory prediction module includes a fourth cross-attention unit, a second self-attention unit, and a second neural network unit; the obstacle trajectory prediction module generates an obstacle prediction trajectory of the obstacle according to the first fusion feature, the initial planned trajectory, and the current position information of the obstacle, including: Fusing the first fusion feature, the initial planned trajectory, and the current position information of the obstacle based on the fourth cross-attention unit to obtain a second fusion feature; Extracting the trajectory feature of the obstacle from the second fusion feature based on the second self-attention unit; Inputting the trajectory feature into the second neural network unit to obtain the obstacle prediction trajectory of the obstacle.

10. The method according to claim 9, characterized in that, The fusing the first fusion feature, the initial planned trajectory, and the current position information of the obstacle based on the fourth cross-attention unit to obtain a second fusion feature includes: After performing embedding encoding and fusion according to the current position information of the obstacle and the initial planned trajectory of the target vehicle, obtaining an obstacle relative target vehicle interaction feature; Generating a fourth query feature according to the first fusion feature and the obstacle relative target vehicle interaction feature; Generating a fourth key feature and a fourth value feature according to the initial planned trajectory and the obstacle relative target vehicle interaction feature; Inputting the fourth query feature, the fourth key feature, and the fourth value feature into the fourth cross-attention unit to generate the second fusion feature.

11. The method according to claim 6, characterized in that, The determining the target planned trajectory of the target vehicle according to the initial planned trajectory and the obstacle prediction trajectory includes: Predicting a collision probability of the target vehicle colliding with the obstacle according to the initial planned trajectory and the obstacle prediction trajectory; When the collision probability is greater than or equal to a preset collision probability threshold, updating the initial planned trajectory according to the obstacle prediction trajectory, and using the updated initial planned trajectory as the target planned trajectory to reduce the collision probability; When the collision probability is less than the preset collision probability threshold, using the initial planned trajectory as the target planned trajectory.

12. A trajectory display method, characterized in that, The method includes: Determining a target planned trajectory of a target vehicle according to environmental perception data of a target area; Displaying the target planned trajectory; Among them, the target planned trajectory is a trajectory generated based on the first fusion feature output by the feature extraction and fusion model. The environmental perception data is used to obtain map element information, the current position information, and the historical position information of the target object in the target area. The target area is an area containing the position of the target vehicle. The target object includes the target vehicle and obstacles. The feature extraction and fusion model includes a target encoder and a target decoder. The target encoder is used to extract features from the current position information, the historical position information, and the map element information to obtain map information features, historical trajectory features of the target object, and a first predicted trajectory feature. The target decoder is used to perform feature fusion on the map information features, the historical trajectory features of the target object, and the first predicted trajectory feature to obtain the first fusion feature. The first fusion feature includes interaction features representing the mutual influence between different target objects and between the target object and the map element.

13. An electronic device, characterized in that, It includes a memory and a processor. The memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of claims 1 to 12.

14. A vehicle, characterized in that, It includes a memory and a processor. The memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A chip, characterized in that, The chip includes a processing unit, and the processing unit is used to execute the method according to any one of claims 1 to 12.

Images