Driving track determination method, vehicle and storage medium

Abstract

The invention provides a driving track determination method, a vehicle and a storage medium, and relates to the technical field of automobiles, and the method comprises the following steps: obtaining observation state information of an environment where the vehicle is located; based on the observation state information, environment coding features and navigation coding features are determined, the environment coding features are used for representing features of the environment, the features of the environment comprise lane features, and the navigation coding features are used for representing a driving intention corresponding to the observation state information; based on the environment coding feature and the navigation coding feature, a target track is determined, and the target track is used for indicating the vehicle to run according to a lane corresponding to the target track. According to the scheme, when the target trajectory is predicted, the environment coding features are limited through the navigation coding features representing the driving intention, so that various environments encountered in the form process can be expressed through the vector form, the generalization of the predicted form trajectory is improved, and the form trajectory prediction can meet various vehicle driving conditions.

Description

Technical Field

[0001] The present application belongs to the field of automobile technology, and in particular relates to a method for determining a driving trajectory, a vehicle, and a storage medium. Background Art

[0002] With the development of automobile technology, autonomous driving technology is becoming more and more mature. Driving trajectory determination is an important part of autonomous driving technology. Among them, driving trajectory determination generally plans the driving trajectory of the vehicle in local space and time based on local environmental information, upper-level decision-making tasks and real-time posture information of the vehicle body, while satisfying certain kinematic constraints.

[0003] In the related art, the vehicle's driving trajectory is generally determined based on a comprehensive solution (model-based) of search (search-based) and numerical optimization (optimization-based). That is, firstly, the initial driving trajectory is determined by a search method, then the initial driving trajectory is numerically optimized to obtain a precise driving trajectory, and finally the precise driving trajectory is speed optimized to obtain the final driving trajectory. However, when the above method is used to determine the vehicle's driving trajectory, the initial trajectory can only be determined from a limited number of driving trajectories based on searching for the initial driving trajectory in a preset database. Therefore, the generalization is weak and it cannot cope with various vehicle driving conditions. Summary of the invention

[0004] The purpose of the present application is to provide a method for determining a driving trajectory, a vehicle and a storage medium, aiming to solve the problem that the traditional trajectory determination process has weak generalization and cannot cope with various vehicle driving conditions.

[0005] A first aspect of an embodiment of the present application provides a method for determining a driving trajectory, the method comprising:

[0006] Obtain observation status information of the vehicle's environment;

[0007] Based on the observation state information, determine an environment coding feature and a navigation coding feature, the environment coding feature is used to represent a feature of the environment, the feature of the environment includes a lane feature, and the navigation coding feature is used to represent a driving intention corresponding to the observation state information;

[0008] A target trajectory is determined based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0009] In some embodiments, determining the environment coding feature and the navigation coding feature based on the observation state information includes:

[0010] Extracting obstacle features, lane features and navigation features from the observed state information;

[0011] The obstacle feature and the lane feature are integrated to obtain the environment coding feature;

[0012] The navigation feature is processed to obtain the navigation coding feature.

[0013] In some embodiments, the environment includes obstacles and lanes, the observation state information of the environment includes obstacle information, lane information and navigation information in an area corresponding to the environment, and extracting obstacle features, lane features and navigation features from the observation state information includes:

[0014] Create a target coordinate system with the current position of the vehicle as the origin and the direction of the vehicle head as the positive direction of the X-axis;

[0015] Determining the obstacle feature based on the vector representation of the obstacle information in the target coordinate system;

[0016] Determining the lane characteristics based on the vectorized representation of the lane information in the target coordinate system;

[0017] The navigation feature is determined based on the vectorized representation of the navigation information in the target coordinate system.

[0018] In some embodiments, the determining the target trajectory based on the environment coding feature and the navigation coding feature includes:

[0019] Concatenating the navigation coding feature and the environment coding feature to obtain a context vector;

[0020] Based on the context vector, the target trajectory is determined.

[0021] In some embodiments, the method further comprises:

[0022] Acquiring a driving state of a vehicle, and determining a predicted speed of the vehicle based on the driving state;

[0023] Based on the predicted speed and the target trajectory, the vehicle is controlled to travel.

[0024] In some embodiments, the method further comprises:

[0025] Acquire sample data, wherein the sample data includes sample environment information and a true value trajectory of the vehicle;

[0026] Constructing an initial trajectory determination model, the initial trajectory determination model comprising an obstacle feature processing network layer for extracting features from obstacle information in the observation state information, a lane feature processing network layer for extracting features from lane information in the observation state information, a navigation feature processing network layer for extracting features from navigation information in the observation state information, a fusion network layer for fusing obstacle features and lane features, and a trajectory determination network layer for determining a driving trajectory based on a coding vector and navigation coding features, wherein outputs of the obstacle feature processing network layer and the lane feature processing network layer are connected to inputs of the fusion network layer, and outputs of the navigation feature processing network layer and the fusion network layer are connected to inputs of the trajectory determination network layer;

[0027] Based on the sample data, model training is performed on the initial trajectory determination model to obtain the trajectory determination model.

[0028] In some embodiments, the performing model training on the initial trajectory determination model based on the sample data to obtain the trajectory determination model includes:

[0029] Determine the imitation loss function;

[0030] Based on the imitation loss function, determining a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data;

[0031] Based on the first loss value, the parameters in the initial trajectory determination model are adjusted until a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

[0032] In some embodiments, the sample data further includes the driving speed of the vehicle, and the method further includes:

[0033] Determine the speed loss function;

[0034] Based on the speed loss function, determining a second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data;

[0035] Based on the second loss value, the parameters in the initial trajectory determination model are adjusted until the second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data is less than a second preset threshold, and the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

[0036] A second aspect of an embodiment of the present application provides a device for determining a driving trajectory, the device comprising:

[0037] A first acquisition unit, used to acquire observation status information of the environment where the vehicle is located;

[0038] A first determining unit is used to determine an environment coding feature and a navigation coding feature based on the observation state information, wherein the environment coding feature is used to represent a feature of the environment, the feature of the environment includes a lane feature, and the navigation coding feature is used to represent a driving intention corresponding to the observation state information;

[0039] The second determination unit is used to determine a target trajectory based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0040] In some embodiments, the first determination unit is used to extract obstacle features, lane features and navigation features from the observation state information; fuse the obstacle features and the lane features to obtain the environment coding features; and process the navigation features to obtain the navigation coding features.

[0041] In some embodiments, the environment includes obstacles and lanes, and the observation state information of the environment includes obstacle information, lane information and navigation information in the area corresponding to the environment. The first determination unit is used to create a target coordinate system with the current position of the vehicle as the origin and the direction of the front of the vehicle as the positive direction of the X-axis; determine the obstacle characteristics based on the vectorized representation of the obstacle information in the target coordinate system; determine the lane characteristics based on the vectorized representation of the lane information in the target coordinate system; determine the navigation characteristics based on the vectorized representation of the navigation information in the target coordinate system.

[0042] In some embodiments, the second determination unit is used to concatenate the navigation coding feature and the environment coding feature to obtain a context vector; and determine the target trajectory based on the context vector.

[0043] In some embodiments, the apparatus further comprises:

[0044] A second acquisition unit, configured to acquire a driving state of a vehicle, and determine a predicted speed of the vehicle based on the driving state;

[0045] A control unit is used to control the vehicle driving based on the predicted speed and the target trajectory.

[0046] In some embodiments, the apparatus further comprises:

[0047] A third acquisition unit, used to acquire sample data, wherein the sample data includes sample environment information and a true value trajectory of the vehicle;

[0048] A construction unit, used to construct an initial trajectory determination model, wherein the initial trajectory determination model includes an obstacle feature processing network layer for extracting features from obstacle information in the observation state information, a lane feature processing network layer for extracting features from lane information in the observation state information, a navigation feature processing network layer for extracting features from navigation information in the observation state information, a fusion network layer for fusing obstacle features and lane features, and a trajectory determination network layer for determining a driving trajectory based on a coding vector and navigation coding features, wherein outputs of the obstacle feature processing network layer and the lane feature processing network layer are connected to inputs of the fusion network layer, and outputs of the navigation feature processing network layer and the fusion network layer are connected to inputs of the trajectory determination network layer;

[0049] A training unit is used to perform model training on the initial trajectory determination model based on the sample data to obtain the trajectory determination model.

[0050] In some embodiments, the training unit is used to determine an imitation loss function; based on the imitation loss function, determine a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data; based on the first loss value, adjust the parameters in the initial trajectory determination model until the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, thereby completing model training and obtaining a trajectory determination model.

[0051] In some embodiments, the sample data also includes the vehicle's driving speed, and the training unit is further used to determine a speed loss function; based on the speed loss function, determine a second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data; based on the second loss value, adjust the parameters in the initial trajectory determination model until the second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data is less than a second preset threshold, and the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, thereby completing model training and obtaining a trajectory determination model.

[0052] A third aspect of an embodiment of the present application provides a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining a driving trajectory as described above when executing the computer program.

[0053] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the driving trajectory as described above is implemented.

[0054] The beneficial effect of the embodiments of the present invention compared with the prior art is: in the embodiments of the present application, by limiting the environmental coding features through the navigation coding features that represent the driving intention when predicting the target trajectory, it is possible to express various environments encountered in the process of prediction in the form of vectors, thereby improving the generalization of the predicted trajectory and enabling the trajectory prediction to meet a variety of vehicle driving conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a system for determining a driving trajectory provided by an exemplary embodiment is shown;

[0056] Figure 2 A schematic diagram of a process flow of a method for determining a driving trajectory provided by the present application is shown;

[0057] Figure 3 A schematic diagram of a process flow of a method for determining a driving trajectory provided by the present application is shown;

[0058] Figure 4 A schematic diagram of feature vectorization provided by the present application is shown;

[0059] Figure 5 A schematic diagram of the structure of a trajectory determination model provided by the present application is shown;

[0060] Figure 6 A schematic diagram of a process flow of a method for training a trajectory determination model provided by the present application is shown;

[0061] Figure 7 A schematic diagram of the structure of a device for determining a driving trajectory provided by the present application is shown;

[0062] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0065] Please refer to Figure 1 , which shows a schematic diagram of a driving trajectory determination system provided by an exemplary embodiment. The system is applied in a vehicle equipped with an automatic driving system. The system includes: an observation state information acquisition module 10, a trajectory determination module 20 for trajectory prediction, and a vehicle control module 30.

[0066] Among them, the observation state information acquisition module 10 is used to collect the observation state information of the surrounding environment where the vehicle is located, and the observation state information can be at least one of multiple types of information such as image information, point cloud information, etc. Different types of observation state information can be collected by different information acquisition devices. For example, image information can be obtained by a camera device, and point cloud information can be obtained by a three-dimensional laser scanner or a photographic scanner. The observation state information includes multiple information about the surrounding environment where the vehicle is located. For example, the observation state information includes a local map of the current scene, traffic lights, lane information, navigation information, indicating the current position of the vehicle, obstacles, the current state trajectory of the vehicle, and the historical state trajectory of the vehicle.

[0067] The trajectory determination module 20 is used to predict the driving trajectory of the vehicle. In some embodiments, the trajectory determination module 20 is configured with a trajectory determination model, which obtains the observation state information collected by the observation state information collection module 10 and predicts the driving trajectory of the vehicle based on the observation state information.

[0068] The vehicle control module 30 is used to obtain the driving trajectory predicted by the trajectory determination module 20, and control the vehicle driving based on the driving trajectory to achieve the demand for automatic driving of the vehicle.

[0069] It should be noted that the trajectory determination module 20 and the vehicle control module 30 can be on a computing device for controlling the vehicle, such as a vehicle-mounted terminal, a handheld computer or a cloud server with computing functions. When the trajectory determination module 20 and the vehicle control module 30 are set on different electronic devices, the trajectory determination module 20 sends the driving trajectory to the vehicle control module 30, and the vehicle control module 30 receives the driving trajectory sent by the trajectory determination module 20, and controls the vehicle driving based on the driving trajectory.

[0070] See also Figure 2 , which shows a flow chart of a method for determining a driving trajectory provided by the present application. As an example but not a limitation, the method is applied to an electronic device, and the electronic device is configured with the above-mentioned trajectory determination module.

[0071] S201, the electronic device obtains observation status information of the environment where the vehicle is located.

[0072] The observed state information is the observed state information of the surrounding environment where the vehicle is located. In some embodiments, the electronic device is provided with a state acquisition module, and accordingly, the electronic device acquires the observed state information of the environment where the vehicle is located. In some embodiments, the observed state information is acquired by the state acquisition module of other devices, and accordingly, the electronic device receives the observed state information sent by the state acquisition module of other devices.

[0073] One point that needs to be explained is that the observation status information of the vehicle's environment includes multiple information, and the multiple information can be collected by multiple observation status information acquisition modules. The multiple observation status information acquisition modules can be respectively set in different electronic devices. In the embodiments of the present application, no specific limitation is made to this.

[0074] The observed status information includes the local map of the current scene, traffic lights, lane information, navigation information, information indicating the current position of the vehicle, obstacle information, the current status trajectory of the vehicle, and the historical status trajectory of the vehicle.

[0075] S202, the electronic device determines an environment coding feature and a navigation coding feature based on the observation state information, the environment coding feature is used to represent the characteristics of the environment, the characteristics of the environment include lane characteristics, and the navigation coding feature is used to represent the driving intention corresponding to the observation state information.

[0076] The environment encoding feature (Environment Encoding Vector) indicates the features of the environment in which the vehicle is located. For example, the environment encoding feature includes lane features, obstacle features, etc. The navigation encoding feature (EmbeddingVector) is used to indicate the driving intention corresponding to the observation state information. For example, the navigation encoding feature includes navigation features, etc.

[0077] The electronic device extracts features from the observation status information to obtain obstacle features, lane features and navigation features of the environment, determines the environment coding features based on the obstacle features and lane features, and determines the navigation coding features based on the navigation features.

[0078] S203, the electronic device determines a target trajectory based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0079] Based on the principle of conditional imitation learning, the electronic device splices the navigation coding feature including the navigation feature as a condition (Condition) with the environmental coding feature to obtain a context vector, and predicts the target trajectory based on the context vector. The process can be: the electronic device splices the navigation coding feature and the environmental coding feature to obtain a context vector; based on the context vector, the target trajectory is determined.

[0080] In this implementation, the electronic device uses the navigation coding features as the conditions for conditional imitation learning and splices them with the environmental coding features. When training the trajectory confirmation model for identifying the context vector, there is no need to set the loss function related to the navigation features, which can ensure that the predicted target trajectory matches the navigation intention corresponding to the navigation features, thereby simplifying the training process of the trajectory confirmation model. In addition, after the navigation coding features corresponding to the navigation features with different intentions are fused with the environmental coding features, the obtained context vector can be distinguished in a low-dimensional space, further improving the accuracy of determining the target trajectory.

[0081] In some embodiments, the electronic device obtains a driving state of a vehicle, determines a predicted speed of the vehicle based on the driving state, and controls the driving of the vehicle based on the predicted speed and the target trajectory.

[0082] The electronic device predicts the vehicle's driving speed by combining the vehicle's driving state and the observation state information of the current environment, and determines the vehicle's speed in different lanes and passing different obstacles by combining the vehicle's driving speed with obstacle features, lane features and other features in different environmental information, and obtains the predicted speed of the vehicle at different time points in the future. In addition, since the vehicle's driving speed is predicted by combining the observation state information of the environment and the driving state information, it avoids learning wrong correlations due to imitation learning, for example, a vehicle that does not start cannot start because its historical state is static, thereby improving the accuracy of predicting the vehicle's driving speed.

[0083] In an embodiment of the present application, when predicting the target trajectory, the environment coding features are restricted by the navigation coding features that represent the driving intention, so that the various environments encountered in the process of prediction can be expressed in the form of vectors, thereby improving the generalization of the predicted trajectory and enabling the trajectory prediction to meet a variety of vehicle driving conditions.

[0084] See also Figure 3 , which shows a flow chart of a method for determining a driving trajectory provided by the present application. As an example but not a limitation, the method is applied to an electronic device, and the electronic device is configured with the above-mentioned trajectory determination module.

[0085] S301, the electronic device obtains observation status information of the environment where the vehicle is located.

[0086] The principle of this step is the same as that of step S201 and will not be repeated here.

[0087] S302, the electronic device extracts obstacle features, lane features and navigation features from the observation state information.

[0088] The environment includes obstacles and lanes, and the observation state information of the environment includes obstacle information, lane information and navigation information in the area corresponding to the environment. The obstacle features, lane features and navigation features are vectorized features. The process includes:

[0089] S3021, the electronic device creates a target coordinate system with the current position of the vehicle as the origin and the direction of the vehicle head as the positive direction of the X-axis.

[0090] The electronic device uses the current position of the vehicle as the coordinate origin of the scene and the body orientation of the vehicle as the X-axis.

[0091] In this step, the information in the observation status information of the electronic device is rotated, so as to convert the information represented in other coordinate systems into the target coordinate system. For example, the observation status information can be at least one of multiple types of information such as image information and point cloud information. Different types of observation status information can be collected by different information acquisition devices. For example, image information can be obtained by a camera device, and point cloud information can be obtained by a three-dimensional laser scanner or a photographic scanner. Accordingly, different observation status information has different coordinate systems. The electronic device can pre-store the coordinate systems corresponding to different types of observation status information, and respectively determine the conversion relationship between the different types of coordinate systems and the target coordinate system. In this step, according to the conversion message, the observation status information is converted to the target coordinate system.

[0092] S3022: The electronic device determines the obstacle feature based on the vector representation of the obstacle information in the target coordinate system.

[0093] Obstacles in the environment include static obstacles and dynamic obstacles. Static obstacles include stationary objects in the environment, such as traffic cones, roadblocks and other stationary objects. Dynamic obstacles include dynamic objects in the environment, such as other vehicles, pedestrians, bicycles, motorcycles and other traffic participants.

[0094] For static obstacles, the electronic device can represent the positional relationship between the static obstacle and the vehicle through position coordinates. For dynamic obstacles, the electronic device predicts the position coordinates and direction of the obstacle at multiple time points based on the coordinates and direction of the obstacle in the target system to obtain the obstacle feature. The process includes: the electronic device determines the position coordinates and direction of the obstacle at multiple time points based on the position coordinates and direction of the obstacle in the target coordinate system; and generates the obstacle feature based on the position coordinates and direction of the obstacle at multiple time points.

[0095] The obstacle feature can be expressed by the characteristic expression 1:

[0096] Characteristic expression 1:

[0097] The feature expression represents the current time point and the past T h The position and direction information of each time point. The interval between each time point can be set as needed. In the embodiment of the present application, the time interval is not specifically limited. For example, the time interval can be 0.1 seconds, etc. t ,y t ) is the horizontal and vertical coordinates of the obstacle in the target coordinate system at time t, (hx t ,hy t ) is the yaw angle component on the horizontal and vertical axes at time t. In some embodiments, the yaw angle component can be calculated by taking sin and cos of the yaw angle.

[0098] S3023, the electronic device determines the lane characteristics based on the vector representation of the lane information in the target coordinate system.

[0099] The lane information in the observation state information can be obtained from a semantic map. The process includes: for each lane in the lane information, the electronic device divides the lane into multiple lane segments; for each lane segment, multiple waypoints and the traffic light information corresponding to each waypoint are sampled; based on the position of each waypoint in the target coordinate system and the traffic light information corresponding to each waypoint, the lane feature is generated.

[0100] For each lane, the lane is divided into multiple lane segments, and the lengths of the multiple segments are the same or different. In the embodiment of the present application, there is no specific limitation on this. For each lane segment, the electronic device collects and samples n waypoints on the lane segment to represent the lane segment, and represents the lane feature of the lane segment by the position coordinates of the waypoints in the target coordinate system and the state of the traffic light, where the state of the traffic light can be represented by a 5-dimensional one-hot vector. The lane feature can be represented by feature expression 2:

[0101] Characteristic expression 2: {(x i ,y i , tl i )} i＝1，...，n

[0102] Among them, (x i ,y i ) is the coordinate of the i-th waypoint, tl i is the traffic light status of the ith road point, represented by a 5-dimensional one-hot vector: (1, 0, 0, 0, 0) indicates no traffic light, (0, 1, 0, 0, 0) indicates an uncertain traffic light status, (0, 0, 1, 0, 0) indicates a red light status, (0, 0, 0, 1, 0) indicates a green light status, and (0, 0, 0, 0, 1) indicates a yellow light status.

[0103] S3024: The electronic device determines the navigation feature based on the vector representation of the navigation information in the target coordinate system.

[0104] The navigation path in the navigation information is the target lane, and the navigation feature is represented by multiple positions in the target lane. The process includes: the electronic device determines the target lane in the navigation information, and the target lane is the lane in the navigation path; divides the target lane into multiple sections; and generates the navigation feature based on the position of each section in the target coordinate system.

[0105] The navigation feature can be expressed by feature expression three:

[0106] Characteristic expression three: {(x i ,y i )} i＝1，...，n

[0107] Among them, (x i ,y i ) are the coordinates of the ith road segment.

[0108] In this implementation, by constructing a target coordinate system, the features in the target coordinate system are all represented in the form of vectors, thereby improving the inductive bias and sample efficiency of the features, so that when predicting the trajectory through the feature vector, the accuracy of trajectory prediction can be improved.

[0109] See also Figure 4 , which shows a schematic diagram of a vectorized visualization of a driving scenario. The lines composed of different points represent different trajectories. For example, the historical state trajectory of the vehicle and the obstacle, the center line of the lane segment, the target lane, the true value trajectory, and the predicted target trajectory.

[0110] S303, the electronic device fuses the obstacle feature and the lane feature to obtain the environment coding feature.

[0111] See also Figure 5 The electronic device fuses the obstacle features and the lane features through the trajectory determination model. The trajectory determination model includes a fusion network layer, and the electronic device fuses the obstacle features and the lane features through the fusion network layer. The fusion network layer can be a FusionNet network layer, which uses a self-attention method to learn the interaction between different elements. In some embodiments, the FusionNet network layer uses a Transformer Encoder to fuse the features of different elements.

[0112] S304: The electronic device processes the navigation feature to obtain the navigation coding feature.

[0113] The electronic device processes the navigation feature through the trajectory determination model. Figure 5 The trajectory determination model includes a navigation feature processing network layer, and the navigation information is input into the navigation feature processing network layer to obtain the navigation coding feature. The navigation feature processing network layer may be RouteNet, and the RouteNet adopts a gmlp structure, which is an MLP structure with channel and spatial attention, and can extract channel and spatial information from the navigation information to obtain the navigation coding feature.

[0114] In this implementation, the electronic device determines the environment coding features and the navigation coding features through multiple network layers in the trajectory determination model, thereby improving the accuracy and efficiency of determining the environment coding features and the navigation coding features.

[0115] S305, the electronic device determines a target trajectory based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0116] The principle of this step is the same as that of step S203 and will not be repeated here.

[0117] One thing that needs to be explained is that the environment coding feature and the navigation coding feature can be fused through the trajectory determination network layer in the trajectory determination model. The trajectory determination network layer can be a PredHead network layer. The PredHead network layer adopts an MLP structure, concatenates the input environment coding feature and the navigation coding feature into a context vector, and predicts the target trajectory based on the context vector.

[0118] In an embodiment of the present application, when predicting the target trajectory, the environment coding features are restricted by the navigation coding features that represent the driving intention, so that the various environments encountered in the process of prediction can be expressed in the form of vectors, thereby improving the generalization of the predicted trajectory and enabling the trajectory prediction to meet a variety of vehicle driving conditions.

[0119] See also Figure 6 , which shows a flow chart of a method for training a trajectory determination model provided by the present application. As an example but not a limitation, the method is applied to an electronic device, and the electronic device is configured with the above-mentioned trajectory determination module.

[0120] S601, the electronic device obtains sample data, where the sample data includes sample environment information and a true value trajectory of the vehicle.

[0121] The sample data may be driving data generated by the driver driving the vehicle. In some embodiments, the sample data may be Among them, S i represents the sample environment information, τ i Represents the true trajectory of the vehicle.

[0122] In some embodiments, the electronic device obtains the sample data from a sample database. The sample database stores multiple sets of environmental information and corresponding data of true value trajectories. In this step, the electronic device obtains the sample data from the database.

[0123] The electronic device labels the sample data according to the target coordinate system to obtain the training label. The labeled true value trajectory can be Among them, (x t ,y t ) represents the horizontal and vertical coordinates of the vehicle at time t, h t In some embodiments, the sample data also includes a driving speed, and the electronic device marks the driving speed. The marked driving speed can be Among them, s t represents the speed at time t.

[0124] S602: The electronic device constructs an initial trajectory determination model.

[0125] See also Figure 5The initial trajectory determination model includes an obstacle feature processing network layer for extracting features of obstacle information in the observed state information, a lane feature processing network layer for extracting features of lane information in the observed state information, a navigation feature processing network layer for extracting features of navigation information in the observed state information, a fusion network layer for fusing obstacle features and lane information, and a trajectory determination network layer for determining the driving trajectory based on the coding vector and the navigation coding features, wherein the outputs of the obstacle feature processing network layer and the lane feature processing network layer are connected to the input of the fusion network layer, and the outputs of the navigation feature processing network layer and the fusion network layer are connected to the input of the trajectory determination network layer.

[0126] Among them, the obstacle feature processing network layer (Agent features), lane feature processing network layer (MapNet) and navigation feature processing network layer (RouteNet) can adopt the gmlp structure. The fusion network layer (FusionNet) uses self-attention to learn the interaction between different elements. Specifically, Transformer Encoder can be used to fuse the features of different elements. The trajectory determination network layer (PredHead) adopts the mlp structure.

[0127] S603: The electronic device performs model training on the initial trajectory determination model based on the sample data to obtain the trajectory determination model.

[0128] The electronic device vectorizes the sample data, and trains the trajectory determination model based on the vectorized sample data. The process of the electronic device vectorizing the sample data is the same as the principle of steps S3021-S3024, which will not be repeated here.

[0129] In some embodiments, the electronic device trains the trajectory determination model by determining an imitation loss function. The process may be: the electronic device determines an imitation loss function. Based on the imitation loss function, the electronic device determines a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data. The electronic device adjusts the parameters in the initial trajectory determination model based on the first loss value until the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

[0130] Among them, the imitation loss function can be based on l 1 The loss function of the norm is calculated by the following formula 1:

[0131] Formula 1:

[0132] Among them, L imitation represents the imitation loss value, p t =(x t ,y t ,h t ) represents the true value trajectory, In this implementation, the trajectory determination model is trained by imitating the loss function, thereby improving the accuracy of the trajectory determination model.

[0133] In some embodiments, the electronic device also trains the trajectory determination model in combination with a speed loss function, and the process may be: the electronic device determines a speed loss function. Based on the speed loss function, the electronic device determines a second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data. The electronic device adjusts the parameters in the initial trajectory determination model based on the second loss value until the second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data is less than a second preset threshold, and the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

[0134] Among them, the speed loss function can be based on l 2 The calculation formula of the loss function of the norm is expressed by the following formula 2:

[0135] Formula 2:

[0136] Among them, s t represents the true value velocity, Indicates the predicted speed.

[0137] In this implementation, the trajectory determination model is trained in combination with the speed loss function and the imitation loss function, so that the trajectory determination model can predict the trajectory while also predicting the speed, thereby avoiding learning incorrect correlations due to imitation learning. For example, a vehicle that does not start cannot be started because its historical state is stationary, thereby improving the accuracy of predicting the vehicle's driving speed.

[0138] It should be noted that the electronic device can also train the trajectory determination model in combination with other loss functions. For example, a collision loss function can be added to avoid collision problems.

[0139] Another point to be explained is that the model training process can also be implemented by other servers with model training functions. Accordingly, the server performs model training on the trajectory determination model and sends the trained trajectory determination model to the electronic device. The process of the server performing model training on the trajectory determination model is the same as the principle of steps S601-S603, which will not be repeated here.

[0140] In an embodiment of the present application, when predicting the target trajectory, the environment coding features are restricted by the navigation coding features that represent the driving intention, so that the various environments encountered in the process of prediction can be expressed in the form of vectors, thereby improving the generalization of the predicted trajectory and enabling the trajectory prediction to meet a variety of vehicle driving conditions.

[0141] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0142] See also Figure 7 , which shows a schematic diagram of the structure of a driving trajectory determination device provided by the present application, and includes various units for executing various steps in the above embodiments, see Figure 7 , the driving trajectory determination device comprises:

[0143] The first acquisition unit 701 is used to acquire the observation state information of the environment where the vehicle is located;

[0144] A first determining unit 702 is used to determine an environment coding feature and a navigation coding feature based on the observation state information, wherein the environment coding feature is used to represent a feature of the environment, the feature of the environment includes a lane feature, and the navigation coding feature is used to represent a driving intention corresponding to the observation state information;

[0145] The second determining unit 703 is used to determine a target trajectory based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0146] In some embodiments, the first determination unit 702 is used to extract obstacle features, lane features and navigation features from the observation state information; fuse the obstacle features and the lane features to obtain the environment coding features; and process the navigation features to obtain the navigation coding features.

[0147] In some embodiments, the environment includes obstacles and lanes, and the observation state information of the environment includes obstacle information, lane information and navigation information in the area corresponding to the environment. The first determination unit 702 is used to create a target coordinate system with the current position of the vehicle as the origin and the direction of the front of the vehicle as the positive direction of the X-axis; determine the obstacle feature based on the vector representation of the obstacle information in the target coordinate system; determine the lane feature based on the vector representation of the lane information in the target coordinate system; determine the navigation feature based on the vector representation of the navigation information in the target coordinate system.

[0148] In some embodiments, the second determination unit 703 is used to concatenate the navigation coding feature and the environment coding feature to obtain a context vector; and determine the target trajectory based on the context vector.

[0149] In some embodiments, the apparatus further comprises:

[0150] A second acquisition unit, used to acquire a driving state of the vehicle, and determine a predicted speed of the vehicle based on the driving state;

[0151] The control unit is used to control the driving of the vehicle based on the predicted speed and the target trajectory.

[0152] In some embodiments, the apparatus further comprises:

[0153] A third acquisition unit is used to acquire sample data, where the sample data includes sample environment information and a true value trajectory of the vehicle;

[0154] A construction unit is used to construct an initial trajectory determination model, which includes an obstacle feature processing network layer for extracting features of obstacle information in the observed state information, a lane feature processing network layer for extracting features of lane information in the observed state information, a navigation feature processing network layer for extracting features of navigation information in the observed state information, a fusion network layer for fusing obstacle features and lane features, and a trajectory determination network layer for determining a driving trajectory based on a coding vector and navigation coding features, wherein outputs of the obstacle feature processing network layer and the lane feature processing network layer are connected to inputs of the fusion network layer, and outputs of the navigation feature processing network layer and the fusion network layer are connected to inputs of the trajectory determination network layer;

[0155] The training unit is used to perform model training on the initial trajectory determination model based on the sample data to obtain the trajectory determination model.

[0156] In some embodiments, the training unit is used to determine an imitation loss function; based on the imitation loss function, determine a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data; based on the first loss value, adjust the parameters in the initial trajectory determination model until the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, thereby completing model training and obtaining a trajectory determination model.

[0157] In some embodiments, the sample data also includes the vehicle's driving speed, and the training unit is further used to determine a speed loss function; based on the speed loss function, determine a second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data; based on the second loss value, adjust the parameters in the initial trajectory determination model until the second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data is less than a second preset threshold, and the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, thereby completing model training and obtaining a trajectory determination model.

[0158] In an embodiment of the present application, when predicting the target trajectory, the environment coding features are restricted by the navigation coding features that represent the driving intention, so that the various environments encountered in the process of prediction can be expressed in the form of vectors, thereby improving the generalization of the predicted trajectory and enabling the trajectory prediction to meet a variety of vehicle driving conditions.

[0159] Figure 8 Schematic diagram of an electronic device provided by an embodiment of the present application. Figure 8 As shown, the electronic device 8 of this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a driving trajectory determination program. When the processor 80 executes the computer program 82, the steps in the above-mentioned driving trajectory determination method embodiments are implemented, such as Figure 2 Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 7 The functions of units 701 to 703 are shown.

[0160] Exemplarily, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 82 in the electronic device 8. For example, the computer program 82 may be divided into a first acquisition unit, a first determination unit, and a second determination unit, and the specific functions of each unit are as follows:

[0161] The first acquisition unit 701 is used to acquire the observation state information of the environment where the vehicle is located;

[0162] A first determining unit 702 is used to determine an environment coding feature and a navigation coding feature based on the observation state information, wherein the environment coding feature is used to represent a feature of the environment, the feature of the environment includes a lane feature, and the navigation coding feature is used to represent a driving intention corresponding to the observation state information;

[0163] The second determining unit 703 is used to determine a target trajectory based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

[0164] The electronic device 8 may be a vehicle-mounted terminal, a handheld computer, a cloud server, or other computing device for controlling the vehicle. The electronic device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will appreciate that Figure 8 It is only an example of the electronic device 8 and does not constitute a limitation of the electronic device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 8 may also include input and output devices, network access devices, buses, etc.

[0165] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0166] The memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. The memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 8. Further, the memory 81 may also include both an internal storage unit of the electronic device 8 and an external storage device. The memory 81 is used to store the computer program and other programs and data required by the terminal device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0167] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0168] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0169] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0170] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0173] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0174] An embodiment of the present application further provides a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-described various method embodiments when executing the computer program.

[0175] The embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0176] The embodiment of the present application also provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0177] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for determining a driving trajectory, characterized in that: The method comprises: Obtain observation status information of the vehicle's environment; Based on the observation state information, determine an environment coding feature and a navigation coding feature, the environment coding feature is used to represent a feature of the environment, the feature of the environment includes a lane feature, and the navigation coding feature is used to represent a driving intention corresponding to the observation state information; A target trajectory is determined based on the environment coding feature and the navigation coding feature, where the target trajectory is used to instruct the vehicle to travel along a lane corresponding to the target trajectory.

2. The method according to claim 1, characterized in that The determining of the environment coding feature and the navigation coding feature based on the observation state information includes: Extracting obstacle features, lane features and navigation features from the observed state information; The obstacle feature and the lane feature are integrated to obtain the environment coding feature; The navigation feature is processed to obtain the navigation coding feature.

3. The method according to claim 2, characterized in that The environment includes obstacles and lanes, the observation state information of the environment includes obstacle information, lane information and navigation information in an area corresponding to the environment, and the extracting obstacle features, lane features and navigation features from the observation state information includes: Create a target coordinate system with the current position of the vehicle as the origin and the direction of the vehicle head as the positive direction of the X-axis; Determining the obstacle feature based on the vector representation of the obstacle information in the target coordinate system; Determining the lane characteristics based on the vectorized representation of the lane information in the target coordinate system; The navigation feature is determined based on the vectorized representation of the navigation information in the target coordinate system.

4. The method according to claim 1, characterized in that The determining of the target trajectory based on the environment coding feature and the navigation coding feature includes: Concatenating the navigation coding feature and the environment coding feature to obtain a context vector; Based on the context vector, the target trajectory is determined.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquiring a driving state of a vehicle, and determining a predicted speed of the vehicle based on the driving state; Based on the predicted speed and the target trajectory, the vehicle is controlled to travel.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire sample data, wherein the sample data includes sample environment information and a true value trajectory of the vehicle; Constructing an initial trajectory determination model, the initial trajectory determination model comprising an obstacle feature processing network layer for extracting features from obstacle information in the observation state information, a lane feature processing network layer for extracting features from lane information in the observation state information, a navigation feature processing network layer for extracting features from navigation information in the observation state information, a fusion network layer for fusing obstacle features and lane features, and a trajectory determination network layer for determining a driving trajectory based on a coding vector and navigation coding features, wherein outputs of the obstacle feature processing network layer and the lane feature processing network layer are connected to inputs of the fusion network layer, and outputs of the navigation feature processing network layer and the fusion network layer are connected to inputs of the trajectory determination network layer; Based on the sample data, model training is performed on the initial trajectory determination model to obtain the trajectory determination model.

7. The method according to claim 6, characterized in that The performing model training on the initial trajectory determination model based on the sample data to obtain the trajectory determination model includes: Determine the imitation loss function; Based on the imitation loss function, determining a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data; Based on the first loss value, the parameters in the initial trajectory determination model are adjusted until a first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

8. The method according to claim 7, characterized in that The sample data also includes the driving speed of the vehicle, and the method further includes: Determine the speed loss function; Based on the speed loss function, determining a second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data; Based on the second loss value, the parameters in the initial trajectory determination model are adjusted until the second loss value between the predicted driving speed output by the initial trajectory determination model and the driving speed of the vehicle in the sample data is less than a second preset threshold, and the first loss value between the predicted driving trajectory output by the initial trajectory determination model and the true value trajectory of the vehicle in the sample data is less than a first preset threshold, and the model training is completed to obtain the trajectory determination model.

9. A vehicle, characterized in that: The vehicle includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for determining the driving trajectory as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method for determining a driving trajectory as described in any one of claims 1 to 8.

Images