Training method, acquisition method, device, equipment and automatic driving vehicle

Abstract

The disclosure provides a training method, an acquisition method, a device, equipment and an automatic driving vehicle, relates to the technical field of computers, and particularly relates to the technical field of artificial intelligence and automatic driving. A specific implementation scheme is as follows: a training data set is acquired, the training data set includes multiple training data, each training data includes historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, road topology information of a road in the driving scene, and a future trajectory of the driving vehicle; according to the future trajectory, positive example pairs and negative example pairs of a contrast learning task are constructed based on the training data set; and a target model is subjected to contrast learning training based on the positive example pairs and the negative example pairs of the contrast learning task, the target model being used for acquiring a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information and the road topology information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, in particular to the technical field of artificial intelligence and automatic driving, and more particularly to a training method, an acquisition method, an apparatus, a device and an automatic driving vehicle. BACKGROUND

[0002] In recent years, with the breakthrough progress of deep learning technology in the field of artificial intelligence such as computer vision and natural language processing, the field of automatic driving has also explored in the research directions of trajectory prediction, behavior decision, trajectory planning, etc.

[0003] Among them, the effective representation learning of the semantics of the driving scene (such as lane lines, obstacles, traffic lights, etc.) where the automatic driving vehicle is located is the core technology of the above research directions. SUMMARY

[0004] The present disclosure provides a training method, an acquisition method, an apparatus, a device and an automatic driving vehicle to solve at least one of the above-mentioned defects.

[0005] According to a first aspect of the present disclosure, a model training method is provided, which comprises:

[0006] obtaining a training data set, the training data set comprising a plurality of training data, each training data comprising historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, road topology information of a road in the driving scene, and a future trajectory of the driving vehicle;

[0007] According to the future trajectory, based on the training data set, a positive example pair and a negative example pair of a contrast learning task are constructed;

[0008] Based on the positive example pair and the negative example pair of the contrast learning task, a target model is trained by contrast learning, the target model being used to acquire a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information and the road topology information.

[0009] According to a second aspect of the present disclosure, an information acquisition method is provided, which comprises:

[0010] obtaining historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, and road topology information of a road in the driving scene;

[0011] inputting the historical motion information, the obstacle motion information and the road topology information into a pre-trained pre-training model to acquire a scene representation tensor of the driving scene;

[0012] The pre-trained model is obtained by using the model training method in the first aspect.

[0013] According to a third aspect of the present disclosure, a model training apparatus is provided, which comprises:

[0014] a data module configured to obtain a training data set, the training data set comprising a plurality of training data, each of the training data comprising historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, road topology information of a road in the driving scene, and a future trajectory of the driving vehicle;

[0015] a label module configured to construct positive pairs and negative pairs of a contrastive learning task based on the training data set according to the future trajectory;

[0016] a training module configured to perform contrastive learning training on a target model based on the positive pairs and the negative pairs of the contrastive learning task, the target model being configured to obtain a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information, and the road topology information.

[0017] According to a fourth aspect of the present disclosure, an information obtaining apparatus is provided, which comprises:

[0018] an information module configured to obtain historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, and road topology information of a road in the driving scene;

[0019] a prediction module configured to input the historical motion information, the obstacle motion information, and the road topology information into a pre-trained model to obtain a scene representation tensor of the driving scene;

[0020] The pre-trained model is obtained by using the model training method in the first aspect.

[0021] According to a fifth aspect of the present disclosure, an electronic device is provided, which comprises:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein

[0024] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the model training method or the information obtaining method.

[0025] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the model training method or the information acquisition method.

[0026] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the model training method or the information acquisition method.

[0027] According to an eighth aspect of the present disclosure, an electronic device is provided, comprising:

[0028] at least one processor; and

[0029] a memory in communication with the at least one processor; wherein

[0030] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform path planning using a scene representation tensor of a driving scene in which a driving vehicle is located, which is acquired according to the information acquisition method, and to perform automatic driving control on the driving vehicle according to a path planning result.

[0031] According to a ninth aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the electronic device according to the eighth aspect of the present disclosure.

[0032] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0034] Figure 1 is a flowchart of a model training method provided by an embodiment of the present disclosure;

[0035] Figure 2 is a flowchart of part of steps of another model training method provided by an embodiment of the present disclosure;

[0036] Figure 3 is a flowchart of part of steps of another model training method provided by an embodiment of the present disclosure;

[0037] Figure 4 is a schematic diagram of a model of another model training method provided by an embodiment of the present disclosure;

[0038] Figure 5is a flowchart of an information acquisition method provided by an embodiment of the present disclosure.

[0039] Figure 6 is a structural diagram of a model training device provided by an embodiment of the present disclosure.

[0040] Figure 7 is a structural diagram of an information acquisition device provided by an embodiment of the present disclosure.

[0041] Figure 8 is a block diagram of an electronic device for implementing the model training method and the information acquisition method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, which should be considered in a descriptive sense only. Thus, it will be apparent to one of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0043] In some related technologies, a scene in which an autonomous vehicle is located can be represented using self-supervised contrastive learning. Specifically, self-supervised contrastive learning is to use a deep learning model to encode the scene in which the autonomous vehicle is located into a vector representation, and to guide the training of the deep learning model by automatically mining commonality signals between scenes as a supervision signal.

[0044] Although the scene in which the autonomous vehicle is located can be represented by self-supervised contrastive learning, the learned representation has poor interpretability, and the learned obstacle importance weight tends to be uniformly distributed, lacking the ability to perceive risk obstacles such as key traffic participants (i.e., traffic participants that have a greater impact on the driving of the autonomous vehicle), thereby affecting the performance of downstream tasks such as trajectory prediction, behavior decision, and trajectory planning.

[0045] The model training method, the information acquisition method, the device, the electronic device, and the autonomous vehicle provided by the embodiments of the present disclosure aim to solve at least one of the above technical problems of the prior art.

[0046] The model training method and the information acquisition method provided by the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0047] Figure 1 A flowchart of a model training method provided by the embodiments of the present disclosure is shown, as shown in Figure 1 The method can mainly include the following steps.

[0048] In step S110, a training data set is acquired.

[0049] The training data set includes a plurality of training data, and each training data includes historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, road topology information of a road in the driving scene, and a future trajectory of the driving vehicle.

[0050] In step S120, based on the training data set, positive and negative example pairs of a contrast learning task are constructed according to the future trajectory.

[0051] In step S130, a target model is trained by contrast learning based on the positive and negative example pairs of the contrast learning task.

[0052] The target model is used to acquire a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information, and the road topology information.

[0053] For example, in step S110, the driving vehicle can be an autonomous driving vehicle. The driving scene where the driving vehicle is located can be an environment where the driving vehicle is located, which can be represented by the features (such as position, speed, acceleration, etc.) of the objects (such as obstacles, lane lines, pedestrian crossings, etc.) that can affect the driving of the driving vehicle in the scene and the interaction relationship between the objects.

[0054] In some specific implementations, the driving scene corresponding to the training data can be an autonomous driving intersection straight driving scene.

[0055] In some possible implementations, the training data can be continuous multiple frames of images in a driving vehicle visual coordinate system. The multiple frames of images are divided into historical frame images and future frame images in chronological order. In some specific implementations, the continuous multiple frames of images include 76 frames of images, of which the first 16 frames of images are historical frame images and the last 60 frames of images are future frame images.

[0056] The historical motion information of the driving vehicle can be obtained from the historical frame images, which can include the position, speed, and acceleration of the driving vehicle at a position corresponding to each historical frame image.

[0057] The obstacles in the driving scene of the driving vehicle can include motor vehicles, non-motor vehicles, pedestrians, roadblocks, and other objects in the driving scene that affect the driving of the driving vehicle. The obstacle motion information includes obstacle motion information of multiple obstacles. The obstacle motion information can also be obtained from the historical frame images, which can include the position, speed, and acceleration of the obstacles at a position corresponding to each historical frame image.

[0058] There can be multiple obstacles in each historical frame image, and the obstacle motion information includes obstacle motion information of multiple obstacles. In some specific implementations, for convenience of calculation, a fixed number (such as 80) of obstacles are selected in each historical frame image, and the obstacle motion information of these obstacles is obtained. If the number of obstacles in a historical frame image does not meet the requirement, zero padding is used to make the number of obstacles meet the requirement.

[0059] In some possible implementations, the road topology information of the driving scene can include ground identification information of the road in the driving scene that affects the driving of the driving vehicle, such as lane line information, pedestrian crossing information, and stop line information.

[0060] The lane line information can include coordinate information of the lane line, number information of the lane line, number information and coordinate information of line segments included in the lane line, and the like. The pedestrian crossing information can include coordinate information of the pedestrian crossing, number information of the pedestrian crossing, number information and coordinate information of line segments included in the pedestrian crossing, traffic light information corresponding to the pedestrian crossing, and the like. The stop line information can include coordinate information of the stop line, number information of the stop line, number information and coordinate information of line segments included in the stop line, traffic light information corresponding to the stop line, and the like.

[0061] The lane line, the pedestrian crossing, and the stop line are relatively important ground identifications in road ground identification. Using the lane line information, the pedestrian crossing information, and the stop line information can reduce the amount of road topology information while ensuring the importance of the obtained road topology information, thereby providing convenience for model training.

[0062] In some possible implementation manners, the future trajectory of the driving vehicle can be obtained from the future frame image, i.e., the driving trajectory of the driving vehicle in the future is obtained from multiple future frame images, which can be represented by position information of the driving vehicle.

[0063] In some possible implementation manners, in step S120, the positive example pair and the negative example pair can be constructed according to the distance between the future trajectories of the two training data.

[0064] In some possible implementation manners, the training data can be grouped into pairs, and the positive example pair and the negative example pair of the positive pair learning task can be determined according to the Euclidean distance of the future trajectories of each pair of training data.

[0065] In some specific implementation manners, two training data can be selected as candidate training data from the training data set; the trajectory distance of the future trajectories of the two candidate training data is calculated; in the case that the trajectory distance is less than a preset threshold, the two candidate training data are determined as the positive example pair of the contrast learning task; in the case that the trajectory distance is not less than the preset threshold, the two candidate training data are determined as the negative example pair of the contrast learning task.

[0066] In some possible implementation manners, in step S130, the scene representation tensor of the driving scene can be a tensor including features (such as position, speed, acceleration, etc.) used to represent objects (such as obstacles, lane lines, pedestrian crossings, etc.) that can affect the driving of the driving vehicle in the driving scene, and a tensor representing the interaction relationship between the objects.

[0067] If the similarity of two scene representation tensors is close, it means that the similarity of the driving scenes corresponding to the two scene representation tensors is also close, and the future trajectories of the driving vehicle in the two driving scenes should also be similar.

[0068] Therefore, the positive example pair and the negative example pair of the contrast learning task can be constructed according to the future trajectory. That is, two training data with close future trajectories are taken as the positive example pair, and two training data with distant future trajectories are taken as the negative example pair.

[0069] Based on the positive example pair and the negative example pair of the contrast learning task, the contrast learning training of the target model can be contrast learning training of the target model using a contrast learning objective function, so that the target model satisfies that the scene representation tensors corresponding to the positive example pair are similar, and the scene representation tensors corresponding to the negative example pair are quite different.

[0070] In some possible implementation manners, the target model includes a first encoder, a second encoder, a third encoder, and a fourth encoder.

[0071] The first encoder is configured to obtain ego motion features and obstacle motion features according to historical motion information and obstacle motion information; the second encoder is configured to obtain road topology features according to road topology information; the third encoder is configured to obtain map interaction features according to the ego motion features and the road topology features; and the fourth encoder is configured to obtain a scene representation tensor according to the map interaction features and the obstacle motion features.

[0072] In some possible implementation manners, the first encoder is a Transformer Encoder (a deep self-attention transformation network encoder); the second encoder is a CNN (a convolutional neural network), which in some specific implementation manners includes a one-dimensional CNN, a Max Pooling (a maximum pooling layer) and a Mean Pooling (a mean pooling layer) connected in sequence; the third encoder is an MCG (Multi-context Gating) structure; and the fourth encoder is also a Transformer Encoder.

[0073] In some possible implementation manners, the fourth encoder is further configured to obtain an obstacle representation tensor of an obstacle in the driving scene according to the map interaction features and the obstacle motion features.

[0074] In some possible implementation manners, the target model further includes a prediction network configured to predict a probability that the obstacle is a key obstacle according to the obstacle representation tensor.

[0075] In some possible implementation manners, the prediction network can be an MLP (a multi-layer perceptron).

[0076] In some possible implementation manners, the target model can be supervised trained.

[0077] In some specific implementation manners, the target model can be supervised trained by determining an obstacle label corresponding to the training data and supervised training the target model based on an output of the prediction network and the obstacle label.

[0078] The obstacle label is used to identify whether the obstacle in the driving scene is a key obstacle.

[0079] In some possible implementation manners, the obstacle label corresponding to the training data can be determined according to a future trajectory of the training data.

[0080] In some specific implementation manners, at least one candidate key obstacle is determined according to the future trajectory of the training data; for each candidate key obstacle, a planning trajectory of a driving vehicle after the candidate key obstacle is removed from the driving scene is obtained through simulation; and whether the candidate key obstacle is a key obstacle is determined according to a trajectory distance between the planning trajectory and the future trajectory.

[0081] The at least one candidate key obstacle determined according to the future trajectory of the training data can be an obstacle for which a driving vehicle makes a yielding behavior because the obstacle exists.

[0082] In some possible implementation manners, the joint training of the supervised training and the contrastive learning training on the target model can be performed.

[0083] In some specific implementation manners, the joint training of the supervised training and the contrastive learning training on the target model can be determining, according to a preset balance factor, a training weight of the supervised training and a training weight of the contrastive learning training, performing the supervised training on the target model according to the training weight of the supervised training, and performing the contrastive learning training on the target model according to the training weight of the contrastive learning training.

[0084] In the model training method provided in the embodiments of the present disclosure, the positive example pair and the negative example pair of the contrastive learning task are constructed according to the known future trajectory in the training data, which is equivalent to converting the self-supervised contrastive learning into supervised learning using a determined label. Compared with the self-supervised contrastive learning, the supervised learning has a stronger ability to learn the features of the training data, and the learned representation is more interpretable. Therefore, the model effect of the target model constructed through the supervised learning is better, and the interpretability of the scene representation tensor obtained through the trained target model is also stronger.

[0085] The model training method of the present disclosure will be described below.

[0086] As described above, in some possible implementation manners, the road topology information of the driving scene road can include ground identification information in the driving scene road that affects driving of the driving vehicle, such as lane line information, pedestrian crossing information, and stop line information.

[0087] In some possible implementation manners, in order to facilitate input of the training data into the target model to train the target model, the training data can be represented in the form of a tensor.

[0088] In some specific implementation manners, the historical motion information and the obstacle motion information of all the training data in one training process can be represented in one tensor, such as a tensor Obstacle.

[0089] In some specific implementations, the tensor Obstacle can be a tensor with a size of [batch, obs_num, frame_num, feature1]. It can be concatenated from a tensor with a size of [batch, 1, frame_num, feature1] and a tensor with a size of [batch, obs_num-1, frame_num, feature1].

[0090] Wherein, batch is the amount of training data used in one training process; obs_num is the number of driving vehicles and obstacles, which can be 80, and when the number of obstacles does not meet the requirements, the zero padding method is used to make the number of obstacles meet the requirements; frame_num is the number of historical frame images, which can be 16; and feature1 is the historical motion information (such as the position, speed, acceleration, etc.) of the driving vehicle of each historical frame image, and the obstacle information (such as the position, speed, acceleration, etc.) of each obstacle.

[0091] The tensor with a size of [batch, 1, frame_num, feature1] is the historical motion information of the driving vehicle of each historical frame image. The tensor with a size of [batch, obs_num-1, frame_num, feature1] is the obstacle information of each obstacle of each historical frame image.

[0092] In some specific implementations, the lane line information can be represented in a tensor, such as tensor Lane, the pedestrian crossing information can be represented in a tensor, such as tensor Stop, and the stop line information can be represented in a tensor, such as tensor Crosswalk.

[0093] The tensors Stop, Lane, and Crosswalk are all tensors with a size of [batch, poly_num, vector_num, feature2]. Wherein, batch is the amount of training data used in one training process; poly_num is the number of lane lines / pedestrian crossings / stop lines; vector_num is the number of line segments included in the lane lines / pedestrian crossings / stop lines; and feature2 is the coordinate information of the lane lines / pedestrian crossings / stop lines, the corresponding traffic light information, etc.

[0094] In some possible implementations, the target model includes a first encoder, a second encoder, a third encoder, and a fourth encoder.

[0095] The first encoder is configured to obtain ego motion features and obstacle motion features according to historical motion information and obstacle motion information; the second encoder is configured to obtain road topology features according to road topology information; the third encoder is configured to obtain map interaction features according to the ego motion features and the road topology features; and the fourth encoder is configured to obtain a scene representation tensor according to the map interaction features and the obstacle motion features.

[0096] In some possible implementations, the first encoder is a Transformer Encoder (a deep self-attention transformation network encoder), the second encoder is a CNN (convolutional neural network), and in some specific implementations, the CNN includes a one-dimensional CNN, a Max Pooling (maximum pooling layer) and a Mean Pooling (mean pooling layer) connected in sequence; the third encoder is an MCG (Multi-context Gating) structure; and the fourth encoder is also a Transformer Encoder.

[0097] Compared with other RNN (Recurrent Neural Network, recurrent network) networks, the Transformer Encoder can focus on different positions of an image frame to obtain more features.

[0098] The MCG structure is a lighter structure compared with the Transformer Encoder, and has a small complexity, so that the use of the MCG structure occupies less resources and can speed up the training and inference of the target model.

[0099] In some specific implementations, the Transformer Encoder is used to extract features from the tensor Obstacle to obtain a tensor with a size of [batch, obs_num, feature], wherein the first feature is the ego motion feature, which is a tensor Adc with a size of [batch, feature], and the other features are obstacle motion features, which are a tensor Obs with a size of [batch, obs_num-1, feature]. The feature is the output dimension of the Transformer Encoder.

[0100] Since the number of lane lines / pedestrian crosswalks / stop lines in the road is large, poly_num and vector_num are large, therefore, one-dimensional CNN connected in sequence, Max Pooling, and Mean Pooling are used to encode tensors Stop, tensors Lane, and tensors Crosswalk, extract features, and obtain tensors Stop, tensors Lane, and tensors Crosswalk with a size of [batch, poly_num, feature4]. feature4 is the output dimension of one-dimensional CNN connected in sequence, Max Pooling, and Mean Pooling.

[0101] The MCG structure is used to learn the interaction features of tensors Adc and tensors Stop, tensors Lane, and tensors Crosswalk, and obtain tensors Scene_Map (i.e., map interaction features) with a size of [batch, poly_num, feature].

[0102] Since tensors Adc are the ego motion features of the driving vehicle, and tensors Stop, tensors Lane, and tensors Crosswalk are road topology features, the MCG structure actually learns the interaction features of the driving vehicle and the road, and therefore, tensors Scene_Map are map semantic enhanced map interaction features.

[0103] The Transformer Encoder is used to learn the interaction features between tensors Scene_Map and tensors Obs, and output tensors Scene with a size of [batch, feature], which is the combination of the scene representation tensors corresponding to batch training data.

[0104] Since tensors Obs are obstacle motion features of obstacles, the Transformer Encoder actually learns the interaction features with obstacles, and therefore, tensors Scene are further semantic enhanced scene representation tensors.

[0105] In some possible implementations, in step S120, positive example pairs and negative example pairs can be constructed according to the distance between the future trajectories of the two training data.

[0106] Figure 2 A flowchart of the specific steps of constructing positive example pairs and negative example pairs according to the distance between the future trajectories of the two training data in the model training method of the present disclosure. Referring to Figure 2 , the positive example pairs and the negative example pairs can be constructed according to the distance between the future trajectories of the two training data, which can include:

[0107] In step S210, two training data are selected from the training data set as candidate training data;

[0108] In step S220, the trajectory distance of the future trajectories of the two candidate training data is calculated;

[0109] In step S230, if the trajectory distance is less than the preset threshold, the two candidate training data are determined as a positive example pair of the contrast learning task;

[0110] In step S240, if the trajectory distance is not less than the preset threshold, the two candidate training data are determined as a negative example pair of the contrast learning task.

[0111] In some possible implementation ways, in step S210, the training data set can be a set composed of training data used in a training process.

[0112] In some possible implementation ways, selecting two training data as candidate training data can be selecting training data in the training data set in order, such as determining the first and second training data in the training data set as candidate training data, determining the first and third training data in the training data set as candidate training data, and so on.

[0113] In some possible implementation ways, in step S220, calculating the trajectory distance of the future trajectories of the two candidate training data can be calculating the Euclidean distance of the future trajectories of the two candidate training data.

[0114] In some possible implementation ways, in step S230, if the two candidate training data are determined as a positive example pair of the contrast learning task, the "label" corresponding to the positive example pair is set to 1.

[0115] In some possible implementation ways, in step S240, if the two candidate training data are determined as a negative example pair of the contrast learning task, the "label" corresponding to the negative example pair is set to -1.

[0116] It should be emphasized that step S230 and step S240 are not in a sequential execution relationship, but in a parallel execution relationship.

[0117] If the similarity of two scene representation tensors is close, it indicates that the similarity of the driving scenes corresponding to the two scene representation tensors is also close, and the future trajectories of the driving vehicle in the two driving scenes should also be similar.

[0118] Therefore, the positive example pair and the negative example pair of the contrast learning task can be constructed according to the future trajectories. That is, two training data with similar future trajectories are taken as a positive example pair, and two training data with distant future trajectories are taken as a negative example pair.

[0119] In some possible implementation manners, in step S130, the target model can be trained by contrastive learning using the loss function.

[0120]

[0121] wherein N is consistent with the value of the batch, and is the amount of training data used in a training process.

[0122] Element is a label matrix, wherein is -1, indicating that the training data i and the training data j are a negative example pair, and is 1, indicating that the training data i and the training data j are a positive example pair.

[0123] The sim function is a function for calculating the cosine similarity between two tensors. Max is a maximum value function.

[0124] is a similarity threshold between the training data i and the training data j, wherein, when the training data i and the training data j are a negative example pair, is 0, when the training data i and the training data j are a positive example pair, is 1.

[0125] is the scene representation tensor Scene output by the target model after the training data is input into the target model, and is the scene identification tensor corresponding to . is the scene representation tensor Scene output by the target model after the training data is input into the target model, and is the scene representation tensor corresponding to .

[0126] In some possible implementation manners, the contrastive learning training of the target model can be that the training data is input into the target model to obtain the tensor Scene. Then, the training data is grouped in pairs, and a label matrix L is calculated according to the Euclidean distance of the future trajectories of the training data in pairs, wherein the size of the label matrix L is [N, N], and the element is -1, indicating that the training data i and the training data j are a negative example pair (i.e., the training data i and the training data j are dissimilar scenes, and the Euclidean distance of the future trajectories corresponding thereto is not less than a preset threshold), and the element is 1, indicating that the training data i and the training data j are a positive example pair or 1, indicating that the training data i and the training data j are positive samples (i.e., the training data i and the training data j are similar scenes, and the Euclidean distance of the future trajectories corresponding thereto is less than a preset threshold). The target model is trained by contrastive learning according to the label matrix L and the above loss function.

[0127] In some possible implementation manners, the fourth encoder is further configured to obtain an obstacle representation tensor of the obstacle of the driving scene according to the map interaction feature and the obstacle motion feature.

[0128] In some possible implementation manners, the interaction feature between the tensor Scene Map and the tensor Obs is learned using a Transformer Encoder, and an obstacle representation tensor Obs semantic with a size of [batch, obs_num-1, feature] can also be output.

[0129] Since the tensor Obs semantic is also obtained according to the tensor Scene Map and the tensor Obs, a semantic-enhanced obstacle representation tensor is also obtained.

[0130] In some possible implementation manners, the target model further includes a prediction network configured to predict a probability that the obstacle is a key obstacle according to the obstacle representation tensor.

[0131] In some possible implementation manners, the prediction network can be an MLP (Multi-Layer Perceptron).

[0132] In some possible implementation manners, the target model can also be supervised trained. Figure 3 For a flowchart of the step of supervising training of the target model, refer to Figure 3 Supervising training of the target model can include:

[0133] In step S310, an obstacle label corresponding to the training data is determined.

[0134] In step S320, the target model is supervised trained based on an output of the prediction network and the obstacle label.

[0135] The obstacle label is used to identify whether the obstacle of the driving scene is a key obstacle.

[0136] In some possible implementation manners, in step S310, an obstacle label corresponding to each obstacle in the training data is determined.

[0137] In some possible implementation manners, the key obstacle is an obstacle that has a greater impact on driving of the driving vehicle.

[0138] In some possible implementation manners, the obstacle label corresponding to the training data can be determined through a future trajectory of the training data.

[0139] In some specific implementations, the at least one candidate key obstacle is determined according to the future trajectory of the training data; for each candidate key obstacle, a planned trajectory of driving the vehicle after the candidate key obstacle is removed from the driving scene is obtained through simulation; and whether the candidate key obstacle is a key obstacle is determined according to the trajectory distance between the planned trajectory and the future trajectory.

[0140] In some specific implementations, the at least one candidate key obstacle is determined according to the future trajectory of the training data; for each candidate key obstacle, a planned trajectory of driving the vehicle after the candidate key obstacle is removed from the driving scene is obtained through simulation; and whether the candidate key obstacle is a key obstacle is determined according to the trajectory distance between the planned trajectory and the future trajectory.

[0141] In some possible implementations, if the candidate key obstacle is a key obstacle, the obstacle label corresponding to the candidate key obstacle is 1, and the obstacle label corresponding to other obstacles is 0.

[0142] In some possible implementations, in step S320, the target model can be supervised trained using the loss function.

[0143]

[0144] In some specific implementations, N is consistent with the value of the batch, and is the amount of training data used in one training process. The number of obstacles in one training data is obs_num-1.

[0145] Yij is the obstacle label corresponding to the jth obstacle of the training data i, and 0 indicates that the obstacle is a non-key obstacle, and 1 indicates that the obstacle is a key obstacle. Pij is the probability of the obstacle being a key obstacle output by the prediction network.

[0146] In some possible implementations, the supervised learning training of the target model can be that the training data is input into the target model to obtain a tensor Obs_semantic, the Obs_semantic is input into the MLP to obtain the probability of the obstacle being a key obstacle, and the target model is supervised learning trained according to the obstacle label corresponding to the obstacle and the above loss function.

[0147] By setting the prediction network, whether the obstacle is a key obstacle can be obtained, and the importance information of the obstacle is provided, so that more effective information can be provided for the downstream task. For example, for the planning module, the model training method provided by the embodiment of the disclosure can provide the importance weight of the surrounding obstacle, which can help the planning module to effectively filter out the key obstacle from a large number of traffic participants, reduce the calculation complexity of the module, and improve the calculation accuracy.

[0148] The current experimental results also prove the effectiveness of the model training method provided by the embodiments of the present disclosure. The importance weight of the obstacle learned using the contrast learning method tends to be uniformly distributed, and the weight ranking of the key obstacle is intelligently arranged at the median. The average ranking of the weight of the obstacle output by the model trained by the model training method provided by the embodiments of the present disclosure is 3.5, which shows strong explainability.

[0149] In some possible implementation manners, the target model can be subjected to joint training of supervised training and contrast learning training.

[0150] In some possible implementation manners, the training weight of the supervised training and the training weight of the contrast learning training are determined according to a preset balance factor; the target model is subjected to supervised training according to the training weight of the supervised training; and the target model is subjected to contrast learning training according to the training weight of the contrast learning training.

[0151] In some specific implementation manners, the loss function can be used to jointly train the target model.

[0152]

[0153] wherein, is a loss function of supervised training, is a loss function of contrast learning training. is a balance factor, which is used to balance the learning relationship of the above two objective functions, so that the model can learn the explainable scene representation and the explainable weight of the surrounding obstacles.

[0154] Figure 4 is a schematic diagram of a specific structure of a target model in the model training method of the embodiments of the present disclosure. Refer to Figure 4 The input information at the bottom is a tensor Obstacle ( Figure 4 on the left side) and tensors Lane, Stop and Crosswalk ( Figure 4 on the right side).

[0155] The tensor Obstacle is encoded using a Transformer Encoder to obtain a tensor Adc and a tensor Obs; the tensors Stop, Lane and Crosswalk are encoded using Conv1d (one-dimensional CNN) + Maxpool1d (maximum pooling) + Meanpool1d (average pooling) to obtain the encoded tensors Stop, Lane and Crosswalk.

[0156] The MCG structure is used to learn the interaction features of tensor Adc, tensor Stop, tensor Lane, and tensor Crosswalk. After passing through Cat (channel dimension concatenation) and MLP, tensor Scene_Map is obtained. After passing through the tensor Scene_Map and tensor Obs Cat (channel dimension concatenation), Transformer Encoder is used to learn the interaction features between tensor Scene_Map and tensor Obs. After passing through Scene Coding, the scene representation tensor and obstacle representation tensor Obs_semantic are output. Then, a prediction network is used to predict whether the obstacle is a critical obstacle (output 1 if the obstacle is a critical obstacle, output 0 if the obstacle is not a critical obstacle).

[0157] Then, the MCG module interacts with the main vehicle (ADC) and the map features, and finally interacts with the surrounding obstacles through the Transformer Encoder to obtain the final scene representation and obstacle importance weights.

[0158] Figure 5 A flowchart illustrating an information acquisition method provided in an embodiment of this disclosure is shown, such as... Figure 5 As shown, the method can mainly include:

[0159] In step S510, the historical motion information of the driving vehicle, the obstacle motion information of the obstacles in the driving scene where the driving vehicle is located, and the road topology information of the road in the driving scene are obtained.

[0160] In step S520, historical motion information, obstacle motion information, and road topology information are input into a pre-trained model to obtain the scene representation tensor of the driving scene.

[0161] The pre-trained model is obtained by training using the model training method provided in the embodiments of this disclosure.

[0162] In some possible implementations, the reasoning process of the information acquisition method provided in this disclosure embodiment is as follows: Figure 4 As shown, refer to Figure 4 The input information at the bottom is the tensor Obstacle ( Figure 4 (left side of the text) and tensor Lane, tensor Stop, tensor Crosswalk ( Figure 4 (The right side of the middle).

[0163] Among them, the tensor Obstacle is obtained based on the historical motion information of the driving vehicle and the obstacle motion information of the obstacles in the driving scene. The tensors Lane, Stop, and Crosswalk are obtained based on the road topology information of the road in the driving scene.

[0164] The information of the tensor Obstacle is encoded using the Transformer Encoder to obtain the tensor Adc and the tensor Obs; the tensor Stop, the tensor Lane, and the tensor Crosswalk are encoded using Conv1d (one-dimensional CNN) + Maxpool1d (maximum pooling) + Meanpool1d (average pooling) to obtain the encoded tensor Stop, the encoded tensor Lane, and the encoded tensor Crosswalk.

[0165] The interaction features of the tensor Adc and the tensor Stop, the tensor Lane, and the tensor Crosswalk are learned using the MCG structure, and the tensor Scene_Map is obtained through Cat (channel dimension splicing) and MLP. After the tensor Scene_Map and the tensor Obs are concatenated, the interaction features between the tensor Scene_Map and the tensor Obs are learned using the Transformer Encoder, and the scene representation tensor is output after scene coding. The obstacle representation tensor Obs_semantic is output, and whether the obstacle is a key obstacle is predicted through the prediction network (1 is output if the obstacle is a key obstacle, and 0 is output if the obstacle is not a key obstacle).

[0166] Then, the host vehicle (ADC) interacts with the Map features through the MCG module, and finally interacts with the surrounding obstacles through the Transformer Encoder to obtain the final scene representation and the importance weight of the obstacle.

[0167] In the information acquisition method of the embodiment of the present disclosure, the pre-trained model is trained by the model training method provided by the embodiment of the present disclosure, and the model training method provided by the embodiment of the present disclosure is to construct positive and negative pairs of contrast learning tasks according to known future trajectories in the training data, which is equivalent to converting self-supervised contrast learning into supervised learning using a determined label. Compared with self-supervised contrast learning, supervised learning has stronger ability to learn the features of the training data, and the learned representation is more interpretable. The model effect of the target model constructed through supervised learning is better, and the scene representation tensor obtained through the trained target model is also more interpretable. Therefore, the scene representation tensor obtained in the information acquisition method of the embodiment of the present disclosure is also a scene representation tensor with stronger interpretability, which can provide more effective scene information for downstream tasks.

[0168] Based on the same principle as the method shown in Figure 1 Figure 6 A structural schematic diagram of a model training device provided by the embodiment of the present disclosure is shown, as​Figure 6 As shown, the model training device 70 may include:

[0169] Data module 610 is used to acquire training datasets. The training datasets include multiple training data, each of which includes the historical motion information of the driving vehicle, the obstacle motion information of the obstacles in the driving scene where the driving vehicle is located, the road topology information of the road in the driving scene, and the future trajectory of the driving vehicle.

[0170] The label module 620 is used to construct positive and negative pairs for the contrastive learning task based on the training dataset and future trajectories.

[0171] Training module 630 is used to perform contrastive learning training on the target model based on positive and negative pairs of contrastive learning tasks. The target model is used to obtain the scene representation tensor of the driving scene based on historical motion information, obstacle motion information, and road topology information.

[0172] In the model training apparatus disclosed herein, positive and negative pairs of contrastive learning tasks are constructed based on known future trajectories in the training data. This is equivalent to converting self-supervised contrastive learning into supervised learning using defined labels. Compared to self-supervised contrastive learning, supervised learning is obviously more capable of learning features from the training data, and the representations it learns are more interpretable. Therefore, the target model constructed through supervised learning performs better, and the scene representation tensor obtained through the trained target model is also more interpretable.

[0173] It is understood that the above-described modules of the model training apparatus in the embodiments of this disclosure have the ability to implement... Figure 1 The embodiments shown illustrate the functionality of the corresponding steps in the model training method. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions. These modules can be software and / or hardware, and each module can be implemented individually or multiple modules can be integrated. For a detailed description of the functions of each module in the above-described model training device, please refer to [link to relevant documentation]. Figure 1 The corresponding descriptions of the model training methods in the embodiments shown are not repeated here.

[0174] Based on and Figure 5 The method shown follows the same principle. Figure 7 A schematic diagram of the structure of an information acquisition device provided in an embodiment of this disclosure is shown, such as... Figure 7 As shown, the information acquisition device 80 may include:

[0175] Information module 710 is used to acquire historical motion information of the driving vehicle, obstacle motion information of obstacles in the driving scene where the driving vehicle is located, and road topology information of the road in the driving scene;

[0176] predicting the future trajectory of the vehicle in the driving scene.

[0177] The pre-trained model is trained by using the model training method provided in the embodiments of the present disclosure.

[0178] In the information acquisition device of the present disclosure, the pre-trained model is trained by using the model training method provided in the embodiments of the present disclosure, and the model training method provided in the embodiments of the present disclosure is to construct positive and negative pairs of the contrast learning task according to the known future trajectory in the training data, which is equivalent to converting the self-supervised contrast learning into supervised learning using a determined label. Compared with the self-supervised contrast learning, the supervised learning has a stronger ability to learn the characteristics of the training data, and the learned representation is more interpretable. The model effect of the target model constructed by the supervised learning is better, and the scene representation tensor obtained by the trained target model is also more interpretable. Therefore, the scene representation tensor obtained in the information acquisition device of the embodiments of the present disclosure is also a scene representation tensor with stronger interpretability, which can provide more effective scene information for the downstream tasks.

[0179] It can be understood that each module of the information acquisition device in the embodiments of the present disclosure has the function of the corresponding steps of the information acquisition method in the embodiments shown in the Figure 5 The function can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and each module can be realized independently or integrated. For the function description of each module of the information acquisition device, please refer to the corresponding description of the information acquisition method in the embodiments shown in the Figure 1 The present disclosure provides an information acquisition device, a model training method and a model training device.

[0180] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations, and do not violate public order and good customs.

[0181] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0182] The electronic device includes at least one processor, and a memory connected with the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the model training method and the information acquisition method provided in the embodiments of the present disclosure.

[0183] Compared with the prior art, the electronic device constructs positive example pairs and negative example pairs of the contrast learning task according to the known future trajectory in the training data, which is equivalent to converting the self-supervised contrast learning into supervised learning using a determined label. Compared with the self-supervised contrast learning, the supervised learning has a stronger ability to learn the features of the training data, and the learned representation is more interpretable. Therefore, the model effect of the target model constructed through the supervised learning is better, and the scene representation tensor obtained through the trained target model is also more interpretable.

[0184] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to perform the model training method and the information acquisition method provided in the embodiments of the present disclosure.

[0185] Compared with the prior art, the readable storage medium constructs positive example pairs and negative example pairs of the contrast learning task according to the known future trajectory in the training data, which is equivalent to converting the self-supervised contrast learning into supervised learning using a determined label. Compared with the self-supervised contrast learning, the supervised learning has a stronger ability to learn the features of the training data, and the learned representation is more interpretable. Therefore, the model effect of the target model constructed through the supervised learning is better, and the scene representation tensor obtained through the trained target model is also more interpretable.

[0186] The computer program product includes a computer program, and the computer program, when executed by a processor, implements the model training method and the information acquisition method provided in the embodiments of the present disclosure.

[0187] Compared with the prior art, the computer program product constructs positive example pairs and negative example pairs of the contrast learning task according to the known future trajectory in the training data, which is equivalent to converting the self-supervised contrast learning into supervised learning using a determined label. Compared with the self-supervised contrast learning, the supervised learning has a stronger ability to learn the features of the training data, and the learned representation is more interpretable. Therefore, the model effect of the target model constructed through the supervised learning is better, and the scene representation tensor obtained through the trained target model is also more interpretable.

[0188] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device and an autonomous vehicle.

[0189] The electronic device comprises at least one processor and a memory connected with the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform path planning by using a scene representation tensor of a driving scene where a driving vehicle is located, which is acquired by the information acquisition method provided in the embodiments of the present disclosure, and to perform automatic driving control on the driving vehicle according to a path planning result.

[0190] Compared with the prior art, the electronic device performs path planning by using a scene representation tensor of a driving scene where a driving vehicle is located, which is acquired by the information acquisition method provided in the embodiments of the present disclosure. Since the information acquisition method provided in the embodiments of the present disclosure constructs positive example pairs and negative example pairs of a contrast learning task according to known future trajectories in training data, it is equivalent to converting self-supervised contrast learning into supervised learning using a determined label. Compared with self-supervised contrast learning, supervised learning has a stronger ability to learn features of training data, and the learned representation is more interpretable. Therefore, the model effect of a target model constructed by supervised learning is better, and the scene representation tensor acquired by the trained target model is also more interpretable.

[0191] Therefore, the electronic device provided in the embodiments of the present disclosure can also acquire more effective scene information and importance weights of surrounding obstacles, thereby effectively filtering out key obstacles from numerous traffic participants, reducing the computational complexity of the electronic device, improving the computational accuracy of the electronic device, improving the effectiveness and accuracy of path planning, and further improving the effectiveness and accuracy of automatic driving control, and improving the driving efficiency and safety of the driving vehicle.

[0192] The autonomous driving vehicle comprises the electronic device as described in the embodiments of the present disclosure.

[0193] Compared with the prior art, the autonomous driving vehicle performs path planning by using a scene representation tensor of a driving scene where the autonomous driving vehicle is located, which is acquired by the information acquisition method provided in the embodiments of the present disclosure. Since the information acquisition method provided in the embodiments of the present disclosure constructs positive example pairs and negative example pairs of a contrast learning task according to known future trajectories in training data, it is equivalent to converting self-supervised contrast learning into supervised learning using a determined label. Compared with self-supervised contrast learning, supervised learning has a stronger ability to learn features of training data, and the learned representation is more interpretable. Therefore, the model effect of a target model constructed by supervised learning is better, and the scene representation tensor acquired by the trained target model is also more interpretable.

[0194] Therefore, the automatic driving vehicle provided by the embodiments of the present disclosure can obtain more effective scene information and importance weights of surrounding obstacles, so as to effectively filter out key obstacles from a large number of traffic participants, reduce the calculation complexity of the electronic device of the automatic driving vehicle, improve the calculation accuracy of the electronic device of the automatic driving vehicle, improve the effectiveness and accuracy of path planning, and further improve the effectiveness and accuracy of automatic driving control, and improve the driving efficiency and safety of the automatic driving vehicle.

[0195] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0196] As shown in Figure 8 The electronic device 800 includes a computing unit 810 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 820 or a computer program loaded from a storage unit 880 into a random access memory (RAM) 830. Various programs and data required for the operation of the device 800 can also be stored in the RAM 830. The computing unit 810, the ROM 820, and the RAM 830 are connected to each other through a bus 840. An input / output (I / O) interface 850 is also connected to the bus 840.

[0197] Various components in the device 800 are connected to the I / O interface 850, including an input unit 860, such as a keyboard, a mouse, etc., an output unit 870, such as various types of displays, speakers, etc., a storage unit 880, such as a magnetic disk, an optical disk, etc., and a communication unit 890, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 890 allows the device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0198] The computing unit 810 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 810 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 810 performs the model training method and the information acquisition method provided in the embodiments of the present disclosure. For example, in some embodiments, the model training method and the information acquisition method provided in the embodiments of the present disclosure can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 880. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 820 and / or the communication unit 890. When the computer program is loaded onto the RAM 830 and executed by the computing unit 810, one or more steps of the model training method and the information acquisition method provided in the embodiments of the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 810 can be configured to perform the model training method and the information acquisition method provided in the embodiments of the present disclosure by any other suitable means, such as by means of firmware.

[0199] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0200] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0201] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0202] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0203] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0204] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0205] It should be understood that the various forms of flow shown above can be re-ordered, added to, or have steps deleted, using the steps described above. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure can be achieved, which is not limited herein.

[0206] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A model training method, comprising: obtaining a training data set, the training data set comprising a plurality of training data, each of the training data comprising historical motion information of a driving vehicle, obstacle motion information of an obstacle of a driving scene in which the driving vehicle is located, road topology information of a road of the driving scene, a future trajectory of the driving vehicle; constructing, based on the training data set, positive example pairs and negative example pairs of a contrastive learning task according to the future trajectory; performing contrastive learning training on a target model based on the positive example pairs and the negative example pairs of the contrastive learning task, the target model being configured to obtain a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information, and the road topology information, the target model further comprising a prediction network configured to predict a probability that an obstacle is a key obstacle; determining an obstacle label corresponding to the training data, the obstacle label being configured to identify whether the obstacle of the driving scene is a key obstacle; performing supervised training on the target model based on an output of the prediction network and the obstacle label; wherein the determining of the obstacle label corresponding to the training data comprises: determining at least one candidate key obstacle according to the future trajectory of the training data; for each candidate key obstacle, obtaining a planned trajectory of the driving vehicle after the candidate key obstacle is removed from the driving scene through simulation; determining whether the candidate key obstacle is a key obstacle according to a trajectory distance between the planned trajectory and the future trajectory.

2. The method of claim 1, wherein, The constructing, based on the training data set, positive example pairs and negative example pairs of a contrastive learning task according to the future trajectory comprises: selecting two training data from the training data set as candidate training data; calculating a trajectory distance of the future trajectories of the two candidate training data; in a case where the trajectory distance is less than a preset threshold, determining that the two candidate training data are positive example pairs of the contrastive learning task; in a case where the trajectory distance is not less than the preset threshold, determining that the two candidate training data are negative example pairs of the contrastive learning task.

3. The method of claim 1, wherein, The target model comprises a first encoder, a second encoder, a third encoder, and a fourth encoder; the first encoder is configured to obtain a self-vehicle motion feature and an obstacle motion feature according to the historical motion information and the obstacle motion information; the second encoder is configured to obtain a road topology feature according to the road topology information; the third encoder is configured to obtain a map interaction feature according to the self-vehicle motion feature and the road topology feature; the fourth encoder is configured to obtain the scene representation tensor according to the map interaction feature and the obstacle motion feature.

4. The method of claim 3, wherein, The fourth encoder is further configured to obtain an obstacle representation tensor of the obstacle of the driving scene according to the map interaction feature and the obstacle motion feature; the prediction network is configured to predict the probability that the obstacle is a key obstacle according to the obstacle representation tensor.

5. The method of claim 1, wherein, The performing contrastive learning training on the target model and the performing supervised training on the target model comprise: determine a training weight of the supervised training and a training weight of the contrastive learning training according to a preset balance factor; perform supervised training on the target model according to the training weight of the supervised training; perform contrastive learning training on the target model according to the training weight of the contrastive learning training.

6. The method of claim 3, wherein, The first encoder is a deep self-attention transformer encoder, the second encoder is a convolutional neural network, the third encoder is a multi-context gate structure, and the fourth encoder is a deep self-attention transformer encoder.

7. The method of claim 3, wherein, The road topology information includes at least one of lane line information, pedestrian crossing information, and stop line information of a road of the driving scene, and the road topology feature includes at least one of lane line features, pedestrian crossing features, and stop line features.

8. An information acquisition method, comprising: acquiring historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, and road topology information of a road of the driving scene; inputting the historical motion information, the obstacle motion information, and the road topology information into a pre-trained pre-training model to acquire a scene representation tensor of the driving scene; wherein the pre-trained pre-training model is trained by the model training method of any one of claims 1-7.

9. A model training apparatus, comprising: a data module configured to acquire a training data set, the training data set comprising a plurality of training data, each of the training data comprising historical motion information of a driving vehicle, obstacle motion information of an obstacle in a driving scene where the driving vehicle is located, road topology information of a road of the driving scene, and a future trajectory of the driving vehicle; a label module configured to construct positive and negative pairs of a contrastive learning task based on the training data set according to the future trajectory; a training module configured to perform contrastive learning training on a target model based on the positive and negative pairs of the contrastive learning task, the target model being configured to acquire a scene representation tensor of the driving scene according to the historical motion information, the obstacle motion information, and the road topology information, the target model further comprising a prediction network configured to predict a probability that an obstacle is a key obstacle; the label module is further configured to determine an obstacle label corresponding to the training data, the obstacle label being configured to identify whether an obstacle in the driving scene is a key obstacle; the training module is further configured to perform supervised training on the target model based on an output of the prediction network and the obstacle label; wherein the determination of the obstacle label corresponding to the training data comprises: determining at least one candidate key obstacle according to the future trajectory of the training data; for each candidate key obstacle, acquiring a planned trajectory of the driving vehicle after the candidate key obstacle is removed from the driving scene through simulation; determining whether the candidate key obstacle is a key obstacle according to a trajectory distance between the planned trajectory and the future trajectory.

10. An information acquisition apparatus, comprising: an information module configured to obtain historical motion information of a driven vehicle, obstacle motion information of obstacles in a driving scene in which the driven vehicle is located, and road topology information of a road in the driving scene; a prediction module configured to input the historical motion information, the obstacle motion information, and the road topology information into a pre-trained model to obtain a scene representation tensor of the driving scene; wherein the pre-trained model is trained by the model training method of any one of claims 1-7. 11.An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the model training method of any one of claims 1-7 or the information obtaining method of claim 8.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, the computer instructions are used to enable the computer to perform the model training method of any one of claims 1-7 or the information obtaining method of claim 8. 13.A computer program product, comprising a computer program, wherein the computer program, when executed by a processor, implements the model training method of any one of claims 1-7 or the information obtaining method of claim 8. 14.An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform path planning using a scene representation tensor of a driving scene in which a driven vehicle is located, which is obtained by the information obtaining method of claim 8, and to perform automatic driving control of the driven vehicle according to a path planning result. 15.An autonomous driving vehicle, comprising the electronic device of claim 14.

Images