Method for processing vehicle driving information

Abstract

The present application provides a method for processing vehicle driving information, the method comprising: obtaining relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene; generating scene features for characterizing the current traffic scene according to the relevant data corresponding to the multiple dynamic objects and scene maps; extracting self-vehicle features related to the target vehicle from the scene features, and non-self-vehicle features related to other dynamic objects other than the target vehicle in the multiple dynamic objects; inputting the self-vehicle features, non-self-vehicle features and navigation data into a prediction model to obtain the first driving route of the target vehicle generated by the prediction model. According to the embodiment of the present application, the result output in the autonomous driving scene is faster, and the driving safety is guaranteed; not only the efficient operation of the prediction process is guaranteed, and the computing power requirements for the vehicle-side chip are reduced; but also the naturalness of the prediction results is improved, making the prediction results richer.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method for processing vehicle driving information. Background Art

[0002] With the development of artificial intelligence technology and modern manufacturing, autonomous driving technology has gradually entered people's daily lives, subtly changing people's travel methods. Autonomous driving technology has great application prospects in both military and civilian applications. In military applications, it can not only be used as an unmanned transportation platform, but also as an unmanned blasting vehicle, unmanned combat platform, unmanned patrol and surveillance vehicle, etc. In civilian applications, in addition to bringing convenience to human life, it can also reduce the incidence of traffic accidents and improve road traffic efficiency, but it also causes a sharp increase in energy consumption and frequent traffic accidents. The update and iteration of autonomous driving technology is the key to solving the above problems. Therefore, autonomous driving technology has become the focus of attention in the automotive industry and academia.

[0003] In order to further improve the safety of autonomous driving, one of the most important technologies is that autonomous vehicles should be able to predict the future state of the surrounding environment in real time like human drivers, and make reasonable decisions and plans in advance based on the prediction results to avoid potential dangers. The accuracy of the current prediction results needs to be further improved. Summary of the invention

[0004] The embodiments of the present application provide a method for processing vehicle driving information, an electronic device, a storage medium and a computer program product to achieve one or more of the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for processing vehicle driving information, including:

[0006] Obtain relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene;

[0007] Generate scene features for representing the current traffic scene according to relevant data corresponding to the multiple dynamic objects and the scene map;

[0008] Extracting ego-vehicle features related to the target vehicle and non-ego-vehicle features related to other dynamic objects except the target vehicle among the multiple dynamic objects from the scene features;

[0009] The vehicle characteristics, non-vehicle characteristics and navigation data are input into the prediction model to obtain a first driving route of the target vehicle generated by the prediction model; the prediction model includes a first prediction head for planning a second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, and the first driving route is generated based on the second driving route and the motion trajectory of other dynamic objects.

[0010] In a second aspect, an embodiment of the present application provides a model training method for vehicle driving route prediction, comprising:

[0011] Obtain relevant data samples corresponding to multiple dynamic objects and scene maps in multiple historical traffic scenes;

[0012] A prediction model is trained based on relevant data samples. The prediction model is used to predict a first driving route of the target vehicle based on self-vehicle features related to the target vehicle, non-self-vehicle features related to other dynamic objects among multiple dynamic objects except the target vehicle, and navigation data. The prediction model includes a first prediction head for planning a second driving route of the target vehicle and a second prediction head for predicting motion trajectories of other dynamic objects. The first driving route is generated based on the second driving route and motion trajectories of other dynamic objects. Self-vehicle features and non-self-vehicle features are extracted from scene features. The scene features are generated based on relevant data corresponding to multiple dynamic objects and scene maps.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, any of the above methods is implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the method provided in any embodiment of the present application is implemented.

[0016] Compared with the prior art, this application has the following advantages:

[0017] In an embodiment of the present application, by acquiring relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene, scene features for characterizing the current traffic scene are generated based on the relevant data corresponding to the multiple dynamic objects and scene maps, and self-vehicle features related to the target vehicle and non-self-vehicle features related to other dynamic objects among the multiple dynamic objects except the target vehicle are extracted from the scene features; finally, the self-vehicle features, non-self-vehicle features and navigation data are input into the prediction model to obtain the first driving route of the target vehicle generated by the prediction model.

[0018] Among them, the prediction model includes a first prediction head for planning the second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, and the first driving route is generated based on the second driving route and the motion trajectory of other dynamic objects. It can be seen that the present application unifies the trajectory prediction of non-self-vehicle objects and the motion trajectory planning of the self-vehicle in one model, so that the model can simultaneously output two prediction results of trajectory prediction of non-self-vehicle objects and motion trajectory planning of the self-vehicle, without calling two models separately, so that the result output in the autonomous driving scene is faster and more timely, effectively ensuring the safety of driving; it also helps to learn the interactive behavior of non-self-vehicle objects and self-vehicle behaviors, and reduce unreasonable conflicts (such as collisions) between the predicted trajectory of non-self-vehicles and the planned trajectory of self-vehicles. In addition, the tasks of multiple prediction heads can promote each other during the training process, making the prediction results more accurate, improving the efficiency of model training, and reducing the computing power requirements for vehicle-side chips in the field of autonomous driving.

[0019] In addition, after obtaining the corresponding relevant data of multiple dynamic objects in the current traffic scene, a new spatiotemporal feature fusion technology is used to fuse the features of the time dimension, compress the time features to one dimension, and then perform spatial dimension fusion features on different dynamic object information, so that the information of the time dimension is retained as much as possible. This is because compressing and fusing the time dimension from T dimensions to one dimension during processing can significantly reduce the dimension of the data while retaining the most important information, further ensuring the efficient operation of the prediction process, avoiding time delays caused by information processing, reducing the computing power requirements for the vehicle-side chip, and having wider adaptability. Feature cross-processing can fully fuse the information of dynamic objects in multiple dimensions. After fusion, the dynamic object features corresponding to each dimension contain other dynamic object features, so that each dimension can represent the features of multiple dynamic objects.

[0020] In addition, the present application designs a new type of prior motion mode design idea for the self-vehicle motion planning. When predicting the self-vehicle, the motion mode corresponding to the specific semantic information is used, so that the output trajectory planning is interpretable and can be used directly. When predicting non-self-vehicles, the motion mode that does not represent the specific semantics is used with the corresponding prior value. Since the prior value (i.e., random number) used is not limited to statistical data, the prediction results can be richer, which can not only better align human driving behavior, improve the naturalness and acceptability of planning, but also be applicable to more driving styles. When predicting the self-vehicle, you can also mix motion modes with specific semantics and motion modes with corresponding prior values ​​that do not represent specific semantics. The advantage of this is that, under the premise of satisfying the interpretability of the output trajectory planning, the naturalness of the prediction results is improved, thereby making the prediction results richer.

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0023] Figure 1 A schematic diagram of an application scenario of the method of an embodiment of the present application;

[0024] Figure 2 A flowchart of a method for processing vehicle driving information according to an embodiment of the present application;

[0025] Figure 3 This is a flow chart of a model training method for vehicle driving route prediction according to an embodiment of the present application;

[0026] Figure 4 is a structural block diagram of a vehicle driving information processing device according to an embodiment of the present application;

[0027] Figure 5 is a structural block diagram of a model training device for vehicle driving route prediction according to an embodiment of the present application; and

[0028] Figure 6 A block diagram of an electronic device used to implement an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0030] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0031] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, without paying creative work, this specification can also be applied to other similar scenarios based on these drawings. Unless it is obvious from the voice environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0032] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0033] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0034] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0035] First of all, before specifically introducing the technical solution of the embodiment of the present disclosure, the technical background or technical evolution context on which the embodiment of the present disclosure is based is introduced. Generally speaking, in the field of autonomous driving, the current technical background is that trajectory prediction and self-vehicle path planning are usually divided into two modules and iterated by different teams. Both trajectory prediction and self-vehicle path planning involve the problem of time series information fusion. The current algorithm either uses a simple maximum pooling of features along the time dimension, or a complex transformer with very large delay. Although the maximum pooling method is simple, it loses a lot of information, which is not conducive to time series learning. The transformer-based design has a relatively large delay, which is not conducive to use on vehicle-side chips with limited computing power.

[0036] In addition, current trajectory prediction and ego-vehicle planning models usually input multiple possible trajectories, representing multiple possible motion modes of the ego-vehicle or other objects in the future, i.e., "multimodality". In order to perform multimodal learning, multiple sub-prediction heads are usually preset (which can be considered as a priori, and different sub-prediction heads represent different motion modes). However, existing algorithms usually obtain a set of motion modes by statistically analyzing a specific data set. This method is not very generalizable and may have poor distribution in new scenarios. In addition, this sub-prediction head does not carry any semantic information (such as acceleration, deceleration, turning, and changing lanes, etc.), which is not conducive to the use of planning systems.

[0037] In view of this, this application is based on the similarities between the two algorithms, that is, the input usually contains scene information (such as maps), the vehicle itself and other dynamic objects that constitute the traffic scene (such as social vehicles, pedestrians, etc.). Improve the algorithm iteration efficiency by unifying the design model. Among them, the trajectory prediction algorithm is a task in the autonomous driving system to predict the position, speed and direction of the vehicle or other objects in the future based on various information about the vehicle's history, such as the vehicle's own position, speed, direction and scene information such as maps in the past second, as well as other vehicle information around the vehicle, such as the front and rear vehicles and social vehicles. The motion path planning algorithm is a task in which the autonomous driving vehicle plans the route of the vehicle in the future based on its own state and scene information.

[0038] In the embodiment of the present application, for the sake of convenience, the target vehicle (i.e., the control vehicle) is recorded as the self-vehicle, and other dynamic objects other than the target vehicle are recorded as non-self-vehicles. Correspondingly, the features related to the target vehicle are recorded as self-vehicle features, and the features of other dynamic objects other than the target vehicle are recorded as non-self-vehicle features.

[0039] Based on the above background, the applicant provides a method for processing vehicle driving information through the embodiments of the present application to solve all or part of the above technical problems.

[0040] The following introduces the technical solutions involved in the embodiments disclosed in this application in combination with the scenarios to which the embodiments disclosed in this application are applied.

[0041] Figure 1 The figure is a schematic diagram of an exemplary application scenario for implementing the method of the embodiment of the present application.

[0042] like Figure 1As shown, the embodiment of the present application can obtain the corresponding object features, map features and navigation features through the encoder respectively based on the relevant data corresponding to the dynamic object, the relevant data corresponding to the scene map and the navigation data; wherein the object features and the map features are fused along the feature dimension to obtain the scene features for characterizing the current traffic scene. The self-vehicle features related to the target vehicle and the non-self-vehicle features related to other dynamic objects other than the target vehicle among the multiple dynamic objects are extracted from the scene features; the self-vehicle features of the multiple dynamic objects are fused with the navigation features of the navigation data, and the first fused feature is used as the input of the first prediction head of the prediction model, and the non-self-vehicle features are used as the input of the second prediction head of the prediction model; the second driving route of the target vehicle is planned by the first prediction head, and the motion trajectory of other dynamic objects is planned by the second prediction head, and the first driving route of the target vehicle is generated according to the second driving route and the motion trajectory of other dynamic objects.

[0043] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0044] like Figure 2 The flowchart of the method for processing vehicle driving information according to an embodiment of the present application may include:

[0045] In step S201, relevant data corresponding to a plurality of dynamic objects and a scene map in the current traffic scene are obtained.

[0046] It is understandable that the vehicle's onboard sensors, radars and other equipment can collect relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene during driving.

[0047] Dynamic objects refer to the target vehicle itself and other traffic participants other than the target vehicle. For example, dynamic objects can be the target vehicle, other vehicles (such as motor vehicles, non-motor vehicles, etc.), pedestrians, animals, etc.

[0048] The relevant data corresponding to the dynamic object is used to reflect the motion information of the dynamic object, such as object category, position, object size, driving direction, and may also include information such as driving time. Object categories include pedestrians, animals, non-motor vehicles, motor vehicles, etc. For the self-vehicle object, the object category of the self-vehicle adopts the pre-defined object category information, the position and object size and driving direction are obtained by the joint control calculation of the sensors around the vehicle itself, and the driving time is intercepted by the vehicle's own built-in clock. For non-self-vehicle objects, the above information can be used to mark the detected non-self-vehicle dynamic objects through the detection frame. For example, the information detected by the detection frame includes coordinates, length, width, and direction, where the coordinates reflect the position of the dynamic object, and the coordinate system may include any one or more of the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system of the detection frame. Length and width refer to the length and width of the detection frame, which reflect the size of the dynamic object. Direction refers to the moving direction of the detection frame, which reflects the moving direction of the dynamic object. For example, when the dynamic object is in a stationary state, the direction of the detection frame can be the direction in which the front of the dynamic object (such as the front of the vehicle, the chest of a pedestrian / animal) is facing. The time is the moment corresponding to when the dynamic object is at the current position or speed, or the moment when the vehicle detects that the dynamic object is in the current state.

[0049] The relevant data corresponding to the scene map includes at least one of lane line position information and road edge position information, such as but not limited to the following categories: road grid, including road geometry, lane lines, intersections, traffic signals and other information; traffic signs and signals, including traffic lights, road signs, speed limit signs, stop signs, etc.; pedestrian and bicycle lanes, including the location and boundaries of pedestrian passages and bicycle lanes; buildings and terrain, including the location, height, shape of buildings and the ups and downs of terrain; parking areas, including public parking lots, private parking spaces, etc.; temporary obstacles, including temporary construction areas, accident scenes, etc.; environmental conditions, including weather conditions, lighting conditions, and road conditions.

[0050] The relevant data corresponding to the scene map can come from a variety of channels, including but not limited to vehicle sensors (such as LiDAR scans), satellite images, geographic information system (GIS) data, etc.

[0051] Step S202: generating scene features for characterizing the current traffic scene according to the relevant data corresponding to the multiple dynamic objects and the scene map.

[0052] In an optional embodiment, feature extraction is performed on relevant data corresponding to multiple dynamic objects to obtain object features for characterizing the dynamic objects; feature extraction is performed on relevant data corresponding to the scene map to obtain map features for characterizing the scene map; the obtained object features and map features are fused along the feature dimension to obtain scene features for characterizing the current traffic scene.

[0053] The object features of the dynamic objects are obtained by fusing the time dimension features and the space dimension features; extracting features from the relevant data corresponding to the multiple dynamic objects to obtain the object features for characterizing the dynamic objects includes: fusing the object features of the dynamic objects and the map features along the feature dimensions to obtain the scene features for characterizing the current traffic scene. Extracting features from the relevant data corresponding to the multiple dynamic objects to obtain the initial object features; extracting the time dimension features and the space dimension features from the initial object features, fusing the time dimension features and the space dimension features to obtain the object features for characterizing the dynamic objects.

[0054] In an optional embodiment, the relevant data corresponding to the dynamic object is represented by O, and the dimension is , the relevant data corresponding to the scene map is represented by M, and the dimension is ,in N Indicates the number, T Indicates the timestamp length, D Represents the dimension of the original information. Fusion of time dimension features and space dimension features includes: compressing and fusing the time dimension features of multiple dynamic objects into one-dimensional features, and fusing the compressed time dimension features with the space dimension features. For example, the encoder used to extract high-dimensional features from low-dimensional information is a multi-layer perceptron (MLP), which is a neural network obtained by stacking linear layers, normalization layers, and activation layers. This process can be formulated as:

[0055]

[0056] in, Represents object characteristics, Represents map features, Represents navigation features, An encoder representing input information for extracting relevant data corresponding to a dynamic object; Represents an encoder for extracting input information corresponding to the scene map.

[0057] Encoder for the relevant data corresponding to the dynamic object In the design of , in the traditional trajectory prediction algorithm, the processing of spatiotemporal features is usually carried out by simply using the maximum pooling method along the temporal dimension, that is,

[0058]

[0059] Although this method is simple and easy to operate, it loses a lot of information in the time dimension.

[0060] In an optional embodiment of the present application, a temporal fusion strategy different from the existing solution is proposed, which compresses the time dimensions of multiple dynamic objects into one-dimensional features, and then fuses the compressed time dimension features with the spatial dimension. For example, through an agent temporal mixer (ATMixer), this process can be formulated as:

[0061]

[0062] in, Represents the relevant data of vehicle driving information after compression and fusion of time dimension features, Indicates the characteristics of the relevant data that completes the time series fusion through the space-time mixer, and is the corresponding multi-layer perceptron.

[0063] In order to better understand the working principle of the space-time mixer, the above formula will be explained with examples.

[0064] first, It means fusing the features of the time dimension (i.e., T dimension) corresponding to multiple objects (n objects) to obtain one-dimensional features. The fusion specifically includes feature extraction in a high dimension after feature crossing, and feature extraction in a low dimension after crossing. It can be understood that the time dimension of multiple objects is greater than or equal to 1. Through the above-mentioned time dimension feature fusion, the past time dimension information is integrated into the present and compressed to the current moment (i.e., one dimension). It means fusing the features of the spatial dimension (i.e., N dimension) corresponding to multiple objects (n objects) to obtain N-dimensional spatial features; it can be understood that the fusion of information of different objects is the fusion feature of spatial dimension. Through the above fusion of spatial dimension features, the input object is N-dimensional and the output object is still N-dimensional, so only feature fusion is performed without compression. Because the time dimension is compressed and fused from T dimensions to one dimension during processing, the dimension of the data can be significantly reduced while retaining the most important information, further ensuring the efficient operation of the prediction process, avoiding the time delay caused by information processing, reducing the computing power requirements for the vehicle-side chip, and adapting to a wider range.

[0065] In an optional embodiment, the spatial dimension features are subjected to feature cross processing, and the feature cross processing includes: performing feature cross processing on the spatial dimension features of multiple dynamic objects, and performing a feature extraction on the cross-processed features; performing a secondary feature cross processing on the spatial dimension features after the primary feature extraction, and performing a secondary feature extraction on the spatial dimension features obtained after the secondary feature cross, to obtain the spatial dimension features after the cross processing. Feature cross processing can fully integrate the information of dynamic objects in multiple dimensions. After integration, the dynamic object features corresponding to each dimension contain other dynamic object features, so that each dimension can represent the features of multiple dynamic objects.

[0066] In an optional embodiment, when object features and map features of dynamic objects are fused along feature dimensions to obtain scene features for characterizing the current traffic scene, the object features and map features can be spliced ​​and fused along the number dimension through a transformer, and different dynamic object features have different fusion weights when being fused with map features, and the fusion weight is determined based on the similarity between the corresponding dynamic object and other dynamic objects.

[0067] When the object features and map features of dynamic objects are fused to obtain scene features for characterizing the current traffic scene, the similarity of different dynamic objects can be calculated based on their object features; the calculated similarity is used as the fusion weight of the dynamic object, and the object features and map features of the dynamic object are fused. Since the information of dynamic objects includes the motion-related information of the dynamic objects, the weights used focus on the motion similarity between dynamic objects. Objects with high similarity to other dynamic objects naturally have more reference value for their object information, and are assigned higher weights. Different weights are configured by object similarity, so that subsequent trajectory predictions refer to the motion similarity between objects. Therefore, trajectory planning and prediction will become more accurate.

[0068] The above process can be formulated as:

[0069]

[0070] Among them, [] means concatenating object features and map features along the number dimension. Indicates the mapping layer that maps the concatenated features to queries. Indicates the mapping layer that maps the concatenated features to keys. Represents the mapping layer that maps the concatenated features to value; Represents the fused scene features, and its dimension is ; Feature dimension representing scene characteristics.

[0071] Step S203 , extracting the self-vehicle features related to the target vehicle and the non-self-vehicle features related to other dynamic objects among the multiple dynamic objects except the target vehicle from the scene features.

[0072] The scene features obtained in step S202 are equivalent to strengthening the features of dynamic objects. Since the prediction task refers to predicting the future motion trajectory of non-ego vehicle elements in the traffic scene, and the planning task is to plan the future trajectory of the ego vehicle, the features required for the two tasks are extracted from the scene features to obtain the ego vehicle features. and non-car features .

[0073] According to the characteristics of the own-car feature and the non-own-car feature, the following rules can be followed to extract the own-car feature and the non-own-car feature from the scene feature. The 0th index is the own-car feature, and the rest are non-own-car features.

[0074] Step S204, inputting the self-vehicle features, non-self-vehicle features and navigation data into the prediction model to obtain the first driving route of the target vehicle generated by the prediction model; the prediction model includes a first prediction head for planning the second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, and the first driving route is generated based on the second driving route and the motion trajectory of other dynamic objects.

[0075] It can be seen that the present application unifies the trajectory prediction of non-self-vehicle objects and the motion trajectory planning of the self-vehicle in one model, so that the model can output two prediction results, the trajectory prediction of non-self-vehicle objects and the motion trajectory planning of the self-vehicle, at the same time, without calling two models separately, so that the results in the autonomous driving scene are output faster and more timely, effectively ensuring the safety of driving, and also helping to learn the interactive behavior of non-self-vehicle objects and self-vehicle behaviors, reducing unreasonable conflicts (such as collisions, etc.) between the predicted trajectory of non-self-vehicles and the planned trajectory of self-vehicles. In addition, the tasks of multiple prediction heads can promote each other during the training process, making the prediction results more accurate, improving the efficiency of model training, and reducing the requirements for the computing power of the autonomous driving model on the vehicle side.

[0076] In an optional embodiment, the vehicle driving information processing method can be used in a single-vehicle driving mode and a vehicle formation mode; wherein the single-vehicle driving mode and the vehicle formation mode use different methods for obtaining navigation data. In the case of a single-vehicle driving mode, the navigation data is based on a positioning system such as the Global Positioning System (GPS) and the BeiDou Satellite Navigation System (BeiDou), and is a recommended route generated based on a map and the starting and ending points of the trip. In the case of a vehicle formation mode, the remaining vehicles in the formation, except for the front vehicle (i.e., the lead vehicle) of the convoy, all use the driving route of the front vehicle sent by the front vehicle through the communication device.

[0077] For example, when the target vehicle is an independently traveling vehicle, or a leading vehicle traveling in front in a platoon, the navigation information is the navigation route provided by the navigation application in the navigation system of the target vehicle; when the target vehicle is a trailing vehicle traveling in the rear in a platoon, the navigation information is the driving track of the leading vehicle in the platoon to which the target vehicle belongs.

[0078] In the case of platoon driving, the target vehicle can also communicate and interact with the front vehicle in the target vehicle formation to obtain the driving trajectory of the front vehicle.

[0079] After obtaining the navigation data, the navigation features corresponding to the navigation data are further extracted. In one example, the navigation data is represented by R, and the dimension is ,in N Indicates the number, D Represents the dimension of the original information. For the navigation data obtained, the encoder is used to extract high-dimensional navigation features from the low-dimensional navigation data. For example, the encoder used to extract high-dimensional features from low-dimensional information is a multi-layer perceptron (MLP), which is a neural network obtained by stacking linear layers, normalization layers, and activation layers. This process can be formulated as:

[0080]

[0081] in, Represents navigation features, Represents an encoder for extracting input information for navigation data.

[0082] Before inputting the own-vehicle features, non-own-vehicle features and navigation data into the prediction model, it also includes: fusing the own-vehicle features of multiple dynamic objects with the navigation features of the navigation data, using the fused first feature as the input of the first prediction head of the prediction model, and using the non-own-vehicle features as the input of the second prediction head of the prediction model.

[0083] In an optional embodiment, multi-context gating (MCG) is used to fuse the vehicle features and navigation features. This process can be formulated as follows:

[0084]

[0085] in, is the sigmoid function; represents the transformation matrix of the linear transformation of the ego-vehicle features, The offset vector representing the linear transformation of the ego-vehicle features; The transformation matrix representing the linear transformation of the navigation features, represents the offset vector of the linear transformation of the navigation feature, , , and Their values ​​can be updated through model training.

[0086] In an optional embodiment, the first prediction head includes a first sub-prediction head that outputs a single motion mode, a second sub-prediction head that outputs multiple motion modes, a third sub-prediction head that outputs a longitudinal acceleration and deceleration operation mode, and a fourth sub-prediction head that outputs a lateral avoidance operation mode.

[0087] For the self-driving car planning task, the output of its prediction head needs to be more interpretable. Therefore, the first prediction head is designed to correspond to the third sub-prediction head and the fourth sub-prediction head with specific semantic information, so that the self-driving car trajectory planning output by the model is more interpretable and can be used directly. In addition, the self-driving car planning task can also add the first sub-prediction head and the second sub-prediction head corresponding to the motion mode with a priori value that does not represent specific semantics on the basis of the third sub-prediction head and the fourth sub-prediction head. Because it adopts the prediction method of prior values, the results can be better aligned with human driving behavior, and the naturalness and acceptability of the planning can be improved. Therefore, when predicting the self-driving car, the mixed use of motion modes with specific semantics and motion modes with prior values ​​that do not represent specific semantics can not only meet the premise that the output trajectory planning is interpretable, but also the naturalness of the predicted results, so that the prediction results are richer. In an example, the first sub-prediction head and the second sub-prediction head use prior values ​​(random number method), and the third sub-prediction head and the fourth sub-prediction head use a mode with specific semantic information. Since the first sub-prediction head is a single motion mode, there is only one classification, and the corresponding motion mode is more detailed, so the prediction is the most accurate. Among them, the third sub-prediction head corresponds to the longitudinal acceleration and deceleration behavior in human driving behavior, such as the need to slow down at traffic lights, the need to slow down when encountering sudden behaviors in front of the car, etc.; the fourth sub-prediction head corresponds to the lateral avoidance behavior in human driving behavior, such as when other vehicles squeeze the car from the side, the car needs to avoid it laterally, etc.

[0088] It can be understood that longitudinal acceleration and deceleration may include: more longitudinal acceleration, more longitudinal deceleration, and longitudinal balanced acceleration and deceleration.

[0089] The motion behaviors of the first and second sub-prediction heads can be represented as n learnable parameter vectors in the model and optimized through gradient back-propagation.

[0090] Through these sub-prediction heads, the first prediction head is combined with the characteristics of the vehicle, and then the multi-layer perceptron can predict the m possible trajectories of the vehicle in the future, corresponding to the m possible behaviors of the vehicle.

[0091] In addition, for the motion actions that non-ego vehicle objects can perform, a second prediction head that can simulate multiple possibilities is proposed. It represents the various motion modes of non-ego vehicle objects, such as the various motion behaviors that the ego vehicle may perform (going straight, turning left, turning right, and parking). It is represented in the model as n learnable parameter vectors and optimized through gradient backpropagation.

[0092] After combining the second prediction head with the non-self-vehicle features, the predicted trajectory of the non-self-vehicle object can be obtained through a multi-layer perceptron, where each non-self-vehicle object corresponds to n predicted trajectories, representing n possibilities.

[0093] Corresponding to the model for predicting the vehicle driving route mentioned in the method provided in the embodiment of the present application, the embodiment of the present application also provides a model training method for predicting the vehicle driving route, such as Figure 3 The flowchart of a model training method for vehicle driving route prediction according to an embodiment of the present application is shown, including:

[0094] Step S301, obtaining relevant data samples corresponding to multiple dynamic objects and scene maps in multiple historical traffic scenes;

[0095] Step S302, training a prediction model based on relevant data samples, the prediction model is used to predict a first driving route of the target vehicle based on self-vehicle features related to the target vehicle, non-self-vehicle features related to other dynamic objects among multiple dynamic objects except the target vehicle, and navigation data, the prediction model includes a first prediction head for planning a second driving route of the target vehicle and a second prediction head for predicting motion trajectories of other dynamic objects, the first driving route is generated based on the second driving route and the motion trajectories of other dynamic objects; the self-vehicle features and the non-self-vehicle features are extracted from scene features, and the scene features are generated based on relevant data corresponding to multiple dynamic objects and scene maps.

[0096] In order to better illustrate a specific training method for a model of a vehicle driving route in one embodiment of the present application, the training is performed through the following training process:

[0097] The model training process for vehicle driving route prediction may include:

[0098] Step 1: Obtain relevant data samples corresponding to multiple dynamic objects and scene maps in multiple historical traffic scenes.

[0099] In an optional embodiment, the relevant data samples are divided into training set data and test set data in a ratio of 8:2.

[0100] Step 2: Input the training set data of relevant data into the vehicle route prediction model, and output the predicted trajectory of non-self-vehicle objects and the movement route planning of the self-vehicle.

[0101] In an optional embodiment, a first prediction head in the prediction model for planning a second driving route of the target vehicle partially uses motion pattern information representing semantic information to represent the motion pattern during training.

[0102] The second prediction head in the prediction model, which is used to predict the motion trajectory of other dynamic objects, partially uses the prior values ​​of all semantic information to represent the motion pattern information during training.

[0103] Step 3: Use the test set data of the relevant data and the output training set data to predict the predicted trajectory of the non-ego vehicle object and the motion route planning of the ego vehicle, and compare the model indicators of the training set data and the test set data.

[0104] In an optional embodiment, comparing the model indicators of the training set data and the test set data may include:

[0105] Accuracy, precision, recall, F1 score, ROC curve and AUC score, mean square error or root mean square error, cross entropy loss, etc.

[0106] Step 4: Adjust the parameters through the stochastic gradient descent algorithm and repeat the method of steps S301-S303 until the test set model index reaches the preset test set judgment threshold and the training set prediction loss value is lower than the set training set judgment threshold to obtain a trained vehicle driving route prediction model.

[0107] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a vehicle driving information processing device. Figure 4 The figure is a structural block diagram of a vehicle driving information processing device according to an embodiment of the present application. The image processing device may include:

[0108] The relevant data acquisition module 401 is used to acquire relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene;

[0109] A scene feature generation module 402, for generating scene features for representing the current traffic scene according to the relevant data corresponding to the plurality of dynamic objects and the scene map;

[0110] The ego vehicle feature and non-ego vehicle feature extraction module 403 is used to extract the ego vehicle feature related to the target vehicle and the non-ego vehicle feature related to other dynamic objects among the multiple dynamic objects except the target vehicle from the scene features;

[0111] The first driving route prediction module 404 is used to input the vehicle characteristics, non-vehicle characteristics and navigation data into the prediction model to obtain the first driving route of the target vehicle generated by the prediction model; the prediction model includes a first prediction head for planning the second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, and the first driving route is generated based on the second driving route and the motion trajectory of other dynamic objects.

[0112] In an optional embodiment, the scene feature generation module 402 includes:

[0113] An object feature extraction submodule is used to extract features from relevant data corresponding to multiple dynamic objects to obtain object features for characterizing the dynamic objects;

[0114] A map feature extraction submodule is used to extract features from relevant data corresponding to the scene map to obtain map features for characterizing the scene map;

[0115] The scene feature fusion submodule is used to fuse the object features and map features of dynamic objects along the feature dimension to obtain scene features for characterizing the current traffic scene.

[0116] In an optional embodiment, the object feature extraction submodule includes:

[0117] An initial object feature extraction module is used to extract features from relevant data corresponding to multiple dynamic objects to obtain initial object features;

[0118] The object feature acquisition submodule is used to extract time dimension features and space dimension features from the initial object features, fuse the time dimension features and the space dimension features, and obtain object features for characterizing dynamic objects.

[0119] In an optional embodiment, the object feature acquisition submodule includes:

[0120] The time-space dimension fusion submodule is used to compress and fuse the time dimension features of multiple dynamic objects into one-dimensional features, and to fuse the compressed time dimension features with the space dimension features.

[0121] In an optional embodiment, the object feature acquisition submodule includes:

[0122] The spatiotemporal dimension processing submodule is used to perform feature cross-processing on the spatial dimension features of multiple dynamic objects and perform a feature extraction on the features after the cross-processing; perform a secondary feature cross-processing on the spatial dimension features after the primary feature extraction, and perform a secondary feature extraction on the spatial dimension features obtained after the secondary feature cross-processing to obtain the spatial dimension features after the cross-processing.

[0123] In an optional embodiment, the first driving route prediction module 404 also includes a vehicle feature and navigation feature fusion module, which is used to fuse the vehicle features of multiple dynamic objects with the navigation features of the navigation data, and the fused first features are used as the input of the first prediction head of the prediction model, and the non-vehicle features are used as the input of the second prediction head of the prediction model.

[0124] In an optional embodiment, the scene feature fusion submodule includes:

[0125] A similarity calculation submodule is used to calculate the similarity of different dynamic objects according to the object features of different dynamic objects;

[0126] The calculated similarity is used as the fusion weight of the dynamic object, and the object features and map features of the dynamic object are fused.

[0127] In an optional embodiment, the first driving route prediction module 404 includes:

[0128] A first prediction head submodule is used to plan a second driving route of the target vehicle and partially use the motion pattern representing semantic information during training;

[0129] The second prediction head submodule is used to predict the motion trajectory of other dynamic objects and partially use prior values ​​that do not represent semantic information to represent the motion pattern during training.

[0130] In an optional embodiment, the first prediction head submodule includes:

[0131] The sub-prediction head submodule includes a first sub-prediction head submodule for outputting a single motion mode, a second sub-prediction head submodule for outputting multiple motion modes, a third sub-prediction head submodule for outputting a longitudinal acceleration and deceleration operation mode, and a fourth sub-prediction head submodule for outputting a lateral avoidance operation mode.

[0132] In an optional embodiment, the vehicle feature and navigation feature fusion module, when the target vehicle is an independently driven vehicle or a leading vehicle in a formation, the navigation information is a navigation route provided by a navigation application in the navigation system of the target vehicle; when the target vehicle is a trailing vehicle in a formation, the navigation information is a driving track of the leading vehicle in the formation to which the target vehicle belongs;

[0133] It also includes a communication submodule, which is used to communicate and interact with the front vehicle of the target vehicle formation to obtain the driving trajectory of the front vehicle.

[0134] In an optional embodiment, the relevant data acquisition module 401 includes:

[0135] A dynamic object range submodule, used to confirm that the dynamic object is a vehicle traveling in a convoy, a pedestrian, or a single vehicle, and the corresponding relevant data includes at least one of the position, driving direction, object size, driving speed, and object category;

[0136] The scene map range submodule is used to confirm that the relevant data corresponding to the scene map includes at least one of lane line position information and road edge position information.

[0137] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a model training device for vehicle driving route prediction. Figure 5 The structure block diagram of a model training device for vehicle driving route prediction according to an embodiment of the present application is shown, which may include:

[0138] A relevant data sample acquisition module 501 is used to acquire relevant data samples corresponding to multiple dynamic objects and scene maps in multiple historical traffic scenes;

[0139] The first driving route prediction module 502 is used to train a prediction model based on relevant data samples. The prediction model is used to predict the first driving route of the target vehicle based on the self-vehicle features related to the target vehicle, the non-self-vehicle features related to other dynamic objects among multiple dynamic objects except the target vehicle, and the navigation data. The prediction model includes a first prediction head for planning the second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects. The first driving route is generated based on the second driving route and the motion trajectory of other dynamic objects; the self-vehicle features and the non-self-vehicle features are extracted from the scene features, and the scene features are generated based on the relevant data corresponding to the multiple dynamic objects and the scene map.

[0140] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0141] Figure 6 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 6 As shown, the electronic device includes: a memory 610 and a processor 620. The memory 610 stores a computer program that can be run on the processor 620. When the processor 620 executes the computer program, the method in the above embodiment is implemented. The number of the memory 610 and the processor 620 can be one or more.

[0142] The electronic device also includes:

[0143] The communication interface 630 is used to communicate with external devices and perform data exchange transmission.

[0144] If the memory 610, the processor 620 and the communication interface 630 are implemented independently, the memory 610, the processor 620 and the communication interface 630 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0145] Optionally, in a specific implementation, if the memory 610, the processor 620 and the communication interface 630 are integrated on a chip, the memory 610, the processor 620 and the communication interface 630 can communicate with each other through an internal interface.

[0146] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0147] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0148] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0149] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0150] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0151] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0152] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0153] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0154] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0155] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0156] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0158] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for processing vehicle driving information, characterized in that: include: Obtain relevant data corresponding to multiple dynamic objects and scene maps in the current traffic scene; Generating a scene feature for characterizing a current traffic scene according to the relevant data respectively corresponding to the plurality of dynamic objects and the scene map; The scene feature is obtained by fusing the object feature of the dynamic object and the map feature corresponding to the scene map; the object feature of the dynamic object is obtained by fusing the time dimension feature and the space dimension feature; when fusing the time dimension feature and the space dimension feature, the time dimension features of the multiple dynamic objects are compressed and fused into a one-dimensional feature, and the compressed time dimension feature is fused with the space dimension feature; Extracting self-vehicle features related to the target vehicle and non-self-vehicle features related to other dynamic objects among the multiple dynamic objects except the target vehicle from the scene features; The vehicle characteristics, non-vehicle characteristics and navigation data are input into a prediction model to obtain a first driving route of the target vehicle generated by the prediction model; the prediction model includes a first prediction head for planning a second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, and the first driving route is generated based on the second driving route and the motion trajectory of the other dynamic objects.

2. The method according to claim 1, characterized in that The generating of scene features for characterizing the current traffic scene according to the relevant data respectively corresponding to the scene maps of the plurality of dynamic objects comprises: Extracting features from relevant data corresponding to the multiple dynamic objects to obtain object features for characterizing the dynamic objects; Extracting features from relevant data corresponding to the scene map to obtain map features for characterizing the scene map; The object features and map features of the dynamic object are fused along the feature dimension to obtain scene features for characterizing the current traffic scene.

3. The method according to claim 2, characterized in that The extracting features of the relevant data corresponding to the multiple dynamic objects to obtain object features for characterizing the dynamic objects comprises: Extracting features from relevant data corresponding to the multiple dynamic objects to obtain initial object features; The time dimension features and the space dimension features are extracted from the initial object features, and the time dimension features and the space dimension features are fused to obtain the object features for characterizing the dynamic object.

4. The method according to claim 3, characterized in that The spatial dimension features are subjected to feature cross processing, and the feature cross processing includes: Performing feature cross processing on the spatial dimension features of the multiple dynamic objects, and performing feature extraction on the cross-featured features; A secondary feature cross processing is performed on the spatial dimension features after the primary feature extraction, and a secondary feature extraction is performed on the spatial dimension features obtained after the secondary feature cross processing to obtain the spatial dimension features after the cross processing.

5. The method according to claim 2, characterized in that: Different dynamic object features have different fusion weights when being fused with the map features, and the fusion weights are determined according to the similarity between the corresponding dynamic object and other dynamic objects; The fusing the object features and the map features of the dynamic object along the feature dimension to obtain the scene features for characterizing the current traffic scene includes: Calculating similarities of different dynamic objects based on their object features; The calculated similarity is used as the fusion weight of the dynamic object, and the object feature and the map feature of the dynamic object are fused.

6. The method according to claim 1, characterized in that Before inputting the self-vehicle features, non-self-vehicle features and navigation data into the prediction model, the method further includes: The ego-vehicle features of the multiple dynamic objects are fused with the navigation features of the navigation data, and the fused first features are used as inputs of the first prediction head of the prediction model, and the non-ego-vehicle features are used as inputs of the second prediction head of the prediction model.

7. The method according to claim 6, characterized in that When the target vehicle is an independently traveling vehicle or a leading vehicle in a convoy, the navigation data is a navigation route provided by a navigation application in a navigation system of the target vehicle; When the target vehicle is a rear vehicle in a formation, the navigation data is a driving track of a front vehicle in the formation to which the target vehicle belongs; the method further includes: Communicate and interact with the front vehicle of the formation of the target vehicle to obtain the driving trajectory of the front vehicle.

8. The method according to claim 1, characterized in that The driving route and the motion trajectory are composed of at least one motion pattern, and the first prediction head used for planning the second driving route of the target vehicle in the prediction model partially uses the motion pattern representing semantic information during training; The second prediction head in the prediction model for predicting the motion trajectory of other dynamic objects partially uses a priori values ​​that do not represent semantic information to represent the motion pattern during training.

9. The method according to claim 1, characterized in that: The first prediction head includes a first sub-prediction head outputting a single motion mode, a second sub-prediction head outputting multiple motion modes, a third sub-prediction head outputting a longitudinal acceleration and deceleration operation mode, and a fourth sub-prediction head outputting a lateral avoidance operation mode.

10. The method according to claim 1, characterized in that The dynamic objects are vehicles traveling in formation, pedestrians or individual vehicles, and the corresponding relevant data include at least one of position, driving direction, object size, driving speed and object category; the relevant data corresponding to the scene map include at least one of lane line position information and road edge position information.

11. A model training method for vehicle driving route prediction, comprising: Obtain relevant data samples corresponding to multiple dynamic objects and scene maps in multiple historical traffic scenes; Generating a scene feature for characterizing a current traffic scene according to the relevant data respectively corresponding to the plurality of dynamic objects and the scene map; Extracting self-vehicle features related to the target vehicle and non-self-vehicle features related to other dynamic objects other than the target vehicle among the multiple dynamic objects from the scene features; the scene features are obtained by fusing the object features of the dynamic objects and the map features corresponding to the scene map; the object features of the dynamic objects are obtained by fusing the time dimension features and the space dimension features; when fusing the time dimension features and the space dimension features, the time dimension features of the multiple dynamic objects are compressed and fused into one-dimensional features, and the compressed time dimension features are fused with the space dimension features; training a prediction model based on the relevant data samples, the prediction model is used to predict a first driving route of the target vehicle according to the self-vehicle features related to the target vehicle, the non-self-vehicle features related to other dynamic objects other than the target vehicle among the multiple dynamic objects, and the navigation data, the prediction model includes a first prediction head for planning a second driving route of the target vehicle and a second prediction head for predicting the motion trajectory of other dynamic objects, the first driving route is generated according to the second driving route and the motion trajectory of the other dynamic objects; the self-vehicle features and the non-self-vehicle features are extracted from the scene features, and the scene features are generated according to the relevant data corresponding to the multiple dynamic objects and the scene map respectively.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. A computer program product, comprising a computer program / instruction, wherein the computer program / instruction implements the method according to any one of claims 1 to 11 when executed by a processor.

Images