Vehicle driving state prediction method, related device, equipment and storage medium

Abstract

The invention discloses a vehicle driving state prediction method, a related device, equipment and a storage medium. The vehicle driving state prediction method comprises the following steps: acquiring current driving state information of a target vehicle at a current moment and initial road environment information at the current moment; performing feature extraction on the initial road environment information at the current moment by using a road network, and performing prediction based on the extracted features to obtain target road environment information at the next moment, wherein the road network is obtained by training sample road environment information obtained by discretizing a map; and determining target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information. According to the scheme, the driving state of the vehicle can be predicted.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle driving state prediction method and related devices, equipment, and storage media. Background Art

[0002] In autonomous vehicles (AVs), the current autonomous driving trajectory motion generation model strictly relies on maps, that is, before each autonomous driving trajectory motion generation model is applied, it needs to be trained according to the map of the place of use, so that the model can subsequently perform trajectory planning in this area.

[0003] However, these models have poor generalization capabilities across data sets. When each model is used in a new area, it is necessary to collect a large amount of driving data and travel data as training data for the new road network environment or map, and retrain it, otherwise the model cannot be applied to the new city for trajectory planning. Summary of the invention

[0004] The embodiments of the present application at least provide a vehicle driving state prediction method and related devices, equipment, and storage media.

[0005] An embodiment of the present application provides a method for predicting a vehicle's driving state, including: obtaining current driving state information of a target vehicle at a current moment and initial road environment information at a current moment; using a road network to extract features of the initial road environment information at the current moment and predicting target road environment information at the next moment based on the extracted features, wherein the road network is trained based on sample road environment information obtained by discretizing a map; based on the current driving state information and the target road environment information, determining the target driving state information of the target vehicle at the next moment.

[0006] In the above scheme, after obtaining the initial road environment information at the current moment, the road network can predict the road environment information at the next moment based on the initial road environment information, so that with the reference of the road environment information, combined with the current driving state information of the target vehicle at the current moment, the trained road network can be used to determine the target driving state information of the target vehicle at the next moment, thereby predicting the driving state of the vehicle without the need to collect a large amount of driving data and driving data of the new road network at a high cost, so that the target vehicle can also achieve accurate trajectory planning when entering an unfamiliar driving environment, thereby improving the generalization ability and scalability of the vehicle driving state prediction and trajectory planning algorithms in autonomous driving and assisted driving, and can effectively reduce the cost of autonomous driving algorithms.

[0007] In some embodiments, the method also includes: obtaining reference driving status information of other vehicles at the current moment, the reference driving status information of other vehicles at the current moment includes the position information and / or posture information of other vehicles at the current moment, and other vehicles include vehicles in the same driving area as the target vehicle; based on the current driving status information and the target road environment information, determining the target driving status information of the target vehicle at the next moment, including: based on the current driving status information, the reference driving status information and the target road environment information, determining the target driving status information of the target vehicle at the next moment.

[0008] In the above scheme, the target driving state of the target vehicle at the next moment is comprehensively determined based on the driving state information of the target vehicle and the driving state information of other vehicles, thereby reducing the probability of the target vehicle colliding with other vehicles during driving.

[0009] In some embodiments, based on the current driving state information and the target road environment information, the target driving state information of the target vehicle at the next moment is determined, including: based on the current driving state information, the reference driving state information and the target road environment information, determining the possibility that the target vehicle will be in several candidate driving state information at the next moment, the candidate driving state information including candidate position information and / or candidate posture information, and each candidate driving state information is pre-trained; selecting candidate driving state information that is likely to meet preset state conditions from several candidate driving state information as the target driving state information.

[0010] In the above scheme, the vehicle's driving trajectory is also complex in a complex traffic environment. For example, at a certain intersection, the vehicle may go straight, turn left or make a U-turn, that is, the vehicle may actually have multiple candidate driving state information at the next moment. This scheme fully considers the possibility of the vehicle being in each candidate driving state information at the next moment, so that the estimated vehicle driving state information is more accurate.

[0011] In some embodiments, the current driving state information of the target vehicle at the current moment is the target driving state information determined at the previous moment, and the method also includes: using the determined target driving state information of the target vehicle at the next moment as the current driving state information at the next moment, so as to predict the target driving state information at the next moment based on the determined current driving state information of the target vehicle at the next moment.

[0012] In the above scheme, the driving state information of the vehicle at the current moment is the driving state information predicted at the previous moment. When the vehicle moves to the next moment, the current driving state information of the target vehicle at the next moment is the target driving state information predicted at the current moment. That is, in the process of predicting the vehicle's driving state, there is no need to rely on the actual driving state of the vehicle, for example, there is no need to rely on the actual position of the vehicle.

[0013] In some embodiments, based on current driving state information, reference driving state information and target road environment information, the possibility of the target vehicle being in several candidate driving state information at the next moment is determined, including: generating value data and index data in a first key-value pair based on the target road environment information; and generating query data in the first key-value pair based on the current driving state information and the reference driving state information; processing the first key-value pair as input to a scene cross-attention module, and outputting the scene cross-attention module as the possibility of several candidate driving state information.

[0014] In the above scheme, the scene cross-attention module can integrate the road environment information into the prediction process of driving state information, so that the driving state of the target vehicle at the next moment can be predicted with the participation of road environment information, which can improve the accuracy of driving state prediction.

[0015] In some embodiments, the current driving status information also includes the current trajectory information of the target vehicle, and the current trajectory information includes the historical driving status information of at least one trajectory point within a preset historical time period in the current trip; the query data in the first key-value pair is generated based on the current driving status information and the reference driving status information, including: determining the query data in the second key-value pair based on the current driving status information and the historical driving status information; and determining the value data and index data in the second key-value pair based on the reference driving status information; the second key-value pair is processed as the input of the motion cross-attention module, and the output of the motion cross-attention module is used as the query data of the first key-value pair.

[0016] In the above scheme, the position information and / or posture information of different vehicles can be integrated through the motion cross-attention module, and on this basis, the collisions between vehicles can be reduced during driving.

[0017] In some embodiments, based on current driving status information and historical driving status information, query data in the second key-value pair is determined, including: generating query data for a third key-value pair based on the historical driving status information; and generating value data and index data in the third key-value pair based on the current driving status information; processing the third key-value pair as input to a time dimension self-attention module, and using the output of the time dimension self-attention module as query data in the second key-value pair.

[0018] In the above solution, by referring to the historical driving state information of the vehicle in the historical time period, the predicted driving state information of the vehicle at the next moment is made smoother for the position of the vehicle.

[0019] In some embodiments, the method also includes the step of training the road network in the following manner: discretizing a map containing road environment information to obtain a number of sample road environment information; inputting preset road environment information from the number of sample road environment information into the road network for processing, and outputting the road network as the predicted road environment information at the next moment in the direction of vehicle travel; and adjusting parameters in the road network based on the difference between the predicted road environment information and the actual road environment information.

[0020] In the above scheme, after discretizing the map, the road network is trained using sample road environment information, so that the road network can learn the connection relationship between road environments, so that the road environment information at the next moment can be predicted based on the road environment information at the current moment. After the road network training is completed, the vehicle's position information at the next moment can be predicted without relying on the map. The vehicle can continue to operate when entering a new city without repeating the road network training.

[0021] In some embodiments, a map containing road environment information is discretized to obtain a number of sample road environment information, including: dividing the map into multiple segments; marking each segment to obtain each sample road environment information, each sample road environment information carries one or more of the starting position, end position, length, turning direction and other semantic features of the corresponding segment.

[0022] In the above scheme, the map is divided into multiple segments, and each segment is also marked with multiple information, so that the road network can learn the connection relationship between the marked segments.

[0023] In some embodiments, the current driving state information includes position information, and based on the current driving state information and the target road environment information, the step of determining the target driving state information of the target vehicle at the next moment is performed by a motion network, and the method also includes a step of training the motion network in the following manner: discretizing a number of sample trajectories at preset time intervals to obtain a trajectory set; determining the initial position information of the vehicle to be predicted at time t from the trajectory set, the initial position information being the preset position information at the first time point, the perturbation position information, or the predicted position information of the vehicle to be predicted at time t-1 predicted by the motion network; processing the initial position information and the sample road environment information where the initial position information is located as inputs of the motion network, and outputting the motion network as the predicted position information of the vehicle to be predicted at time t+1; and adjusting the parameters in the motion network based on the difference between the predicted position information and the actual position information.

[0024] In the above scheme, by discretizing the trajectory of the vehicle to be predicted, the position of the vehicle in each time period or each time point can be obtained, which facilitates the subsequent motion network to learn the movement change rules between each position, thereby realizing the prediction of the vehicle's position information.

[0025] In some embodiments, the predicted position information is selected by a motion network from a number of preset candidate position information; based on the difference between the predicted position information and the actual position information, the parameters in the motion network are adjusted, including: determining the cross entropy loss between the predicted position information and the actual position information; and adjusting the parameters in the motion network based on the cross entropy loss.

[0026] In the above scheme, the motion network is actually a classification model. By classifying the vehicle at the next moment, the motion network can learn the vehicle's motion plan at the next moment, fully consider the possibility of the vehicle being at each candidate position at the next moment, and make the estimated vehicle position information more accurate.

[0027] In some embodiments, the initial position information is disturbed position information, and the initial position information of the vehicle to be predicted at time t is determined from the trajectory set, including: in response to receiving the predicted position information at time t-1 output by the motion network, selecting one from other interfering position information corresponding to time t-1 as the initial position information; wherein the other interfering position information is other position information other than the predicted position information among several preset candidate position information corresponding to time t-1.

[0028] In the above scheme, considering that the prediction process of the position information may result in large errors in the predicted position information due to the existence of cumulative errors, this scheme sets the initial position information as interference position information during the movement of the motion model, so that the motion network can have better prediction results in the future even when there are errors.

[0029] The present application provides a vehicle driving state prediction device, comprising: an information acquisition unit, a road prediction unit, and a state information prediction unit. The information acquisition unit is used to acquire the current driving state information of the target vehicle at the current moment and the initial road environment information at the current moment; the road prediction unit is used to extract features of the initial road environment information at the current moment using a road network and predict the target road environment information at the next moment based on the extracted features, wherein the road network is obtained by training based on sample road environment information obtained by discretizing a map; the state information prediction unit is used to determine the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information.

[0030] An embodiment of the present application provides a device, including a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned vehicle driving state prediction method.

[0031] An embodiment of the present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned vehicle driving state prediction method is implemented.

[0032] In the above scheme, after obtaining the initial road environment information at the current moment, the road network can predict the road environment information at the next moment based on the initial road environment information. Therefore, with the reference of the road environment information, combined with the current driving state information of the target vehicle at the current moment, the target driving state information of the target vehicle at the next moment can be determined, thereby realizing vehicle trajectory planning without relying on maps, so that the target vehicle can also realize trajectory planning during driving in an unfamiliar city, thereby improving the accuracy of driving state information prediction.

[0033] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0035] Figure 1 It is a flow chart of an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0036] Figure 2 is a schematic diagram of a road shown in an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0037] Figure 3 yes Figure 1Schematic diagram of the sub-process of step S13;

[0038] Figure 4 It is another flow chart of an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0039] Figure 5 This is a schematic diagram of a training process for a road network according to an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0040] Figure 6 This is another schematic diagram of a training process for a road network according to an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0041] Figure 7 This is a schematic diagram showing the training process of the motion network according to an embodiment of the method for predicting the driving state of a vehicle of the present application;

[0042] Figure 8 This is a schematic diagram of a trajectory set obtained by discretizing the trajectory according to an embodiment of a method for predicting a driving state of a vehicle of the present application;

[0043] Fig. 9 This is a schematic diagram showing various types of position information at various times according to an embodiment of a method for predicting a vehicle's driving state of the present application;

[0044] Fig.10 It is a structural schematic diagram of an embodiment of a vehicle driving state prediction device of the present application;

[0045] Fig.11 It is a schematic diagram of the framework of an embodiment of the device of the present application;

[0046] Fig.12 A schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0047] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0048] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0049] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0050] The execution subject of the vehicle's driving state prediction method may be a vehicle's driving state prediction device. The position prediction device may be set in a vehicle or in a device that establishes a communication connection with the vehicle. The vehicle's driving state prediction device may be any terminal device or server or other processing device that can execute the embodiments of the present application, wherein the terminal device may be a vehicle, an electronic device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the vehicle's driving state prediction method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0051] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a method for predicting the driving state of a vehicle of the present application. Specifically, the method may include the following steps:

[0052] Step S11: Acquire the current driving state information of the target vehicle at the current moment and the initial road environment information at the current moment.

[0053] Among them, the target vehicle is a vehicle traveling on the road. Specifically in this embodiment, the target vehicle may be an automatic driving or assisted driving vehicle whose driving trajectory is to be decided. In some embodiments, the driving state information is information that characterizes the vehicle's own state and / or the relative state with other traffic participants (such as other vehicles, pedestrians, etc.) when driving on the road, and may include position information and / or posture information. The current driving state information of the target vehicle at the current moment includes the position information and / or posture information of the target vehicle at the current moment. In some application scenarios, the current driving state information of the target vehicle at the current moment may include the position information of the vehicle. In some application scenarios, the vehicle position information of the target vehicle at the current moment may include the position information and posture information of the vehicle. In some application scenarios, the vehicle position information of the target vehicle at the current moment may include the posture information of the vehicle. In some application scenarios, the current driving state information of the target vehicle at the current moment may also include the speed information, acceleration information, lane change and / or turning of the target vehicle at the current moment and other information related to the driving control of the vehicle. Combining more information to determine the target driving state information at the next moment can make the determined target driving state information more accurate. Optionally, the current driving status information can be determined based on the driving status information predicted at the previous moment, or the driving status information at the first moment can be determined based on positioning methods such as GPS, or it can be determined according to any other feasible method, for example, it can also be obtained through a pre-set method, which is not specifically limited here.

[0054] The initial road environment information may include lane line information, road indication information and / or the drivable area identification of the current road. Optionally, the initial road environment information may be in the form of a vector representation of each information. The initial road environment information at the current moment may be obtained by sensing devices such as a vehicle-mounted camera, a vehicle-mounted laser radar, a position sensor, or the like. Alternatively, the initial road environment information may be obtained by predicting the road environment information at the previous moment using the method of this embodiment. The road environment information of the vehicle at the beginning of the journey may be obtained by collecting the environment based on the image acquisition component, or by GPS positioning, or by road simulation based on the vehicle's driving trajectory in a historical time period. For example, a straight trajectory and a left turn or right turn trajectory may be used to predict the road environment information.

[0055] Step S12: Using the road network to extract features from the initial road environment information at the current moment and predicting the target road environment information at the next moment based on the extracted features.

[0056] The road network may include a feature extraction module and a road environment information prediction module. As long as the neural network structure can predict the road environment information at the next moment based on the road environment information at one moment, it can be used as the road network provided by the embodiment of the present application. Among them, the road network is obtained by training based on the sample road environment information obtained by discretizing the map. Optionally, the road network can be pre-trained to extract the features of the initial road environment information at the current moment according to the road network and predict the target road environment information at the next moment based on the extracted features. The map here can be a map of any city or region. By discretizing the map, the road data in the map is divided into blocks or segments, and the road network is trained using the characteristics of the road data of the blocks or segments and / or the spatial relationship between each other. By discretizing the map to obtain discrete road environment information, a sequential correlation relationship between each discrete road environment information can be established, so that the road network is trained according to each discrete road environment information, so that the road network can predict the road environment information at the next moment according to the initial road environment information at the current moment.

[0057] Step S13: Based on the current driving state information and the target road environment information, determine the target driving state information of the target vehicle at the next moment.

[0058] The target driving state information at the next moment may include position and / or posture, and may also include at least one of speed, acceleration, lane change, and turn. Optionally, a sample data set containing multiple time-series continuous driving state information of the vehicle and corresponding road environment information may be pre-constructed and used to train the motion network, and the current driving state information and the target road environment information may be input into the trained motion network, and the output of the motion network may be the target driving state information of the target vehicle at the next moment. Alternatively, the output of the motion network may be processed, such as data cleaning or data conversion, to obtain the target driving state information of the target vehicle at the next moment.

[0059] In the above scheme, after obtaining the initial road environment information at the current moment, the road network can predict the road environment information at the next moment based on the initial road environment information, so that with the reference of the road environment information, combined with the current driving state information of the target vehicle at the current moment, the target driving state information of the target vehicle at the next moment can be determined, thereby without the need to collect a large amount of driving data and driving data of the new road network at a high cost, the vehicle's driving state prediction can be performed, so that the target vehicle can also achieve accurate trajectory planning when entering an unfamiliar driving environment, thereby improving the generalization ability and scalability of vehicle driving state prediction and trajectory planning algorithms in autonomous driving and assisted driving, and can effectively reduce the cost of autonomous driving algorithms.

[0060] In some embodiments, the current driving state information of the target vehicle at the current moment may be the target driving state information determined at the previous moment using the method of the embodiment of the present application. At this time, the above-mentioned vehicle driving state prediction method may also include: using the determined target driving state information of the target vehicle at the next moment as the current driving state information at the next moment, so as to predict the target driving state information at the next moment based on the determined current driving state information of the target vehicle at the next moment.

[0061] Exemplarily, the position information of the vehicle at the current moment is determined by the driving state prediction method of the above embodiment at the previous moment, and the position information at the current moment can be used to predict the position information of the vehicle at the next moment. For example, the current driving state at moment n is the target driving state information predicted based on moment n-1, and the target driving state information predicted at moment n+1 by moment n can be used as the current driving state information at moment n+1 to predict the target driving state information at moment n+2. Among them, moment n in the embodiment of the present application can be any sampling moment during the driving process of the target vehicle, and n+1 and n+2 are two moments sampled sequentially after moment n. In this way, the position information of the target vehicle can be predicted by a rolling prediction method. In some application scenarios, the posture information of the vehicle at the current moment is determined by information such as the posture information at the previous moment, and the posture information at the current moment can be used to predict the posture information at the next moment.

[0062] In the above scheme, the driving state information of the vehicle at the current moment is the driving state information predicted at the previous moment. When the vehicle moves to the next moment, the current driving state information of the target vehicle at the next moment is the target driving state information predicted at the current moment. That is, in the process of predicting the vehicle's driving state, there is no need to rely on the actual driving state of the vehicle, for example, there is no need to rely on the actual position of the vehicle.

[0063] In some embodiments, the vehicle driving state prediction method also includes: obtaining reference driving state information of other vehicles at the current moment. The reference driving state information of other vehicles at the current moment includes the position information and / or posture information of other vehicles at the current moment, and other vehicles are vehicles in the same driving area as the target vehicle. The above step S13 may include the following steps: based on the current driving state information, the reference driving state information and the target road environment information, determine the target driving state information of the target vehicle at the next moment. Exemplarily, the vehicle position information of the target vehicle at the current moment, the position information and / or posture information of other vehicles at the current moment and the target road environment information are used as the input of the motion network, and the output of the motion network is the target driving state information of the target vehicle at the next moment. Or after processing the output of the motion network, the target driving state information of the target vehicle at the next moment is obtained.

[0064] The driving area here can be an area in the road network, in which other vehicles and the target vehicle may be driving on the same road, passing the same driving position within a short time difference, or the two vehicles may meet at a certain point on the road. Figure 2 As shown, the target vehicle may be vehicle 2, and the other vehicle may be vehicle 1 at the same intersection as vehicle 2. The same driving area may be the same road or a road that intersects with the road where the target vehicle is located, and the distance between the target vehicle and the other vehicles is less than or equal to the preset distance. If the target vehicle and the other vehicles are separated by several hundred meters, there is no need to consider the collision problem of the two vehicles at the next moment. The preset distance can be determined based on parameters such as the braking distance of the vehicle. In other embodiments, the vehicle driving state prediction method provided in the embodiments of the present application can also be applied to any road, not limited to Figure 2 The number of other vehicles may be one or more. The reference driving state information and the current driving state information may include information of the same category of the respective vehicles. For example, in some embodiments, the reference driving state information and the current driving state information may include parameters such as the driving speed of the target vehicle and other vehicles. For example, the reference driving state information and the current driving state information may include speed information, lane change information, turning information, acceleration information, position information and / or posture information at several moments during the vehicle's driving process. The position information may include the position of the vehicle at the current moment, and may also include the position at several historical moments. Similarly, the speed information may include the speed of the vehicle at the current moment, and may also include the speed of the vehicle at several historical moments. The acceleration information may include the acceleration of the vehicle at the current moment, and may also include the acceleration of the vehicle at several historical moments. The method of obtaining the reference driving state information of other vehicles at the current moment may be to send a data acquisition request to other vehicles and then receive the reference driving state information sent by other vehicles based on the data acquisition request. The data acquisition request may include the identification of the target vehicle and / or the reference driving state information of the target vehicle, so that other vehicles can plan their trajectories according to the current driving state information of the target vehicle. The driving status information between vehicles can be obtained through data sharing, and the specific data sharing method is not specifically limited here. In the Internet of Vehicles environment, real-time data exchange can be carried out between vehicles and between vehicles and roadside infrastructure to share their location, speed, acceleration or planned trajectory information.

[0065] In the above scheme, the target driving state of the target vehicle at the next moment is comprehensively determined based on the driving state information of the target vehicle and the driving state information of other vehicles, thereby reducing the probability of the target vehicle colliding with other vehicles during driving.

[0066] In some embodiments, the target driving state information includes target position information and / or target posture information. Figure 3 and Figure 4 , the above step S13 may further include the following steps:

[0067] Step S131: Based on the current driving state information, the reference driving state information and the target road environment information, determine the possibility that the target vehicle will be in several candidate driving state information at the next moment.

[0068] The candidate driving state information may include candidate position information and / or candidate posture information, or may also include other candidate information such as candidate speed and candidate acceleration that characterize the driving state of the vehicle. Each candidate driving state information is obtained by pre-training. In some application scenarios, the association between multiple candidate positions and the current position in the current driving state information can be established in advance, so as to facilitate the determination of several candidate positions for selection directly based on the association relationship between the current moment and several candidate positions after knowing the current position in the vehicle position prediction process, and the possibility of determining multiple candidate positions in combination with the target road environment information. Among them, the association between multiple candidate positions and the current position can be the relative position relationship between each candidate driving state information and the current position, and each candidate state information can be determined based on the current position and each relative position relationship. Similarly, the association between multiple candidate postures and the position and / or posture in the current driving state information can be established to determine each candidate posture. That is, after the current position of the vehicle is determined, multiple candidate positions and / or candidate postures for selection can be determined. Alternatively, an association may be established between the candidate positions and candidate postures and the current position and current posture, that is, the possibility of being at each candidate position and multiple candidate postures at each candidate position at the next moment may be predicted by the current position and posture.

[0069] The possibility may be any form that can indicate probability, such as confidence or probability. The trained motion network may be used to predict the target driving state information of the target vehicle at the next moment, and the possibility may be output by the motion network.

[0070] Step S132: Select candidate driving state information that is likely to meet a preset state condition from a plurality of candidate driving state information as target driving state information.

[0071] For example, the preset state condition may be the most likely, that is, the candidate driving state information with the most likely possibility may be selected as the target driving state information. Alternatively, in other scenarios, the target driving state information may be obtained by randomly sampling from the first few candidate positions with the most likely possibility.

[0072] In the above scheme, the vehicle's driving trajectory is also complex in a complex traffic environment. For example, at a certain intersection, the vehicle may go straight, turn left or make a U-turn, that is, the vehicle may actually have multiple candidate driving state information at the next moment. This scheme fully considers the possibility of the vehicle being in each candidate driving state information at the next moment, so that the estimated vehicle driving state information is more accurate.

[0073] In some embodiments, the prediction of the target driving state information of the target vehicle at the next moment can be performed by a trained motion network, and the motion network can include a scene cross-attention module. The above step S131 can include the following steps: generating value data and index data in the first key-value pair based on the target road environment information. And, generating query data in the first key-value pair based on the current driving state information and the reference driving state information. On this basis, the first key-value pair is processed as the input of the scene cross-attention module, and the output of the scene cross-attention module is used as the possibility of several candidate driving state information.

[0074] Among them, the method of generating the value data and index data in the first key-value pair based on the target road environment information can be to map the target road environment information into a set of index data (K) and value data (V). Specifically, it can be achieved through different linear transformations. The above-mentioned method of generating the query data in the first key-value pair based on the current driving state information and the reference driving state information can be to map the current driving state information and the reference driving state information to obtain the query data, ensuring that the query, key and value are in the same vector space.

[0075] In the above scheme, the scene cross-attention module can integrate the road environment information into the prediction process of driving state information, so that the driving state of the target vehicle at the next moment can be predicted with the participation of road environment information, which can improve the accuracy of driving state prediction.

[0076] In some embodiments, the current driving state information also includes the current trajectory information of the target vehicle, and the current trajectory information includes the historical driving state information of at least one trajectory point in the preset historical time period in the current trip. The preset historical time period can be a period of time before the current moment, for example, the end time point of the preset time period is the previous moment of the current moment. The historical driving state information can include historical position information and / or historical posture information. In other embodiments, the historical driving state information can also include speed information, lane change information, acceleration information, and turning, and any other information related to the driving control of the vehicle.

[0077] In some embodiments, the prediction of the target driving state information of the target vehicle at the next moment can be performed by a trained motion network, and the motion network can include a motion cross-attention module. The above-mentioned method of generating the query data in the first key-value pair based on the current driving state information and the reference driving state information can be: determining the query data in the second key-value pair based on the current driving state information and the historical driving state information. And determining the value data and index data in the second key-value pair based on the reference driving state information; processing the second key-value pair as the input of the motion cross-attention module, and the output of the motion cross-attention module as the query data of the first key-value pair.

[0078] Among them, based on the current driving state information and the historical driving state information, the method for determining the query data in the second key-value pair can be to splice and map the current driving state information with the historical driving state information to obtain the query data in the second key-value pair, or input the current driving state information and the historical driving state information into a pre-trained attention module, which can be a module in a motion network, and its output is the query data. In some application scenarios, the target driving state information can include vehicle position information, and the reference driving state can include vehicle position information. Here, based on the current driving state information and the historical driving state information, the method for determining the query data in the second key-value pair can be to splice and map the vehicle position information of the target vehicle at the current moment with the historical vehicle position information to obtain the query data in the second key-value pair, or input the vehicle position information of the target vehicle at the current moment with the historical vehicle position information into a pre-trained attention module, and the output of the attention module is the query data. Optionally, the target driving state information includes information such as speed, turning, and lane change, which will not be repeated here.

[0079] As mentioned above, the method for determining the value data and index data in the second key-value pair based on the reference driving state information may be to map the position information and / or posture information of other vehicles at the current moment to obtain the value data and index data in the second key-value pair. Optionally, in the process of determining the value data and index data in the second key-value pair based on the reference driving state information, the position information and / or posture information of other vehicles at the current moment may be fused with the relative position relationship to obtain the value data and index data. The relative position relationship may be the relative position relationship between the origins of the coordinate systems of the target vehicle and other vehicles, or the relative position relationship between the vehicle position information of the target vehicle and the vehicle position information of other vehicles. By embedding the relative position relationship, the local coordinate systems of different vehicles can be distinguished, thereby achieving symmetric encoding.

[0080] In the above scheme, the position information and / or posture information between different vehicles can be fused through the motion cross-attention module. On this basis, the motion interaction between different traffic objects on the road (such as between different vehicles, vehicles and other types of traffic objects) can be modeled, thereby reducing collisions between vehicles.

[0081] In some embodiments, a trained motion network can be used to predict the target driving state information of the target vehicle at the next moment. The motion network can include a time dimension self-attention module. The above-mentioned method of determining the query data in the second key-value pair based on the current driving state information and the historical driving state information can be: based on the historical driving state information of the target vehicle within a preset historical time period, generate query data of the third key-value pair; and, based on the current driving state information, generate value data and index data in the third key-value pair; process the third key-value pair as input to the time dimension self-attention module, and the output of the time dimension self-attention module is used as the query data in the second key-value pair.

[0082] Based on the historical driving status information of the target vehicle within a preset historical time period, the query data of the third key-value pair can be generated by mapping the historical driving status information to obtain the query data. The value data and index data in the third key-value pair can also be generated based on the current driving status information by mapping. The time dimension self-attention module is used to pay attention to the changes in the position information and / or posture information of the target vehicle over time.

[0083] In the above solution, by referring to the historical driving state information of the vehicle in the historical time period, the predicted driving state information of the vehicle at the next moment is made smoother for the position of the vehicle.

[0084] In some embodiments, see Figure 5 , the vehicle driving state prediction method also includes the step of training the road network in the following manner:

[0085] Step S21: discretizing the map containing the road environment information to obtain a number of sample road environment information.

[0086] In some embodiments, a map containing road environment information is discretized to obtain a number of sample road environment information by dividing the map into multiple segments; marking each segment to obtain sample road environment information, each sample road environment information carries one or more of the starting position, end position, length, turning direction and other semantic features of the corresponding segment.

[0087] The length of each segment can be determined according to the distance traveled by the target vehicle between the current moment and the next moment. For example, the distance between the current moment and the next moment is 0.5S. If the vehicle travels a maximum of 5 meters during this period, the length of the segment can be controlled within 5 meters. Exemplarily, each segment can be marked with different colors or different forms of markers. The original road vector feature is represented as a continuous sequence of map points. This scheme divides the original map into multiple segments, each of which is within 5 meters in length, and then uses discrete markers for matching. The final map consists of road vector markers represented by line segments of different colors.

[0088] To enhance the generalization ability of the model, this scheme applies a tokenization process to the road environment information. Optionally, the road network can be a large language model. Each road environment information can be a road vector and a directed lane segment with features, including the start and end positions, length, turn direction, and other semantic features. In order to obtain fine-grained road network input, all road vectors can be divided into labeled segments no longer than 5 meters.

[0089] The process of discretizing the map can be considered as the process of segmenting the road vector. Road vector segmentation (NTP) aims to enable the road network to learn the spatial structure of the road vector input. For example, the original topological information of the road is extracted, and the road network models the road vector markings based on their previous and subsequent relationships, and uses the previous road environment information in the road topology map to predict the subsequent road environment information. In some application scenarios, the road environment information can be understood as road markings. This method requires the road network to understand the connectivity and continuity between disordered road vectors.

[0090] In the above scheme, the map is divided into multiple segments, and each segment is also marked with multiple information, so that the road network can learn the connection relationship between the marked segments.

[0091] Step S22: inputting preset road environment information from a number of sample road environment information into the road network for processing, and outputting the road network as the predicted road environment information at the next moment in the vehicle's traveling direction.

[0092] A multi-head attention module (MHSA) can be used in the road network to model the relationship between road environment information. The road network can be understood as a road sign encoder. The road environment information updated by the road network will be used to assist the decoding of the target vehicle's position information at the current moment to obtain the target driving state information of the target vehicle at the next moment. In some embodiments, please refer to Figure 6In the training process of the road network, a multilayer perceptron (MLP) is set at the output end of the road network, and the multilayer perceptron can obtain the road environment information at the next moment according to the data output by the road network. In other embodiments, the road network includes a backbone network and a multilayer perceptron, and the multilayer perceptron can obtain the road environment information at the next moment according to the data output by the backbone network, that is, the road network may include a multilayer perceptron or may not include a multilayer perceptron.

[0093] Step S23: Adjusting parameters in the road network based on the difference between the predicted road environment information and the actual road environment information.

[0094] Exemplarily, the road prediction loss can be determined based on the difference between the road environment information and the actual road environment information, and then the parameters in the road network can be adjusted based on the road prediction loss. In some application scenarios, the preset road environment information output by the road network is selected from multiple candidate road environment information, and the road prediction loss can be a classification loss. In some application scenarios, the preset road environment information output by the road network is obtained by regression, and the road prediction loss can be a regression loss.

[0095] In the above scheme, after discretizing the map, the road network is trained using sample road environment information, so that the road network can learn the connection relationship between road environments, so that the road environment information at the next moment can be predicted based on the road environment information at the current moment. After the road network training is completed, the vehicle's position information at the next moment can be predicted without relying on the map. The vehicle can continue to operate when entering a new city without repeating the road network training.

[0096] In some embodiments, the current driving state information includes the current location information. The above step of determining the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information can be performed by a trained motion network. The motion network can be a large language model, which is used to serialize and model one or more motion data such as the speed, direction, position, and posture of the vehicle, and use the existing motion data to predict the motion data at the next moment. Exemplarily, the motion network can be a large language model that only includes a Transformer decoder. Please refer to Figure 7 , the method also includes a step of training the motion network:

[0097] Step S31: discretize a number of sample trajectories according to a preset time interval to obtain a trajectory set.

[0098] Exemplarily, each sample trajectory is divided into a trajectory set at a fixed time interval, for example, the interval time t=0.5s. In some embodiments, the trajectory set is clustered according to the position of the trajectory point, for example, using the K-disk algorithm to cluster, and the clustered trajectory set is obtained, and the clustered trajectory set can be used to predict the position information later.

[0099] Step S32: Determine the initial position information of the vehicle to be predicted at time t from the trajectory set.

[0100] The initial position information is the preset position information at the first time point, the disturbed position information, or the predicted position information output by the motion network at time t-1. The initial position may include the initial position and / or the initial posture. Among them, time t refers to the sampling time of the tth trajectory point in the trajectory set. In some application scenarios, if time t is the first moment of the journey, the initial position information may be the preset position information at the first time point. The preset position information may be pre-set, for example, by controlling the vehicle to depart from a preset position, or by visually positioning the vehicle using a shooting component. There are many ways to obtain the predicted position information, which are not specifically limited here. The disturbed position information can be considered as erroneous position information that is different from the correct position information at time t, and is used to simulate the erroneous prediction caused by the cumulative error of the position information prediction process.

[0101] The multiple candidate positions obtained by segmenting the trajectory can be referred to Figure 8 , and the trajectory sampled from it is used as the final proxy motion tag. Fig. 9 As shown, frame 1 at t = 0.5s, frame 2 at 1.0s, and frame 3 at 1.5s represent the tags obtained after discretizing the true trajectory. In each 0.5 second interval, the candidate tags are searched in the tag vocabulary, and the most appropriate (closest) tag is selected to represent the current moment, for example, frame 4 is selected as the position information at 0.5s at 0.5s.

[0102] Step S33: the initial position information and the sample road environment information where the initial position information is located are processed as inputs of the motion network, and the output of the motion network is used as the predicted position information of the vehicle to be predicted at time t+1.

[0103] The method for obtaining the predicted position information can refer to the method for obtaining the target position information mentioned above, which will not be repeated here.

[0104] Step S34: Adjust parameters in the motion network based on the difference between the predicted position information and the actual position information.

[0105] In the above scheme, by discretizing the trajectory of the vehicle to be predicted, the position of the vehicle in each time period or each time point can be obtained, which facilitates the subsequent motion network to learn the movement change rules between each position, thereby realizing the prediction of the vehicle's position information.

[0106] In some embodiments, the predicted position information is selected by the motion network from a plurality of preset candidate position information. Step S34 may be implemented by: determining the cross entropy loss between the predicted position information and the actual position information. Then, the parameters in the motion network are adjusted based on the cross entropy loss.

[0107] For example, the preset candidate positions include straight, left turn, U-turn, and right turn. If the predicted position information is determined to be straight, but the actual position information is a left turn, the classification result is considered inaccurate, and the cross entropy loss is calculated and the parameters in the motion network are adjusted according to the cross entropy loss. The training of the motion network aims to make the motion network understand not only the temporal dependency in vehicle motion, but also the spatial dependency between vehicle motion-map and vehicle motion-other vehicle motion. The motion network is trained by minimizing the cross entropy between the true label distribution and the predicted distribution. The combination of the motion network and the road network as a SMART network, SMART performs autoregression through classification, where the road network can be considered as an encoder for encoding the road map, and the motion network can be considered as a motion decoder based on the motion marker (for example, the motion marker can be the vehicle position information at each moment) embedded in the predicted category distribution. Selecting the category output distribution has a key advantage: it has no restrictions on the structure of the output distribution, enabling the model to learn arbitrary distributions, including multimodal distributions, which can be straight mode, left turn, U-turn, right turn, etc. Vehicle trajectories and road markings from different data sets may follow different output distribution patterns.

[0108] In the above scheme, the motion network is actually a classification model. By classifying the vehicle at the next moment, the motion network can learn the vehicle's motion plan at the next moment, fully consider the possibility of the vehicle being at each candidate position at the next moment, and make the estimated vehicle position information more accurate.

[0109] In some embodiments, the initial position information is disturbed position information. Determining the initial position information of the vehicle to be predicted at time t from the trajectory set includes: in response to receiving the predicted position information at time t-1 output by the motion network, selecting one from other disturbed position information corresponding to time t-1 as the initial position information. The other disturbed position information is other position information other than the predicted position information among several preset candidate position information corresponding to time t-1.

[0110] In some application scenarios, rolling matching is implemented for the entire continuous motion sentence within a given period T, which means that the token at the next moment is matched with reference to the position of the current matching token, rather than relying on the actual correct position, that is, at t = 1.0s, the initial position information is box 4 at 0.5s instead of box 1. However, since the Transformer decoder must perform sequential reasoning step by step, this approach may lead to out-of-distribution problems due to accumulated errors. Especially in the field of autonomous driving, these accumulated errors may lead to collisions and out-of-bounds events. To address this problem, noise is introduced in the tokenization process so that the model simulates distribution shifts during training. Specifically, the current matching token is perturbed by selecting one of the top k tokens that are closest to the true token in the vocabulary. Then, at the next time step, the motion token is matched according to the perturbed vehicle state, for example Fig. 9 At t=1.0s, the closest box to the true box 2 is box 5, but box 6 is selected as the initial position information as a perturbation. That is, at time t=0s, the current vehicle state is used as a reference to select the marker closest to the true bounding box. At time t=0.5s, the matching marker from the previous step is used to select the next predicted marker. At time t=1.0s, the noise-processed marker is used as a reference to determine the marker at t=1.5s. This data augmentation method enables the model to effectively handle problems such as distribution shifts and accumulated errors, thereby enhancing the robustness of the generation task.

[0111] In the above scheme, considering that the prediction process of the position information may result in large errors in the predicted position information due to the existence of cumulative errors, this scheme sets the initial position information as interference position information during the movement of the motion model, so that the motion network can have better prediction results in the future even when there are errors.

[0112] In some application scenarios, the vectorized map and agent trajectory data are modeled as discrete sequence tokens and processed by a decoder-only Transformer architecture to train the next token prediction task across spatiotemporal sequences. This method allows the model to learn the motion distribution in real driving scenarios. That is, the vectorized map and agent trajectory data are converted into discrete sequence tokens. These tokens are processed using a decoder-only Transformer architecture to train the model to predict the next token. During the model training process, an autoregressive prediction task is used to enable the model to gradually predict the interaction of the agent's motion. In addition, the scalability and zero-shot generalization capabilities of the model are ensured through the collection and verification of a large number of data sets.

[0113] That is, in order to apply discrete sequence modeling in continuous vehicle trajectories, this method chooses to discretize real trajectories and map features. Specifically, the continuous trajectories of all agents (vehicles) in the dataset are divided into trajectory sets at fixed time intervals t = 0.5s. Then, the K-disk algorithm is used to cluster the trajectory sets. In each 0.5 second interval, candidate tags are searched in the tag vocabulary, and the most appropriate (closest) tag is selected to represent the current moment. Rolling matching is implemented for the entire continuous motion sentence within a given period T, which means that the tag of the next time step is matched with reference to the position of the current matching tag, rather than relying on the actual correct position. Since the Transformer decoder must perform sequential reasoning step by step, it may cause out-of-distribution problems due to accumulated errors. Especially in the field of autonomous driving, these accumulated errors may lead to collisions and out-of-bounds events. To address this problem, noise is introduced in the tokenization process so that the model simulates distribution shifts during training. Specifically, the current matching tag is perturbed by selecting one of the top k tags closest to the true tag in the vocabulary. Then, at the next time step, the motion tag is matched according to the perturbed vehicle state.

[0114] This data augmentation method enables the model to effectively handle issues such as distribution shifts and accumulated errors, thereby enhancing the robustness of the generation task. To enhance the generalization ability of the model, a tokenization process similar to the agent motion is applied to the road vectors. Each road vector is a directed lane segment with features, including the start and end positions, length, turn direction, and other semantic features in the dataset. To obtain fine-grained road network input, we divide all road vectors into labeled segments no longer than 5 meters. Unlike motion sequences, the tokenization process of road sentences has no time series dependencies. Optionally, the tokenization of road sentences is performed in parallel, directly tokenizing all raw road vector segments.

[0115] The SMART model includes an encoder for road map encoding (road network) and a motion decoder (motion network) based on motion marker embedding prediction category distribution. Road network: road marker encoder. This embodiment uses multi-head self-attention (MHSA) to model the relationship between road markers. The updated road marker encoding will assist in motion marker decoding. Motion network: a factorized agent motion decoder that prioritizes capturing the temporal dynamics of agent motion and then integrates the agent-map and agent-agent interactions. Prioritize capturing the temporal dynamics of agent motion and then integrate the agent-map and agent-agent interactions. Factored attention effectively captures detailed agent-map interactions across time scales. The factored Transformer architecture is used with multi-head cross attention (MHCA) to decode complex road-agent and agent-agent relationships along the time series. In this process, relative position embedding can be used to distinguish the local coordinate frame of the agent to achieve symmetric encoding.

[0116] The training of the road network enables the road network to learn the spatial structure of the road vector input. Unlike agent motion, road vectors form a graph rather than a sequence, making it challenging to directly apply the next label prediction task. To address this problem, the present embodiment extracts the original topological information of the road and models the road vector labels based on their contextual relationships. During the road network training process, the previous road label based on the road topology is used to predict the subsequent road label. This method requires the road network to understand the connectivity and continuity between disordered road vectors. The training of the motion network enables the motion network to understand not only the temporal dependencies in agent motion, but also the spatial dependencies between agent-maps and agent-agents. SMART is trained by minimizing the cross entropy between the true label label distribution and the predicted distribution. Optionally, the motion network predicts the future motion labels of multiple agents based on previous motion labels, interactive agent motion labels, and encoded road labels, and the model is trained to predict the next motion label.

[0117] In this solution, the features of both time and space dimensions (vehicle historical trajectory and map lane vector) are discretized and marked, making the trained SMART model more generalizable. In addition, by using classification loss as the task loss function, strict alignment with the loss function of the large language model (LLM model) is achieved. In addition, the SMART model provided by this solution has been verified to have the ability to generalize across data sets and the scalability of the model capability to increase as the data increment continues to accumulate.

[0118] In some application scenarios, the vehicle driving state prediction method provided by this solution can solve practical problems in autonomous driving simulation modeling. For example, during the testing and development of autonomous driving vehicles, users can use the method of the present invention to generate realistic traffic scenarios to evaluate and optimize the performance of autonomous driving algorithms.

[0119] The above scheme can be applied to autonomous vehicles, especially for autonomous driving strategy planning in urban traffic and complex road conditions. In the connected vehicle environment, multiple autonomous vehicles use the SMART model for motion prediction and path planning. The agent uses the motion tokens generated by the model to adjust its driving path in real time to avoid collisions and optimize driving efficiency, thereby navigating safely and efficiently in the urban environment during peak hours.

[0120] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0121] See also Fig.10 , Fig.10 It is a structural diagram of an embodiment of a driving state prediction device for a vehicle of the present application. The driving state prediction device 40 of the vehicle includes an information acquisition unit 41, a road prediction unit 42 and a state information prediction unit 43. The information acquisition unit 41 is used to obtain the current driving state information of the target vehicle at the current moment and the initial road environment information at the current moment; the road prediction unit 42 is used to extract features of the initial road environment information at the current moment using the road network and predict the target road environment information at the next moment based on the extracted features, and the road network is obtained by training based on the sample road environment information obtained by discretizing the map; the state information prediction unit 43 is used to determine the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information.

[0122] In some embodiments, the information acquisition unit 41 is also used to: obtain reference driving state information of other vehicles at the current moment, where the other vehicles are vehicles in the same driving area as the target vehicle, and the state information prediction unit 43 determines the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information, including: determining the target driving state information of the target vehicle at the next moment based on the current driving state information, the reference driving state information and the target road environment information.

[0123] In some embodiments, the target driving state information includes target position information and / or target posture information. The state information prediction unit 43 determines the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information, including: based on the current driving state information, the reference driving state information and the target road environment information, determining the possibility that the target vehicle will be in several candidate driving state information at the next moment, respectively, the candidate driving state information includes candidate position information and / or candidate posture information, and each candidate driving state information is pre-trained; selecting candidate driving state information that is likely to meet a preset state condition from several candidate driving state information as the target driving state information.

[0124] In some embodiments, the current driving state information of the target vehicle at the current moment is the target driving state information determined at the previous moment, and the state information prediction unit 43 is also used to: use the determined target driving state information of the target vehicle at the next moment as the current driving state information at the next moment, so as to predict the target driving state information at the next moment based on the determined current driving state information of the target vehicle at the next moment.

[0125] In some embodiments, the state information prediction unit 43 determines the possibility that the target vehicle will be in several candidate driving state information at the next moment based on the current driving state information, the reference driving state information and the target road environment information, including: generating value data and index data in the first key-value pair based on the target road environment information; and generating query data in the first key-value pair based on the current driving state information and the reference driving state information; processing the first key-value pair as the input of the scene cross-attention module, and the output of the scene cross-attention module as the possibility of several candidate driving state information.

[0126] In the above scheme, the scene cross-attention module can integrate the road environment information into the prediction process of driving state information, so that the driving state of the target vehicle at the next moment can be predicted with the participation of road environment information, which can improve the accuracy of driving state prediction.

[0127] In some embodiments, the current driving status information also includes the current trajectory information of the target vehicle, and the current trajectory information includes the historical driving status information of at least one trajectory point within a preset historical time period of the current trip; the status information prediction unit 43 generates query data in the first key-value pair based on the current driving status information and the reference driving status information, including: determining the query data in the second key-value pair based on the current driving status information and the historical driving status information; and determining the value data and index data in the second key-value pair based on the reference driving status information; processing the second key-value pair as input to the motion cross-attention module, and the output of the motion cross-attention module as the query data of the first key-value pair.

[0128] In some embodiments, the state information prediction unit 43 determines the query data in the second key-value pair based on the current driving state information and the historical driving state information, including: generating query data of the third key-value pair based on the historical driving state information of the target vehicle within a preset historical time period; and generating value data and index data in the third key-value pair based on the current driving state information; processing the third key-value pair as input of the time dimension self-attention module, and the output of the time dimension self-attention module as the query data in the second key-value pair.

[0129] In some embodiments, the vehicle's driving state prediction device 40 also includes a training module (not shown), which is used to execute the steps of training the road network in the following manner: discretizing the map containing the road environment information to obtain a number of sample road environment information; inputting the preset road environment information in the number of sample road environment information into the road network for processing, and the output of the road network is used as the predicted road environment information at the next moment in the vehicle's traveling direction; and adjusting the parameters in the road network based on the difference between the predicted road environment information and the actual road environment information.

[0130] In some embodiments, the training module discretizes the map containing the road environment information to obtain a number of sample road environment information, including: dividing the map into multiple segments; marking each segment to obtain each sample road environment information, each sample road environment information carries one or more of the starting position, end position, length, turning direction and other semantic features of the corresponding segment.

[0131] In some embodiments, the current driving state information includes position information. Based on the current driving state information and the target road environment information, the step of determining the target driving state information of the target vehicle at the next moment is performed by the motion network, and the training module is used to perform the training steps for the motion network: discretizing a number of sample trajectories according to a preset time interval to obtain a trajectory set; determining the initial position information of the vehicle to be predicted at time t from the trajectory set, the initial position information being the preset position information at the first time point, the perturbation position information or the predicted position information of the vehicle to be predicted at time t-1 predicted by the motion network; processing the initial position information and the sample road environment information where the initial position information is located as the input of the motion network, and the output of the motion network being the predicted position information of the vehicle to be predicted at time t+1; adjusting the parameters in the motion network based on the difference between the predicted position information and the actual position information.

[0132] In some embodiments, the predicted position information is selected by the motion network from a number of preset candidate position information; the training module adjusts the parameters in the motion network based on the difference between the predicted position information and the actual position information, including: determining the cross entropy loss between the predicted position information and the actual position information; adjusting the parameters in the motion network based on the cross entropy loss.

[0133] In some embodiments, the initial position information is disturbed position information, and the training module determines the initial position information of the vehicle to be predicted at time t from the trajectory set, including: in response to receiving the predicted position information at time t-1 output by the motion network, selecting one from other interfering position information corresponding to time t-1 as the initial position information; wherein the other interfering position information is other position information other than the predicted position information among several preset candidate position information corresponding to time t-1.

[0134] The above-mentioned driving state prediction device corresponds one-to-one to each step in the aforementioned embodiment of the driving state prediction method. Its specific implementation method and corresponding technical effects refer to the description of the corresponding steps in the aforementioned embodiment of the driving state prediction method, and will not be repeated here.

[0135] See also Fig.11 , Fig.11 The device 50 includes a memory 51 and a processor 52 coupled to each other, and the processor 52 is used to execute program instructions stored in the memory 51 to implement the steps in any of the above-mentioned vehicle driving state prediction method embodiments.

[0136] The device 50 may be a vehicle, or a device that establishes a communication connection with a vehicle, or may also be a simulation device. Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps of the driving state prediction method embodiment of any of the above-mentioned vehicles, or to implement the steps in the driving state prediction method embodiment of any of the above-mentioned vehicles. The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 52 may be implemented by an integrated circuit chip.

[0137] See also Fig.12 , Fig.12The computer-readable storage medium 60 stores program instructions 601 that can be executed by a processor, and the program instructions 601 are used to implement the steps in any of the above-mentioned vehicle driving state prediction method embodiments.

[0138] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0139] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0143] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A method for predicting a vehicle's driving state, characterized in that: include: Obtain the current driving state information of the target vehicle at the current moment and the initial road environment information at the current moment; Extracting features from the initial road environment information at the current moment using a road network and predicting target road environment information at the next moment based on the extracted features, wherein the road network is trained based on sample road environment information obtained by discretizing a map; Based on the current driving state information and the target road environment information, target driving state information of the target vehicle at the next moment is determined.

2. The method according to claim 1, characterized in that The method further comprises: Acquire reference driving state information of other vehicles at the current moment, wherein the other vehicles include vehicles in the same driving area as the target vehicle; The determining, based on the current driving state information and the target road environment information, target driving state information of the target vehicle at the next moment includes: Based on the current driving state information, the reference driving state information and the target road environment information, the target driving state information of the target vehicle at the next moment is determined.

3. The method according to claim 2, characterized in that The determining, based on the current driving state information, the reference driving state information and the target road environment information, the target driving state information of the target vehicle at the next moment comprises: Based on the current driving state information, the reference driving state information and the target road environment information, determine the possibility that the target vehicle is in a plurality of candidate driving state information at the next moment, wherein the candidate driving state information includes candidate position information and / or candidate posture information, and each of the candidate driving state information is obtained by pre-training; The candidate driving state information that is likely to meet the preset state condition is selected from the plurality of candidate driving state information as the target driving state information.

4. The method according to claim 3, characterized in that The current driving state information of the target vehicle at the current moment is the target driving state information determined at the previous moment, and the method further includes: The determined target driving state information of the target vehicle at the next moment is used as the current driving state information at the next moment, so as to predict the target driving state information at the next moment based on the determined current driving state information of the target vehicle at the next moment.

5. The method according to claim 3 or 4, characterized in that: The determining, based on the current driving state information, the reference driving state information and the target road environment information, the possibility that the target vehicle is in a plurality of candidate driving state information at the next moment, comprises: Generate value data and index data in a first key-value pair based on the target road environment information; and generate query data in the first key-value pair based on the current driving state information and the reference driving state information; The first key-value pair is processed as the input of a scene cross-attention module, and the output of the scene cross-attention module is used as the possibility of the several candidate driving status information.

6. The method according to claim 5, characterized in that The current driving state information also includes current track information of the target vehicle, and the current track information includes historical driving state information of at least one track point within a preset historical time period in the current trip; the query data in the first key-value pair is generated based on the current driving state information and the reference driving state information, including: Determine query data in a second key-value pair based on the current driving state information and the historical driving state information; and determine value data and index data in the second key-value pair based on the reference driving state information; The second key-value pair is processed as input of a motion cross-attention module, and the output of the motion cross-attention module is used as query data for the first key-value pair.

7. The method according to claim 6, characterized in that The determining the query data in the second key-value pair based on the current driving state information and the historical driving state information includes: Based on the historical driving state information, generate query data of a third key-value pair; and based on the current driving state information, generate value data and index data in the third key-value pair; The third key-value pair is processed as input of a time dimension self-attention module, and the output of the time dimension self-attention module is used as query data in the second key-value pair.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises the step of training the road network in the following manner: Discretize the map containing the road environment information to obtain a number of sample road environment information; Inputting preset road environment information from the plurality of sample road environment information into the road network for processing, and outputting the road network as predicted road environment information at the next moment in the vehicle's traveling direction; Based on the difference between the predicted road environment information and the actual road environment information, parameters in the road network are adjusted.

9. The method according to claim 8, characterized in that The map containing the road environment information is discretized to obtain a number of sample road environment information, including: dividing the map into a plurality of segments; Each of the segments is marked to obtain each of the sample road environment information, wherein each of the sample road environment information carries one or more of the starting position, end position, length, turning direction and other semantic features of the corresponding segment.

10. The method according to any one of claims 1 to 9, characterized in that The current driving state information includes position information, and the step of determining the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information is performed by a motion network. The method further includes a step of training the motion network in the following manner: Discretize a number of sample trajectories according to a preset time interval to obtain a trajectory set; Determine the initial position information of the vehicle to be predicted at time t from the trajectory set, where the initial position information is the preset position information at the first time point, the disturbed position information, or the predicted position information of the vehicle to be predicted at time t-1 predicted by the motion network; The initial position information and the sample road environment information where the initial position information is located are processed as inputs of the motion network, and the output of the motion network is used as the predicted position information of the vehicle to be predicted at time t+1; Based on the difference between the predicted position information and the actual position information, parameters in the motion network are adjusted.

11. The method according to claim 10, characterized in that The predicted position information is selected by the motion network from a plurality of preset candidate position information; and adjusting the parameters in the motion network based on the difference between the predicted position information and the actual position information includes: Determining a cross entropy loss between the predicted position information and the true position information; Parameters in the motion network are adjusted based on the cross entropy loss.

12. The method according to claim 10, characterized in that The initial position information is the disturbed position information, and the determining the initial position information of the vehicle to be predicted at time t from the trajectory set includes: In response to receiving the predicted position information at time t-1 output by the motion network, selecting one from other interference position information corresponding to the time t-1 as the initial position information; The other interference position information is other position information among several preset candidate position information corresponding to the time t-1 except the predicted position information.

13. A vehicle driving state prediction device, characterized in that: include: An information acquisition unit, used to acquire the current driving state information of the target vehicle at the current moment and the initial road environment information at the current moment; A road prediction unit, configured to extract features from the initial road environment information at a current moment using a road network and predict target road environment information at a next moment based on the extracted features, wherein the road network is trained based on sample road environment information obtained by discretizing a map; The state information prediction unit is used to determine the target driving state information of the target vehicle at the next moment based on the current driving state information and the target road environment information.

14. A device, characterized in that It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the vehicle driving state prediction method as described in any one of claims 1 to 12.

15. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the vehicle driving state prediction method described in any one of claims 1 to 12 is implemented.

Images