Vehicle state prediction method and device

Abstract

The present application discloses a method and device for predicting vehicle status, and relates to the field of software technology. The method of the present application includes: obtaining the vehicle's driving status information, vehicle-road relationship information, and historical decision information, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving status information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within the historical target period; based on the driving status information, the vehicle-road relationship information, and the historical decision information, a vehicle status prediction model is used to perform a prediction operation to obtain a prediction result, wherein the vehicle status prediction model is trained by an attention mechanism algorithm using historical behavior status information, historical vehicle-road relationship information, and historical decision information. The present application is used to realize the prediction function of the vehicle status.

Description

Technical Field

[0001] The present application relates to the field of software technology, and in particular to a method and device for predicting vehicle status. Background Art

[0002] With the development of technology, the prediction of vehicle behavior has also become a major development direction in intelligent transportation and simulation scenarios.

[0003] At present, in the existing vehicle state prediction process, prediction is generally performed based on the vehicle's speed and acceleration, combined with a pre-set logical expression, that is, after determining the vehicle's speed and acceleration, the logical expression is used for calculation and analysis to determine the vehicle's driving state. However, in actual applications, the parameters set in these logical expressions are generally set based on experience, which results in the vehicle state prediction result being affected if the expert who sets the expression is not experienced enough, resulting in a low accuracy of the vehicle state prediction result. Summary of the invention

[0004] The embodiments of the present application provide a method and device for predicting a vehicle state, the main purpose of which is to implement a method for predicting a vehicle state to solve the problem of low accuracy of existing vehicle state prediction results.

[0005] In order to solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, the present application provides a method for predicting a vehicle state, the method comprising:

[0007] Acquire driving state information, vehicle-road relationship information, and historical decision information of the vehicle, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within a historical target period;

[0008] Based on the driving state information, the vehicle-road relationship information and the historical decision information, a prediction operation is performed using a vehicle state prediction model to obtain a prediction result, wherein the vehicle state prediction model is trained using the historical driving state information, the historical vehicle-road relationship information and the historical decision information via an attention mechanism algorithm.

[0009] In a second aspect, the present application further provides a vehicle state prediction device, the device comprising:

[0010] A setting unit is used to set a model framework of the vehicle state prediction model; the model framework includes a first input module, a second input module, a third input module and a prediction execution module; wherein the first input module is used to input vehicle-road relationship information, the second input module is used to input driving state information, and the third input module is used to input historical decision information; the prediction execution module is used to input the input results of the first input module, the second input module and the third input module and execute prediction;

[0011] A training set acquisition unit, used to acquire a target training set, wherein the training samples in the target training set include historical driving state information, historical vehicle-road relationship information, historical decision information, and corresponding actual operating state information;

[0012] A training unit is used to perform training operations on the target training set and the model framework based on the attention mechanism algorithm to generate the vehicle state prediction model.

[0013] In a third aspect, the present application further provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the vehicle status prediction method described in the first aspect.

[0014] In a fourth aspect, the present application also provides a vehicle status prediction device, the device comprising a storage medium; and one or more processors, the storage medium is coupled to the processor, the processor is configured to execute program instructions stored in the storage medium; when the program instructions are executed, the vehicle status prediction method described in any one of the first aspects is executed.

[0015] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:

[0016] The present application provides a method and device for predicting a vehicle state. The present application can first obtain the vehicle's driving state information, vehicle-road relationship information and historical decision information, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within a historical target time period; then, based on the driving state information, the vehicle-road relationship information and the historical decision information, a vehicle state prediction model is used to perform a prediction operation to obtain a prediction result, wherein the vehicle state prediction model is trained through an attention mechanism algorithm using historical driving state information, historical vehicle-road relationship information and historical decision information, thereby realizing the vehicle state prediction function. Compared with the prior art, in the prediction process of the vehicle state in the present application, the prediction is based on the vehicle state prediction model, which is trained based on historical driving state information, historical vehicle-road relationship information and historical decision information, that is, the vehicle state prediction model is actually trained based on the historical driving situation of the vehicle as a sample. Different from the parameters of the expression based on manual setting in the prior art, the model does not need to set the parameters manually, which avoids the threshold requirement for setting experience in the manual setting process, and also avoids the problem that the set parameters are not accurate enough due to insufficient experience, thus affecting the accuracy of the subsequent prediction results, thereby improving the accuracy of the prediction results. At the same time, in the present application, the vehicle state prediction method can use the three aspects of vehicle driving state information, vehicle-road relationship and historical decision information as the basis for prediction. Compared with the prior art, which only predicts the vehicle's own driving situation such as the acceleration and speed of the vehicle, the present application adds the consideration of the relationship between the vehicle and the road and the vehicle's historical decision, so that the final predicted vehicle state can be comprehensively analyzed and predicted from more aspects, so that the accuracy of the prediction results can be further improved.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0019] Figure 1 A flow chart of a method for predicting vehicle status provided by an embodiment of the present application is shown;

[0020] Figure 2 A flow chart of another method for predicting vehicle status provided by an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of the structure of a model framework when executing a method for predicting vehicle status provided by an embodiment of the present application is shown;

[0022] Figure 4 A block diagram showing a vehicle state prediction device provided by an embodiment of the present application is shown;

[0023] Figure 5 A block diagram showing the composition of another vehicle status prediction device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0025] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0026] The present application embodiment provides a flow chart of a method for predicting vehicle status, such as Figure 1 As shown, the method includes: 101, obtaining vehicle driving state information, vehicle road relationship information and historical decision information.

[0027] Among them, the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within the historical target period.

[0028] In this embodiment, the driving state information can be understood as some driving-related information involved in the vehicle driving process, such as the vehicle's speed, acceleration, the vehicle's destination, the vehicle's starting point, etc. Of course, in actual applications, users can select from these and other related information based on their needs, and this is not limited here. In addition, the vehicle-road relationship information in this embodiment can be understood as a type of information that can express the relationship between the vehicle and the road calculated based on the driving state information, such as the section of the road the vehicle is traveling on, the specific position of the vehicle on the road, etc. This information can be calculated based on the vehicle's driving state information combined with road-related information. In addition, the historical decision information in this embodiment can be understood as the relationship between the vehicle and the road during the vehicle's previous driving on the same road, and can also be understood as the relationship between the vehicle and the road at the previous moment.

[0029] It should be noted that the above-mentioned driving status information, vehicle-road relationship information and historical decision information can be collected and calculated through the vehicle's own vehicle terminal, or can be obtained through the interaction between the vehicle terminal and the remote cloud. In this regard, the method of obtaining the above-mentioned data includes but is not limited to any of the above-mentioned methods, and users can choose according to their needs.

[0030] 102. Based on the driving state information, the vehicle-road relationship information and the historical decision information, a prediction operation is performed using a vehicle state prediction model to obtain a prediction result.

[0031] Among them, the vehicle state prediction model is trained by the attention mechanism algorithm through historical driving state information, historical vehicle-road relationship information and historical decision information.

[0032] After the aforementioned steps have acquired the driving state information, the vehicle-road relationship information and the historical decision information, in this embodiment, a prediction operation can be performed based on these three data through a pre-trained prediction model, wherein the prediction model described in this embodiment, i.e., the vehicle state prediction model, is based on the historical driving state information, the historical vehicle-road relationship information and the historical decision information, and is obtained by training through an attention mechanism algorithm. Since the samples in the training process are the historical data of the three data acquired in the aforementioned steps, prediction based on the model can accurately analyze the current vehicle driving conditions and obtain corresponding prediction results. Compared with conventional predictions through expressions and related parameters, the prediction model can analyze the characteristics of vehicle driving based on samples, and based on this, make predictions from the currently collected data (i.e., driving state information, vehicle-road relationship information and historical decision information), which is more scientific and accurate, and avoids the problem of affecting the accuracy of prediction results due to low parameter accuracy when manually setting expression parameters.

[0033] In addition, in this embodiment, the prediction model is trained based on the attention mechanism algorithm, which can be understood as a way of simulating human attention, and realizing the attention to important information and ignoring unimportant information by weighted processing of input data. The algorithm usually includes three main steps: 1. Calculate the attention weight: According to the characteristics of the input data, calculate the attention weight corresponding to each feature to determine its importance. This is usually achieved by using a neural network to learn and predict the attention weight; 2. Weighted processing: multiply the input data by the attention weight to obtain a weighted data representation, emphasizing important features and weakening unimportant features; 3. Attention aggregation: aggregate the weighted data representation to obtain the final attention output. This usually includes summing or averaging the weighted data to obtain the final attention representation; through the above steps, the attention mechanism algorithm can realize the attention and ignoring of the input data, thereby improving the model's perception of important information. In other words, by calculating several important input items in the above manner, the "score" of the "attention" concerned at the end can be obtained, and the "score" can reflect the calculation result of paying more "attention" to certain data. Therefore, by training the model through the attention mechanism algorithm, it is possible to ensure that when there are many input data items, the characteristics of "attention" can be combined to obtain relatively more accurate prediction results, thereby avoiding the interference of certain less important data items on the entire prediction results, and improving the accuracy of the prediction results. In this way, it can be ensured that in the scenario of intelligent transportation, the method of this embodiment can be used to select more important data items from a variety of data items for "attention", and then predict the operation of the vehicle, avoiding the influence of other unimportant parameters on the prediction results, thereby achieving more accurate prediction of the vehicle operation.

[0034] Based on this, the present embodiment provides a method for predicting vehicle status. Compared with the prior art, in the prediction process of the vehicle status in the present application, the prediction is based on the vehicle status prediction model, which is trained based on historical driving status information, historical vehicle-road relationship information and historical decision information, that is, the vehicle status prediction model is actually trained based on the historical driving situation of the vehicle as a sample. Different from the parameters of the expression based on manual setting in the prior art, the model does not need to manually set the parameters, which avoids the threshold requirement for setting experience in the manual setting process, and also avoids the problem that the set parameters are not accurate enough due to insufficient experience, thereby affecting the accuracy of subsequent prediction results, thereby improving the accuracy of the prediction results. At the same time, in the present application, the method for predicting the vehicle status can use the three aspects of the vehicle's driving status information, the vehicle-road relationship and the historical decision information as the basis for prediction. Compared with the prior art, which only predicts the vehicle's own driving situation such as the acceleration and speed of the vehicle, the present application adds the consideration of the mutual relationship between the vehicle and the road and the vehicle's historical decision, so that the final predicted vehicle status can be comprehensively analyzed and predicted from more aspects, so that the accuracy of the prediction results can be further improved.

[0035] Furthermore, as a further description and refinement of the above-mentioned embodiment, the present application also provides a prediction of vehicle status, specifically as follows: Figure 2 As shown:

[0036] 201. Set a model framework of the vehicle state prediction model.

[0037] The model framework includes a first input module, a second input module, a third input module and a prediction execution module.

[0038] Among them, the first input module is used to input vehicle-road relationship information, the second input module is used to input driving status information, and the third input module is used to input historical decision information; the prediction execution module is used to input the input results of the first input module, the second input module and the third input module and execute the prediction.

[0039] In this embodiment, the model framework of the vehicle state prediction model is specifically divided into two parts, one of which is a module specifically used to input various information, and the other is a module specifically used to make predictions, that is, the first input module, the second input module and the third input module are all the first part of the aforementioned model framework, and the prediction execution module is the second part. Of course, in practical applications, the input data may still need to undergo certain encoding processing, that is, different input data needs to be converted into encoding information that can be easily identified during subsequent predictions. Therefore, after the first input module, the second input module and the third input module, a corresponding encoding module needs to be added on the basis of each input module, so as to convert the received vehicle-road relationship information, driving state information and historical decision information into corresponding encoding information.

[0040] In addition, in practical applications, the schematic diagram of the structure of the model framework can be found in Figure 3 , wherein the vehicle-road relationship information described in this embodiment may not be a single pre-processed data, but may be data determined based on the road information combined with the vehicle's driving state information. In this case, the first input module in the model framework of this embodiment may also be specifically composed of the Figure 3 The lane information module is used to receive lane-related information, while the vehicle-road interaction module is used to analyze and calculate the relationship between the vehicle and the road based on the driving status information transmitted from the second input module and the lane-related information, that is, the vehicle-road relationship information. Of course, in this embodiment, the vehicle driving status information can actually also include the relationship between vehicles. In this embodiment, the model framework can be used to analyze and calculate the driving information module and the vehicle interaction module included in the second input module. The driving information module is used to receive the driving status information of the vehicle, and in the Figure 3 In the example, the third input module is actually composed of a historical decision module, which is used to receive historical decision information. Of course, as described above, in the actual prediction process, the prediction is not directly based on the vehicle-road relationship information, driving state information and historical decision information, but needs to be converted into corresponding coded information. Figure 3 A corresponding encoding module is also provided in order to convert the information received from the outside into corresponding encoding information.

[0041] 202. Obtain a target training set.

[0042] The training samples in the target training set include historical driving status information, historical vehicle-road relationship information, historical decision information and corresponding actual operating status information.

[0043] In this embodiment, since the training process is actually the process of adjusting the specific parameters based on the model framework based on the sample data, it is necessary to first obtain the training samples required for training in this process, that is, the target training set. Of course, the information contained in the target training set is data of known specific driving conditions of the vehicle, that is, the recorded historical data, which includes historical driving state information, historical vehicle-road relationship information and actual operating state information. The actual operating state information is information that characterizes the specific driving conditions of the vehicle, corresponding to the prediction results after the execution of the prediction model of this embodiment, that is, by training the data of the known results as samples, it can be ensured that the model can determine the specific characteristics of the vehicle's driving based on the samples, thereby ensuring that when the subsequent prediction is made again, the vehicle's subsequent driving conditions and trajectories can be analyzed based on the real-time driving state information, vehicle-road relationship information and historical decision information during the current vehicle's driving process, thereby ensuring that the prediction results are closer to the real results, making the prediction results more accurate.

[0044] 203. Perform training operations on the target training set and the model framework based on the attention mechanism algorithm to generate the vehicle state prediction model.

[0045] Specifically, during the model training process, this step may further include the following steps:

[0046] First, a loss function is set by using the actual result and the predicted result, and a distortion ratio parameter of the loss function is determined;

[0047] Then, the historical vehicle-road relationship information of the training sample is input into the first input module of the model framework, the historical driving state information of the training sample is input into the second input module of the model framework, and the historical decision information of the training sample is input into the third input module, and the prediction module is controlled to perform prediction in combination with the attention mechanism algorithm to obtain a simulation prediction result;

[0048] Afterwards, the simulation prediction result is compared with the corresponding actual operation status information, and the prediction parameters of the model framework are determined based on the comparison result and the loss function;

[0049] Finally, the vehicle state prediction model is generated based on the model framework and the prediction parameters.

[0050] In this embodiment, the loss function can be understood as a function for converging and controlling the model during training, which can ensure the closeness between the predicted results and the actual results. Specifically, the loss function can be:

[0051]

[0052] in, and are the average error and true error between the predicted results and the true results, respectively. is the loss ratio parameter, while λ1 and λ2 are the prediction parameters.

[0053] Then, based on the control of the loss function during the model training process, we can ensure that and as well as The prediction parameters that are specifically suitable for the model are deduced in reverse, that is, the more suitable λ1 and λ2 are solved, so that the prediction parameters that match the model framework are obtained, realizing the parameter optimization function of the vehicle state prediction model constructed based on the neural network.

[0054] In addition, based on the description of the above steps, it can be seen that the model framework in the vehicle state prediction model in this embodiment includes a vehicle interaction module for calculating the relationship between vehicles and a lane interaction module for calculating the relationship between vehicles and roads. In this case, the actual calculation process of these two modules can be:

[0055] The vehicle interaction module is mainly used to calculate the impact of surrounding vehicles on the target vehicle. The function of this vehicle interaction module mainly depends on the attention mechanism algorithm. Specifically, the calculation method of the attention mechanism algorithm is as follows:

[0056] Q=qW Q K = kW K V=vW V

[0057]

[0058] Among them, q, k, v are the inputs of the attention mechanism algorithm; W Q , W K , W V are matrices with different parameters, used to convert the input vector to a higher dimension for calculation; d k is the dimension of vector K. In the vehicle interaction module, the target vehicle’s code is used as the q vector, and the surrounding vehicle’s code is used as the k vector and v vector. The attention mechanism is used to calculate and obtain a new target vehicle code. In addition, q, k, v are generally in the form of [N, D], where N is the number of vehicles or roads, and D is the dimension of vehicle or road information. Q , W K , W V They are matrices for information representation, that is, converting the original input information from D dimensions to a higher-dimensional space. The parameter values ​​in the matrix are determined during deep learning training. k It is the dimension of the high-dimensional space after being represented by the W matrix.

[0059] In addition, the lane interaction module is used to calculate the impact of lane information on the target vehicle. The vehicle-road interaction module is also composed of a mutual attention mechanism. In this module, the target vehicle code output by the vehicle interaction module is used. As q vector, the encoding vector l of the lane segment i As k vector and v vector. The vehicle-road interaction module will calculate a new target vehicle code as output.

[0060] In addition, in this embodiment, the encoding vector l of the lane segment i The determination process of can be as follows: according to the Euclidean distance from the vehicle position at time t, the vectors of the sampling points and lane segments on the lane centerline with a relatively close distance are selected and written as {(p i ,V i ),i∈{1,2,…,N l}}, where N l is the number of lane segments after screening. The lane information encoding module encodes the information of each lane segment. The lane information encoding module is composed of a multi-layer perceptron. The input is a 4-dimensional vector consisting of a sampling point and a vector connection. The output is a 64-dimensional high-dimensional encoding vector, denoted as l i .

[0061] 204. Obtain vehicle driving status information, vehicle-road relationship information, and historical decision information.

[0062] Among them, the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within the historical target period.

[0063] Based on the construction and training of the prediction model in the aforementioned steps, it can be known that the vehicle-road relationship in this embodiment can be performed based on the description of the aforementioned step 203. Of course, in some cases, in this embodiment, the determination process of the vehicle-road relationship can also be determined by another method, which is specifically:

[0064] Acquiring lane information, wherein the lane information at least includes a lane route and the lane width;

[0065] Acquiring driving status information of the vehicle, the driving status information including at least speed, acceleration, driving time, starting position, and width from the roadside;

[0066] Calculating the position coordinates of the vehicle based on the speed, acceleration, driving time and starting position, and determining the route position of the vehicle based on the position coordinates and the lane route;

[0067] Calculating a Euclidean distance from the vehicle to a lane centerline based on the lane width and the width from the roadside, and determining a lane position of the vehicle based on the Euclidean distance;

[0068] The line position and the lane position are determined as the vehicle-road relationship information.

[0069] In this way, it is ensured that in the process of obtaining the vehicle-road relationship information, the line position of the vehicle in the lane route can be determined based on the acceleration, speed, driving time and starting position of the vehicle, and the Euclidean distance from the vehicle to the lane centerline can be calculated based on the lane width and the distance to the roadside width, and the lane position is determined based on this, which is equivalent to determining the x-axis and y-axis of the vehicle in the line formed by this road. In this way, the accuracy of the obtained vehicle-road relationship information is ensured.

[0070] Furthermore, in this embodiment, the process of obtaining lane information may also be specifically as follows:

[0071] First, collecting road sign information of the lane, and determining a lane route of the lane based on the road sign information;

[0072] Then, based on a preset distance, the lane route is divided into a plurality of lane segments to obtain a lane queue, wherein an arrangement order of the lane segments in the lane queue corresponds to positions of the lane segments in the lane route;

[0073] Finally, the centerline position and the edge position of each lane segment are marked respectively, and the lane width of the lane segment is determined based on the centerline position and the edge position to obtain the lane information.

[0074] In this embodiment, the road sign information can actually be collected by using a drone hovering at a fixed position above the collection scene to shoot a specific traffic section. At the same time, the lane route can be obtained based on cloud data. After the lane route is determined, the route can be segmented, that is, divided according to a preset distance to obtain multiple lane segments, and then a queue is formed based on these lane segments, namely a lane queue. Specifically, the specific content and form of the lane queue can be: In this example, the position of the lane in the video screenshot taken by the drone is marked, and the transfer matrix between the marked map coordinate system and the trajectory data coordinate system is calculated. The map scene is usually composed of multiple lanes. In this example, each lane is split into multiple segments, each segment is represented by a vector, and the final extracted map lane information can be expressed as follows: {(p i ,V i ),i∈{1,2,…,N ξ}}, where i is the number of the lane segment, N ξ is the number of all lane segments in the scene, p i is the starting point coordinate of the i-th lane segment, V i is the vector of the i-th lane segment.

[0075] 205. Based on the driving state information, the vehicle-road relationship information and the historical decision information, a prediction operation is performed using a vehicle state prediction model to obtain a prediction result.

[0076] Among them, the vehicle state prediction model is trained by the attention mechanism algorithm through historical driving state information, historical vehicle-road relationship information and historical decision information.

[0077] It should be noted that, in this embodiment, the historical decision information is specifically a vehicle-road relationship queue; the vehicle-road relationship in the historical decision information is specifically a lane segment relationship, wherein each lane segment relationship is determined based on the lane segment and the driving status information.

[0078] Specifically, in combination with the above description of the steps, it can be seen that the prediction model of this embodiment is actually composed of multiple different modules. Based on this, this step can be as follows when performing the prediction behavior:

[0079] Based on the above, the function of this module is to store the output of the vehicle-road interaction module at the time step before time t. Vectors, which are calculated in the form of sequences in the temporal coding module through the self-attention mechanism. In the coding module corresponding to the historical decision module, a 64-dimensional, randomly initialized vector is first added to the end of the historical coding vector sequence, and the sequence is stacked as the q vector, k vector and v vector of the attention mechanism. After calculation, a 64-dimensional coding vector sequence is obtained. The last element of the sequence is taken and recorded as As the calculation result of the timing encoding module.

[0080] When making a specific prediction, the prediction execution module uses and As input, the information of the two encoding vectors is integrated through the structure of the multi-layer perceptron, and finally the acceleration a that the target vehicle should take at time t is output. output .

[0081] When the prediction is completed, the target vehicle code output by the vehicle-road interaction module is The queue module will be added. If the queue is full at this time, the code that entered the queue earliest will be deleted.

[0082] It should be noted that the historical decision module of this embodiment will store the prediction results made by the model at each moment. The specific stored information is vector.

[0083] By storing the encoding of the historical time steps and adding the encoding of the current moment, a decision queue is obtained, which reflects the model's behavioral decision on the target vehicle and the complete process of controlling the vehicle's driving. The purpose of using the self-attention mechanism to calculate the queue is to construct the interaction of different elements in the sequence, so that the decision result at the current moment can comprehensively consider the decision information at the historical moment, and ensure the consistency and coherence of vehicle driving.

[0084] In addition, the additional random initialization vector described above is used to store all the information of the entire queue. The calculation of the self-attention mechanism will integrate the information of the decision queue into this additional random vector, and the final output of the temporal encoding module is also this additional vector that integrates all the information.

[0085] Furthermore, as a response to the above Figure 1 and Figure 2 In order to realize the method shown in the figure, another embodiment of the present application also provides a vehicle state prediction device. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. In order to realize a vehicle state prediction method, the device is specifically as follows Figure 4 As shown, the device comprises:

[0086] The acquisition unit 31 may be used to acquire the driving state information, the vehicle-road relationship information and the historical decision information of the vehicle, wherein the vehicle-road relationship information may be used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information may be used to characterize the vehicle-road relationship information of the vehicle within a historical target period;

[0087] The execution unit 32 can be used to perform a prediction operation based on the driving state information, the vehicle-road relationship information and the historical decision information using a vehicle state prediction model to obtain a prediction result, wherein the vehicle state prediction model is trained using the historical driving state information, the historical vehicle-road relationship information and the historical decision information via an attention mechanism algorithm.

[0088] Further, such as Figure 5 As shown, the device also includes:

[0089] The setting unit 33 can be used to set the model framework of the vehicle state prediction model; the model framework includes a first input module, a second input module, a third input module and a prediction execution module; wherein the first input module can be used to input vehicle-road relationship information, the second input module can be used to input driving state information, and the third input module can be used to input historical decision information; the prediction execution module can be used to input the input results of the first input module, the second input module and the third input module and execute prediction;

[0090] The training set acquisition unit 34 may be used to acquire a target training set, wherein the training samples in the target training set include historical driving state information, historical vehicle-road relationship information, historical decision information, and corresponding actual operating state information;

[0091] The training unit 35 can be used to perform training operations on the target training set and the model framework based on the attention mechanism algorithm to generate the vehicle state prediction model.

[0092] Further, such as Figure 5 As shown, the training unit 35 comprises:

[0093] A setting subunit 351 may be used to set a loss function according to the actual result and the predicted result, and determine a distortion ratio parameter of the loss function;

[0094] The input subunit 352 can be used to input the historical vehicle-road relationship information of the training sample into the first input module of the model framework, input the historical driving state information of the training sample into the second input module of the model framework, input the historical decision information of the training sample into the third input module, and control the prediction module to perform prediction in combination with the attention mechanism algorithm to obtain a simulation prediction result;

[0095] The determination subunit 353 may be used to compare the simulation prediction result with the corresponding actual operation status information, and determine the prediction parameters of the model framework based on the comparison result and the loss function;

[0096] The generating subunit 354 may be used to generate the vehicle state prediction model based on the model framework and the prediction parameters.

[0097] Further, such as Figure 5 As shown, the device also includes:

[0098] A lane information acquisition unit 36 ​​may be used to acquire lane information, wherein the lane information at least includes a lane route and the lane width;

[0099] The driving state information acquisition unit 37 can be used to acquire the driving state information of the vehicle, and the driving state information at least includes speed, acceleration, driving time, starting position and width from the roadside;

[0100] A first determining unit 38 may be used to calculate the position coordinates of the vehicle based on the speed, acceleration, driving time and starting position, and determine the route position of the vehicle based on the position coordinates and the lane route;

[0101] A second determining unit 39 may be configured to calculate a Euclidean distance from the vehicle to a lane centerline based on the lane width and the width from the roadside, and determine a lane position of the vehicle based on the Euclidean distance;

[0102] The third determining unit 40 may be configured to determine the line position and the lane position as the vehicle-road relationship information.

[0103] Further, such as Figure 5 As shown, the lane information acquisition unit 36 ​​includes:

[0104] The collecting subunit 361 may be used to collect the road sign information of the lane and determine the lane route of the lane based on the road sign information;

[0105] A division subunit 362 may be configured to divide the lane route into a plurality of lane segments based on a preset distance to obtain a lane queue, wherein an arrangement order of the lane segments in the lane queue corresponds to a position of the lane segments in the lane route;

[0106] The determination subunit 363 may be used to mark the centerline position and the edge position of each lane segment respectively, determine the lane width of the lane segment based on the centerline position and the edge position, and obtain the lane information.

[0107] Further, such as Figure 5 As shown, the historical decision information is specifically a vehicle-road relationship queue; the vehicle-road relationship in the historical decision information is specifically a lane segment relationship, wherein each lane segment relationship is determined based on the lane segment and the driving state information.

[0108] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application further provides a storage medium, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned vehicle status prediction method.

[0109] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an embodiment of the present application also provides a vehicle status prediction device, which includes a storage medium; and one or more processors, the storage medium is coupled to the processor, and the processor is configured to execute program instructions stored in the storage medium; when the program instructions are executed, the above-mentioned vehicle status prediction method is executed.

[0110] The embodiment of the present application provides a method and device for predicting vehicle status. Compared with the prior art, in the prediction process of the vehicle status in the present application, the prediction is based on the vehicle status prediction model, which is trained based on historical driving status information, historical vehicle-road relationship information and historical decision information. That is to say, the vehicle status prediction model is actually trained based on the historical driving situation of the vehicle as a sample. Different from the parameters of the expression set manually in the prior art, the model does not need to set the parameters manually, which avoids the threshold requirement for setting experience in the manual setting process, and also avoids the problem that the set parameters are not accurate enough due to insufficient experience, thereby affecting the accuracy of the subsequent prediction results, thereby improving the accuracy of the prediction results. At the same time, in the present application, the method for predicting the vehicle status can use the three aspects of the vehicle's driving status information, the vehicle-road relationship and the historical decision information as the basis for prediction. Compared with the prior art, which only predicts the vehicle's own driving situation such as the acceleration and speed of the vehicle, the present application adds the consideration of the relationship between the vehicle and the road and the vehicle's historical decision, so that the final predicted vehicle status can be comprehensively analyzed and predicted from more aspects, so that the accuracy of the prediction results can be further improved.

[0111] The vehicle status prediction device includes a processor and a memory. The above-mentioned transmission unit, calculation verification unit, application verification unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0112] The processor includes a kernel, and the kernel calls the corresponding program unit from the memory. One or more kernels can be set, and a vehicle state prediction method is implemented by adjusting kernel parameters to solve the problem of low accuracy of prediction results when predicting the existing vehicle state.

[0113] An embodiment of the present application provides a vehicle status prediction device, which includes a storage medium; and one or more processors, wherein the storage medium is coupled to the processor, and the processor is configured to execute program instructions stored in the storage medium; when the program instructions are executed, any of the vehicle status prediction methods described above is executed.

[0114] An embodiment of the present application provides a storage medium, which includes a stored program, wherein when the program is executed, a device where the storage medium is located is controlled to execute the above-mentioned vehicle state prediction method.

[0115] The storage medium may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0116] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining driving state information, vehicle-road relationship information, and historical decision information of a vehicle, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within a historical target time period; based on the driving state information, the vehicle-road relationship information, and the historical decision information, a vehicle state prediction model is used to perform a prediction operation to obtain a prediction result, wherein the vehicle state prediction model is trained using the historical driving state information, the historical vehicle-road relationship information, and the historical decision information through an attention mechanism algorithm.

[0117] Furthermore, before performing a prediction operation using a vehicle state prediction model based on the driving state information, the vehicle-road relationship information, and the historical decision information to obtain a prediction result, the method further includes:

[0118] A model framework of the vehicle state prediction model is set; the model framework includes a first input module, a second input module, a third input module and a prediction execution module; wherein the first input module is used to input vehicle-road relationship information, the second input module is used to input driving state information, and the third input module is used to input historical decision information; the prediction execution module is used to input the input results of the first input module, the second input module and the third input module and execute prediction;

[0119] Acquire a target training set, wherein the training samples in the target training set include historical driving state information, historical vehicle-road relationship information, historical decision information, and corresponding actual operating state information;

[0120] The target training set and the model framework are trained based on the attention mechanism algorithm to generate the vehicle state prediction model.

[0121] Furthermore, the target training set and the model framework are trained based on the attention mechanism algorithm to generate the vehicle state prediction model, including:

[0122] Setting a loss function according to the actual result and the predicted result, and determining a distortion ratio parameter of the loss function;

[0123] Inputting the historical vehicle-road relationship information of the training sample into the first input module of the model framework, inputting the historical driving state information of the training sample into the second input module of the model framework, inputting the historical decision information of the training sample into the third input module, and controlling the prediction module to perform prediction in combination with the attention mechanism algorithm to obtain a simulation prediction result;

[0124] Comparing the simulation prediction result with the corresponding actual operating state information, and determining the prediction parameters of the model framework based on the comparison result and the loss function;

[0125] Based on the model framework and the prediction parameters, the vehicle state prediction model is generated.

[0126] Furthermore, the method further comprises:

[0127] Acquiring lane information, wherein the lane information at least includes a lane route and the lane width;

[0128] Acquiring driving status information of the vehicle, the driving status information including at least speed, acceleration, driving time, starting position, and width from the roadside;

[0129] Calculating the position coordinates of the vehicle based on the speed, acceleration, driving time and starting position, and determining the route position of the vehicle based on the position coordinates and the lane route;

[0130] Calculating a Euclidean distance from the vehicle to a lane centerline based on the lane width and the width from the roadside, and determining a lane position of the vehicle based on the Euclidean distance;

[0131] The line position and the lane position are determined as the vehicle-road relationship information.

[0132] Furthermore, the obtaining of lane information includes:

[0133] collecting road sign information of the lane, and determining a lane route of the lane based on the road sign information;

[0134] Based on a preset distance, the lane route is divided into a plurality of lane segments to obtain a lane queue, wherein an arrangement order of the lane segments in the lane queue corresponds to positions of the lane segments in the lane route;

[0135] The centerline position and the edge position of each lane segment are marked respectively, and the lane width of the lane segment is determined based on the centerline position and the edge position to obtain the lane information.

[0136] Furthermore, the historical decision information is specifically a vehicle-road relationship queue; the vehicle-road relationship in the historical decision information is specifically a lane segment relationship, wherein each lane segment relationship is determined based on the lane segment and the driving status information.

[0137] The present application also provides a computer program product that can execute corresponding functions, including: after metadata is generated in a generation environment, transmitting the metadata to a computing environment; obtaining the vehicle's driving state information, vehicle-road relationship information, and historical decision information, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within a historical target period; based on the driving state information, the vehicle-road relationship information, and the historical decision information, a vehicle state prediction model is used to perform a prediction operation to obtain a prediction result, wherein the vehicle state prediction model is trained using the historical driving state information, the historical vehicle-road relationship information, and the historical decision information through an attention mechanism algorithm.

[0138] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0143] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0144] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for predicting vehicle status, characterized in that: The method comprises: Acquire the driving state information, vehicle-road relationship information and historical decision information of the vehicle, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within the historical target time period; the vehicle-road relationship information at least includes the road section where the vehicle is traveling on the road and the specific position of the vehicle on the road; the historical decision information at least includes the vehicle-road relationship information before the current moment; the vehicle-road relationship information is determined by lane information; the lane information specifically includes a lane queue formed by lane segments, wherein each lane segment is a vector determined after annotated division based on road video data collected by a drone; the lane segment specifically includes an encoding vector l of the lane segment i ; Wherein, the encoding vector l of the lane segment i The specific process of determining is as follows: according to the Euclidean distance from the vehicle position at time t, the vectors of the sampling points and lane segments on the lane centerline with a relatively close distance are selected and written as {(p i ,V i ),i∈{1,2,…,N l }}, where N l The number of lane segments after screening is obtained; the lane information encoding module is used to encode the information of each lane segment; the lane information encoding module is composed of a multi-layer perceptron, the input is a 4-dimensional vector composed of sampling points and vectors, and the output is a 64-dimensional high-dimensional encoding vector. i ; Based on the driving state information, the vehicle-road relationship information and the historical decision information, a vehicle state prediction model is used to perform a prediction operation to obtain a prediction result, wherein the vehicle state prediction model is trained through an attention mechanism algorithm using historical driving state information, historical vehicle-road relationship information and historical decision information; the prediction result also specifically includes the acceleration that the vehicle should take at the target time; the vehicle state prediction model includes: a lane interaction module and a vehicle interaction module; the vehicle interaction module is provided with an attention mechanism algorithm for calculating the impact of surrounding vehicles on the target vehicle; the lane interaction module is provided with an attention mechanism algorithm for calculating the impact of lane information on the target vehicle; wherein the q vector input of the lane interaction module is the target vehicle code output by the vehicle interaction module in the vehicle state prediction model The k vector input and v vector input of the lane interaction module are the encoding vector l of the lane segment i .

2. The method according to claim 1, characterized in that Before performing a prediction operation based on the driving state information, the vehicle-road relationship information and the historical decision information using a vehicle state prediction model to obtain a prediction result, the method further includes: A model framework of the vehicle state prediction model is set; the model framework includes a first input module, a second input module, a third input module and a prediction execution module; wherein the first input module is used to input vehicle-road relationship information, the second input module is used to input driving state information, and the third input module is used to input historical decision information; the prediction execution module is used to input the input results of the first input module, the second input module and the third input module and execute prediction; Acquire a target training set, wherein the training samples in the target training set include historical driving state information, historical vehicle-road relationship information, historical decision information, and corresponding actual operating state information; The target training set and the model framework are trained based on the attention mechanism algorithm to generate the vehicle state prediction model.

3. The method according to claim 2, characterized in that The training operation of the target training set and the model framework based on the attention mechanism algorithm to generate the vehicle state prediction model includes: Setting a loss function according to the actual result and the predicted result, and determining a distortion ratio parameter of the loss function; Inputting the historical vehicle-road relationship information of the training sample into the first input module of the model framework, inputting the historical driving state information of the training sample into the second input module of the model framework, inputting the historical decision information of the training sample into the third input module, and controlling the prediction execution module to perform prediction in combination with the attention mechanism algorithm to obtain a simulation prediction result; Comparing the simulation prediction result with the corresponding actual operating state information, and determining the prediction parameters of the model framework based on the comparison result and the loss function; Based on the model framework and the prediction parameters, the vehicle state prediction model is generated.

4. The method according to claim 1 or 3, characterized in that: The method further comprises: Acquiring lane information, wherein the lane information at least includes a lane route and the lane width; Acquiring driving status information of the vehicle, the driving status information including at least speed, acceleration, driving time, starting position, and width from the roadside; Calculating the position coordinates of the vehicle based on the speed, acceleration, driving time and starting position, and determining the route position of the vehicle based on the position coordinates and the lane route; Calculating a Euclidean distance from the vehicle to a lane centerline based on the lane width and the width from the roadside, and determining a lane position of the vehicle based on the Euclidean distance; The line position and the lane position are determined as the vehicle-road relationship information.

5. The method according to claim 4, characterized in that The acquiring of lane information includes: collecting road sign information of the lane, and determining a lane route of the lane based on the road sign information; Based on a preset distance, the lane route is divided into a plurality of lane segments to obtain a lane queue, wherein an arrangement order of the lane segments in the lane queue corresponds to positions of the lane segments in the lane route; The centerline position and the edge position of each lane segment are marked respectively, and the lane width of the lane segment is determined based on the centerline position and the edge position to obtain the lane information.

6. The method according to claim 4, characterized in that The historical decision information is specifically a vehicle-road relationship queue; the vehicle-road relationship in the historical decision information is specifically a lane segment relationship, wherein each lane segment relationship is determined based on the lane segment and the driving state information.

7. A vehicle state prediction device, characterized in that: The device comprises: An acquisition unit is used to acquire driving state information, vehicle-road relationship information and historical decision information of a vehicle, wherein the vehicle-road relationship information is used to characterize the positional relationship between the vehicle and the road, and the vehicle-road relationship information is calculated based on the driving state information; the historical decision information is used to characterize the vehicle-road relationship information of the vehicle within a historical target period; the vehicle-road relationship information at least includes the road section on which the vehicle is traveling and the specific position of the vehicle on the road; the historical decision information at least includes the vehicle-road relationship information before the current moment; the vehicle-road relationship information is determined by lane information; the lane information specifically includes a lane queue formed by lane segments, wherein each lane segment is a vector determined after annotated division based on road video data collected by a drone; the lane segment specifically includes an encoding vector l of the lane segment i ; Wherein, the encoding vector l of the lane segment i The specific process of determining is as follows: according to the Euclidean distance from the vehicle position at time t, the vectors of the sampling points and lane segments on the lane centerline with a relatively close distance are selected and written as {(p i ,V i ),i∈{1,2,…,N l }}, where N l The number of lane segments after screening is obtained; the lane information encoding module is used to encode the information of each lane segment; the lane information encoding module is composed of a multi-layer perceptron, the input is a 4-dimensional vector composed of sampling points and vectors, and the output is a 64-dimensional high-dimensional encoding vector. i ; An execution unit is used to perform a prediction operation using a vehicle state prediction model based on the driving state information, the vehicle-road relationship information and the historical decision information to obtain a prediction result, wherein the vehicle state prediction model is trained through an attention mechanism algorithm using historical driving state information, historical vehicle-road relationship information and historical decision information; the prediction result also specifically includes the acceleration that the vehicle should take at the target time; the vehicle state prediction model includes: a lane interaction module and a vehicle interaction module; the vehicle interaction module is provided with an attention mechanism algorithm for calculating the impact of surrounding vehicles on the target vehicle; the lane interaction module is provided with an attention mechanism algorithm for calculating the impact of lane information on the target vehicle; wherein the q vector input of the lane interaction module is the target vehicle code output by the vehicle interaction module in the vehicle state prediction model The k vector input and v vector input of the lane interaction module are the encoding vector l of the lane segment i .

8. The device according to claim 7, characterized in that The device also includes: A setting unit is used to set a model framework of the vehicle state prediction model; the model framework includes a first input module, a second input module, a third input module and a prediction execution module; wherein the first input module is used to input vehicle-road relationship information, the second input module is used to input driving state information, and the third input module is used to input historical decision information; the prediction execution module is used to input the input results of the first input module, the second input module and the third input module and execute prediction; A training set acquisition unit, used to acquire a target training set, wherein the training samples in the target training set include historical driving state information, historical vehicle-road relationship information, historical decision information, and corresponding actual operating state information; A training unit is used to perform training operations on the target training set and the model framework based on the attention mechanism algorithm to generate the vehicle state prediction model.

9. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the vehicle state prediction method according to any one of claims 1 to 6.

10. A vehicle state prediction device, characterized in that: The device includes a storage medium; and one or more processors, the storage medium is coupled to the processor, and the processor is configured to execute program instructions stored in the storage medium; when the program instructions are executed, the vehicle state prediction method described in any one of claims 1-6 is executed.

Images