Test method and apparatus for predicting vehicle trajectory based on multimodal data of autonomous driving

Abstract

This application provides a testing method and apparatus for predicting vehicle trajectories based on multimodal data from autonomous driving. The method includes: obtaining a first state transition matrix and first state estimates of a second target based on the interaction between the vehicle under test and the first target, using a first predicted probability of the vehicle under test being in each state and a second predicted probability of a first target being in each state; obtaining second state estimates and second state transition matrices of the vehicle under test being in each state based on the first predicted probability, the matching degree between the observed data of the second target and each first state estimate, the first state transition matrix, and the first state estimates; predicting the driving trajectory of the vehicle under test based on the second state estimates and the second state transition matrix; and comparing the driving trajectory with the actual trajectory to obtain the trajectory test result of the vehicle under test. The method of this application can effectively test vehicle trajectories to determine the safety of the autonomous driving algorithm installed on the corresponding vehicle model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving testing, and more specifically, to a testing method and apparatus for predicting vehicle trajectories based on autonomous driving multimodal data. Background Technology

[0002] With the rapid development and widespread application of artificial intelligence technology, autonomous vehicles have become a core trend in the modern automotive industry. Autonomous vehicles are intelligent vehicles that achieve driverless operation by incorporating advanced autonomous driving algorithms. Conducting autonomous driving tests is crucial to ensuring the driving safety of these vehicles.

[0003] However, the black-box technology used in the trajectory planning process of autonomous vehicles prevents us from knowing the specific parameters involved. This makes it difficult to effectively test the planned vehicle trajectory. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a testing method and apparatus for predicting vehicle trajectories based on multimodal data of autonomous driving, which can effectively test vehicle trajectories to ensure the driving safety of autonomous vehicles.

[0005] In a first aspect, embodiments of this application provide a testing method for predicting vehicle trajectories based on multimodal data from autonomous driving, the testing method for predicting vehicle trajectories based on multimodal data from autonomous driving includes:

[0006] The system obtains the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment when the autonomous driving algorithm plans the trajectory for the vehicle under test in the driving scenario, and obtains the actual trajectory of the vehicle under test at the current moment.

[0007] Based on all first predicted probabilities and all second predicted probabilities, the test vehicle and the first target are interacted to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state.

[0008] Based on all the first predicted probabilities, the matching degree between the observation data of the second target at the current time and each of the first state estimates, the first state transition matrix, and all the first state estimates, the second state estimates and the second state transition matrix of the vehicle under test in each state are obtained.

[0009] All second-state estimates and the second-state transition matrix are input into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test;

[0010] By comparing the driving trajectory with the actual trajectory, the trajectory test results of the vehicle under test are obtained.

[0011] In one possible implementation, the step of interacting the vehicle under test and the first target based on all first predicted probabilities and all second predicted probabilities to obtain a first state transition matrix of the second target after the interaction and first state estimates in each state includes:

[0012] For each state, the average probability of the vehicle under test being in that state is calculated based on the first predicted probability of the vehicle under test being in each state and the theoretical probability of the vehicle under test transitioning from each state to that state.

[0013] Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, and the theoretical probability of the vehicle under test transitioning from each state to each state, the initial first state estimate of the second target being in the state after interaction and the probability of transitioning from the state to each state in the initial first state transition matrix are calculated.

[0014] Based on the theoretical state transition matrix of the second target and the preset process noise covariance matrix corresponding to the weather type at the current moment, all initial first state estimates and the initial first state transition matrices are corrected to obtain the target first state estimates and target first state transition matrices of the second target in all states.

[0015] In one possible implementation, based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, and the theoretical probability of the vehicle under test transitioning from each state to each state, an initial first state estimate of the second target being in the state after interaction and the probabilities of transitioning from the state to each state in the initial first state transition matrix are calculated, including:

[0016] Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, the initial first state estimate of the second target being in the state after the interaction is calculated.

[0017] Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, the theoretical probability of the vehicle under test being in each state, and the initial first state estimate of the second target being in the state, the probability of the second target transitioning from the state to each state in the initial first state transition matrix after interaction is calculated.

[0018] In one possible implementation, the step of calculating the initial first state estimate of the second target in the state after interaction, based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, includes:

[0019] Substituting the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment into the following formula, we obtain the initial first state estimate of the second target being in the state after the interaction.

[0020]

[0021] in, For the initial first state estimate of the second objective i in state m. Let μ be the average probability that the vehicle a is in state m, M be the number of states, and μ be the average probability that the vehicle a is in state m. (b) n p represents the second predicted probability that the first objective is in state n. (b) nm Let m be the theoretical probability that the first objective is in a state transitioning from state n to state m. This is the theoretical state estimate of the vehicle under test, a, in state n at the current moment.

[0022] In one possible implementation, obtaining the second state estimate and second state transition matrix of the vehicle under test in each state based on all first predicted probabilities, the matching degree between the observation data of the second target at the current time and each of the first state estimates, the first state transition matrix, and all first state estimates includes:

[0023] The first predicted probabilities are updated based on the matching degree between the observation data of the second target at the current time and all first state estimates, to obtain the third predicted probabilities of the vehicle under test being in each state.

[0024] Based on all third predicted probabilities and all first state estimates, predict the second state estimates of the test vehicle in each state;

[0025] Based on all the third predicted probabilities, the first state transition matrix, all the first state estimates, and all the second state estimates, predict the second transition probabilities of the vehicle under test transitioning from each state to each state at the current time.

[0026] In one possible implementation, a third prediction probability of the vehicle under test being in any state is predicted by the following steps:

[0027] Substituting the matching degree between the observation data of the second target at the current moment and the first state estimate of the second target in the stated state, and the first predicted probability of the vehicle under test being in the stated state, into the following formula, a third predicted probability of the vehicle under test being in the stated state is obtained, including:

[0028]

[0029] Where, μ′ m Let be the third predicted probability of the vehicle under test being in state m. γ is the average probability that the vehicle under test is in state m. m The degree of matching between the observed data of the second target at the current moment and the first state estimate of the second target in state m, where M is the number of states. γ is the average probability that the vehicle under test is in state n. n The degree of matching between the observation data of the second target at the current moment and the first state estimate of the second target in state n.

[0030] In one possible implementation, the comparison of the driving trajectory and the actual trajectory to obtain the trajectory test result of the vehicle under test includes:

[0031] If the driving trajectory matches the actual trajectory, then the trajectory test result is a pass.

[0032] If the driving trajectory is inconsistent with the actual trajectory, the trajectory test result is a test failure.

[0033] Secondly, embodiments of this application also provide a testing device for predicting vehicle trajectories based on multimodal data from autonomous driving, the device comprising:

[0034] The acquisition module is used to acquire the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment, which are predicted by the autonomous driving algorithm when planning the trajectory for the vehicle under test in the driving scenario, and to acquire the actual trajectory of the vehicle under test at the current moment.

[0035] An interaction module is used to interact the vehicle under test and the first target based on all first predicted probabilities and all second predicted probabilities to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state.

[0036] The update prediction module is used to obtain the second state estimate and the second state transition matrix of the vehicle under test in each state based on all first prediction probabilities, the matching degree between the observation data of the second target at the current time and each first state estimate, the first state transition matrix and all first state estimates.

[0037] The input module is used to input all the second state estimates and the second state transition matrix into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test;

[0038] The comparison module is used to compare the driving trajectory with the actual trajectory to obtain the trajectory test results of the vehicle under test.

[0039] In one possible implementation, the interaction module is specifically configured to, for each state, calculate the average probability of the vehicle under test being in that state based on a first predicted probability of the vehicle under test being in each state and a theoretical probability of the vehicle under test transitioning from each state to that state; calculate, based on the average probability of the vehicle under test being in that state, a second predicted probability of the first target being in each state, a theoretical probability of the first target transitioning from each state to that state, a theoretical state estimate of the vehicle under test being in each state at the current time, and a theoretical probability of the vehicle under test transitioning from each state to each state, calculate the initial first state estimate of the second target being in that state after interaction and the probability of transitioning from that state to each state in the initial first state transition matrix; and correct all initial first state estimates and the initial first state transition matrix based on the theoretical state transition matrix of the second target and a preset process noise covariance matrix corresponding to the weather type at the current time, to obtain the target first state estimate and target first state transition matrix of the second target being in all states.

[0040] In one possible implementation, the interaction module is further configured to:

[0041] Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, the initial first state estimate of the second target being in the state after the interaction is calculated.

[0042] Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, the theoretical probability of the vehicle under test being in each state, and the initial first state estimate of the second target being in the state, the probability of the second target transitioning from the state to each state in the initial first state transition matrix after interaction is calculated.

[0043] In one possible implementation, the interaction module is further configured to:

[0044] Substituting the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment into the following formula, we obtain the initial first state estimate of the second target being in the state after the interaction.

[0045]

[0046] in, For the initial first state estimate of the second objective i in state m. Let μ be the average probability that the vehicle a is in state m, M be the number of states, and μ be the average probability that the vehicle a is in state m. (b) n p represents the second predicted probability that the first objective is in state n. (b) nm Let m be the theoretical probability that the first objective is in a state transitioning from state n to state m. This is the theoretical state estimate of the vehicle under test, a, in state n at the current moment.

[0047] In one possible implementation, the update prediction module is specifically configured to update each first prediction probability based on the matching degree between the observation data of the second target at the current time and all first state estimates, to obtain a third prediction probability of the vehicle under test being in each state; predict a second state estimate of the vehicle under test being in each state based on all third prediction probabilities and all first state estimates; and predict a second transition probability of the vehicle under test transitioning from each state to each state at the current time based on all third prediction probabilities, the first state transition matrix, all first state estimates, and all second state estimates.

[0048] In one possible implementation, the update prediction module is further configured to predict a third prediction probability that the vehicle under test is in any state by means of the following steps:

[0049] Substituting the matching degree between the observation data of the second target at the current moment and the first state estimate of the second target in the stated state, and the first predicted probability of the vehicle under test being in the stated state, into the following formula, a third predicted probability of the vehicle under test being in the stated state is obtained, including:

[0050]

[0051] Where, μ′ m Let be the third predicted probability of the vehicle under test being in state m. γ is the average probability that the vehicle under test is in state m. m The degree of matching between the observed data of the second target at the current moment and the first state estimate of the second target in state m, where M is the number of states. γ is the average probability that the vehicle under test is in state n. n The degree of matching between the observation data of the second target at the current moment and the first state estimate of the second target in state n.

[0052] In one possible implementation, the comparison module is specifically configured to determine the trajectory test result as "test passed" if the driving trajectory matches the actual trajectory, and "test failed" if the driving trajectory does not match the actual trajectory.

[0053] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the test method for predicting vehicle trajectories based on autonomous driving multimodal data as described in any of the first aspects.

[0054] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the test method for predicting vehicle trajectories based on multimodal data of autonomous driving as described in any of the first aspects.

[0055] This application provides a testing method and apparatus for predicting vehicle trajectories based on multimodal data from autonomous driving. The method includes: acquiring the actual trajectory of the vehicle under test at the current moment; interacting with the vehicle under test and the first target based on a first predicted probability of the vehicle under test being in each state at the current moment and a second predicted probability of a first target being in each state at the current moment, to obtain a first state transition matrix of the second target after interaction and estimates of its first state in each state; obtaining second state estimates and second state transition matrices of the vehicle under test in each state based on all first predicted probabilities, the matching degree between the observed data of the second target at the current moment and each first state estimate, the first state transition matrix, and all first state estimates; inputting all second state estimates and second state transition matrices into a vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test; and comparing the driving trajectory with the actual trajectory to obtain the trajectory test result of the vehicle under test. The method of this application can effectively test vehicle trajectories to ensure the driving safety of autonomous vehicles. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart of a test method for predicting vehicle trajectories based on multimodal data of autonomous driving, provided in an embodiment of this application, is shown.

[0058] Figure 2 This application provides an embodiment of the interaction flowchart between the vehicle under test and the first target.

[0059] Figure 3 This paper shows a schematic diagram of the structure of a test device for predicting vehicle trajectories based on multimodal data of autonomous driving, provided in an embodiment of this application.

[0060] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0062] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0063] To enable those skilled in the art to utilize the content of this application, and in conjunction with the specific application scenario of "autonomous driving testing," the following implementation methods are provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is primarily described in the context of "autonomous driving testing," it should be understood that this is merely an exemplary embodiment.

[0064] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0065] The following is a detailed description of a test method for predicting vehicle trajectories based on multimodal data of autonomous driving, provided in an embodiment of this application.

[0066] Reference Figure 1 The diagram shown is a flowchart illustrating a test method for predicting vehicle trajectories based on multimodal data of autonomous driving, provided in an embodiment of this application. The exemplary steps of this embodiment are described below:

[0067] S101. Obtain the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment, as predicted by the autonomous driving algorithm when planning the trajectory for the vehicle under test in the driving scenario, and obtain the actual trajectory of the vehicle under test at the current moment.

[0068] In this application embodiment, the autonomous driving algorithm refers to a system installed on a vehicle under test to achieve unmanned driving functionality. The targets in the driving scenario of the vehicle under test include a first target and the vehicle under test. The first target refers to any target in the driving scenario other than the vehicle under test. The state can be a preset state such as a stationary state or an accelerating state. The vehicle under test is equipped with numerous sensors (such as lidar, cameras, and ultrasonic sensors) to acquire multimodal data (such as point clouds, images, and ultrasonic signals) corresponding to the driving scenario. Specifically, the multimodal data detected in real-time by the sensors is preprocessed; then, the autonomous driving algorithm plans a trajectory for the vehicle under test based on the preprocessed multimodal data; the algorithm obtains the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment, and acquires the actual trajectory of the vehicle under test at the current moment.

[0069] Specifically, preprocessing is performed on the multimodal data detected in real time by the sensor, including:

[0070] (1) Use Gaussian filtering to remove noise points from each modal data: Perform sensitivity analysis on each modal data to adjust filtering parameters (such as standard deviation σ), evaluate the impact of noise reduction on subsequent trajectory planning, and select the optimal filtering parameters for the trajectory planning task. The specific formula is:

[0071]

[0072] in, Midpoint of space The Gaussian function value, σ is the standard deviation (controlling the width of the Gaussian function, thus affecting the smoothness of the filter).

[0073] Here, the impact of denoising on subsequent trajectory planning is evaluated by adjusting the value of σ. For example, by setting different σ values ​​and applying Gaussian filtering, the performance changes of the filtered data on the trajectory planning task are analyzed.

[0074] (2) Using spatiotemporal alignment techniques: Alignment techniques generally refer to temporal alignment and spatial alignment. Spatial alignment refers to aligning data from different sources, such as matching pixel positions in an image with spatial coordinates in a point cloud, and integrating the data into a unified spatial coordinate system through methods such as translation, rotation, and scaling. Temporal alignment here generally refers to software-based alignment, that is, aligning the timestamps of multiple sensor data. The specific formula is as follows:

[0075] S′=S·R·S+T;

[0076] Where S represents the spatially aligned modal data, R is the rotation matrix, s is the scaling matrix, and T is the translation vector.

[0077] t′=t+Δt;

[0078] Where t′ is the timestamp after time alignment, and Δt is the time offset (used to correct time differences between different sensors).

[0079] (3) Dynamic weight allocation: Compared with traditional static weight allocation, dynamic weight allocation can better adapt to complex scenarios and dynamically changing needs, thus improving the performance of trajectory planning. For example, in autonomous driving perception targets, the weights of point cloud error and image classification error are dynamically adjusted according to changes in these two factors, thereby improving trajectory planning accuracy. The specific formula is as follows:

[0080]

[0081] Among them, w p For the weights of point cloud data, w I The weights of the image data, ∈ I For image data error, ∈ p This refers to the error in the point cloud data.

[0082] Here, if the point cloud error increases, the weight of the point cloud data is reduced and the weight of the image data is increased, and vice versa. This dynamic adjustment can better adapt to complex scenarios and dynamically changing needs, improving the performance of trajectory planning.

[0083] Specifically, the autonomous driving algorithm predicts the probability of each target (the vehicle under test and the first target) being in each state at the current moment based on the probability of each target being in each state at the previous moment (i.e., the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment). The calculation formula can be expressed as:

[0084]

[0085] in, Let be the first predicted probability that the target is in state m at the current time t. Let be the probability that the target was in state m at the previous time t-1. Let m be the theoretical probability of the target transitioning from state m to state m. Let be the probability that the target was in state n at the previous time t-1. The theoretical probability of the target transitioning from state n to state m.

[0086] Furthermore, this application embodiment constructs a multi-target interaction model, which analyzes the relative positions and interaction patterns between targets. The target states output by the model can provide more comprehensive and robust perception information, providing high-quality input for subsequent trajectory prediction. The specific formula is as follows:

[0087] State t =f(Data t );

[0088] Where, State t Data represents the overall state of all targets at time t. t The model input data is obtained based on the multimodal data at time t, such as targets a and b. The model input data at time t includes the following data: (1) Position a (1) The position of the vehicle to be tested, a, in the alignment space at time t, such as the xyz coordinates; (2) Interaction a (3) Velocity a Let be the velocity of the vehicle a under test at time t (which can be represented as a vector, including the magnitude and direction of the velocity); (4) Acceleration a Let be the acceleration of the vehicle a under test at time t (which can be represented as a vector); (5) Orientation a (6) Size a (7) Visibility a Let t represent the visibility of the vehicle being tested, a, at time t (i.e., whether the vehicle being tested, a, is occluded; 0 indicates occlusion, greater than or equal to 85%, and 1 indicates no occlusion, less than 15%).

[0089] Here, the specific calculation process obtained after obtaining the perceived information based on the above model is as follows: S102 to S103.

[0090] S102. Based on all first predicted probabilities and all second predicted probabilities, the vehicle to be tested and the first target interact to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state.

[0091] In this embodiment, target interaction is performed based on the state of the vehicle under test and the state of the first target. This yields a first state transition matrix and first state estimates for each state of the second target as a whole after the interaction (e.g., the whole after the collision between the vehicle under test and the first target is considered as one target). The first state transition matrix includes the probability of the second target transitioning from each state to that state, and the first state estimate for each state refers to the state parameter matrix of the second target in that state. (Refer to...) Figure 2 The diagram shown is an interaction flowchart between the vehicle under test and the first target provided in an embodiment of this application:

[0092] S201. For each state, calculate the average probability of the vehicle being in that state based on the first predicted probability of the vehicle being in each state and the theoretical probability of the vehicle being in that state from each state.

[0093]

[0094] in, Let M be the average probability that the vehicle a is in state m, and M be the number of states. Let be the first predicted probability that the vehicle to be tested, a, is in state n. Let be the theoretical probability that the vehicle to be tested, a, will transition from state n to state m.

[0095] For example, assume the states include a stationary state, an accelerating state, and a constant speed state. For the stationary state, calculate the average probability that the vehicle is stationary based on the first predicted probabilities of the vehicle being in a stationary, accelerating, and constant speed state, respectively, and the theoretical probabilities of the vehicle transitioning from a stationary, accelerating, and constant speed state to a stationary state, respectively. For the accelerating state, calculate the average probability that the vehicle is accelerating based on the first predicted probabilities of the vehicle being in a stationary, accelerating, and constant speed state, respectively, and the theoretical probabilities of the vehicle transitioning from a stationary, accelerating, and constant speed state to an accelerating state, respectively. For the constant speed state, calculate the average probability that the vehicle is in a constant speed state based on the first predicted probabilities of the vehicle being in a stationary, accelerating, and constant speed state, respectively, and the theoretical probabilities of the vehicle transitioning from a stationary, accelerating, and constant speed state to a constant speed state, respectively.

[0096] S202. Based on the average probability of the vehicle under test being in this state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to this state, the theoretical state estimate of the vehicle under test being in each state at the current moment, and the theoretical probability of the vehicle under test transitioning from each state to each state, calculate the initial first state estimate of the second target being in this state after the interaction, and the probability of transitioning from this state to each state in the initial first state transition matrix.

[0097] In the embodiments of this application, for each state, the initial first state estimate of the second target after the interaction in that state and the probability of transitioning from that state to each state in the initial first state transition matrix are calculated.

[0098] Taking a stationary state as an example, based on the average probability of the test vehicle being stationary, the second predicted probabilities of the first target being stationary, accelerating, and moving at a constant speed, respectively, the theoretical probabilities of the first target transitioning from stationary, accelerating, and moving at a constant speed to stationary, respectively, the theoretical state estimates of the test vehicle at the current time from stationary, accelerating, and moving at a constant speed, respectively, the theoretical probabilities of the test vehicle transitioning from stationary to stationary, accelerating, and moving at a constant speed, respectively, the theoretical probabilities of the test vehicle transitioning from accelerating to stationary, accelerating, and moving at a constant speed, respectively, the theoretical probabilities of the test vehicle transitioning from moving at a constant speed to stationary, accelerating, and moving at a constant speed, respectively, the initial first state estimate of the second target being stationary after interaction, and the probabilities of transitioning from stationary to stationary, accelerating, and moving at a constant speed in the initial first state transition matrix, are calculated.

[0099] Specifically, the implementation process of S202 is as follows:

[0100] Step 1: Based on the average probability of the vehicle under test being in this state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to this state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, calculate the initial first state estimate of the second target being in this state after the interaction.

[0101] In this application embodiment, the average probability of the vehicle under test being in this state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to this state, and the theoretical state estimate of the vehicle under test being in each state at the current moment are substituted into the following formula to obtain the initial first state estimate of the second target being in this state after interaction.

[0102]

[0103] in, For the initial first state estimate of the second objective i in state m. Let μ be the average probability that the vehicle a is in state m, M be the number of states, and μ be the average probability that the vehicle a is in state m. (b) n p represents the second predicted probability that the first objective is in state n. (b) nm Let m be the theoretical probability that the first objective is in a state transitioning from state n to state m. This is the theoretical state estimate of the vehicle under test, a, in state n at the current moment.

[0104] Taking a stationary state as an example, based on the average probability that the vehicle under test is stationary, the second predicted probabilities that the first target is stationary, accelerating, and moving at a constant speed, respectively, the theoretical probabilities that the first target transitions from a stationary, accelerating, and moving at a constant speed to a stationary state, respectively, the theoretical state estimates that the vehicle under test is stationary, accelerating, and moving at a constant speed at the current moment, respectively, and the initial first state estimate that the second target is stationary after the interaction is calculated.

[0105] Step 2: Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, the theoretical probability of the vehicle under test being in each state, and the initial first state estimate of the second target being in the state, calculate the probability of the second target transitioning from the state to each state in the initial first state transition matrix after interaction.

[0106] In this embodiment of the application, the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, the theoretical probability of the vehicle under test being in each state, and the initial first state estimate of the second target being in the state are substituted into the following formula to obtain the probability of transitioning from the state to each state in the initial first state transition matrix of the second target after interaction.

[0107]

[0108] in, Let i be a matrix containing the probabilities of transitioning from state m to each state in the initial first state transition matrix. Let μ be the average probability that the vehicle a is in state m, M be the number of states, and μ be the average probability that the vehicle a is in state m. (b) n p represents the second predicted probability that the first objective is in state n. (b) nm Let P be the theoretical probability of the first objective transitioning from state n to state m.(a) n Let be the theoretical probability that the vehicle to be tested, a, is in state n at the current moment. This is a theoretical state estimate for the vehicle under test, a, in state n at the current moment. The initial first state estimate for the second objective i in state m.

[0109] Taking a stationary state as an example, based on the average probability of the vehicle under test being stationary, the second predicted probabilities of the first target being stationary, accelerating, and moving at a constant speed, the theoretical probabilities of the first target transitioning from a stationary, accelerating, and moving at a constant speed to a stationary state, the theoretical state estimates of the vehicle under test being stationary, accelerating, and moving at a constant speed at the current moment, the theoretical probabilities of the vehicle under test being stationary, accelerating, and moving at a constant speed, the initial first state estimate of the second target being stationary, and the probability of transitioning from a stationary state to each state in the initial first state transition matrix of the second target after interaction are calculated.

[0110] It should be noted that theoretical probability refers to the likelihood of a target object being in different states (such as position, velocity, acceleration, etc.) under ideal conditions, calculated through mathematical models and algorithms based on the physical laws governing target motion, traffic rules, and environmental factors. Theoretical state estimation refers to predicting and inferring the future state of a target using sensor data (such as radar, cameras, etc.) and known physical models, combined with the target's control strategy and environmental information.

[0111] S203. Based on the theoretical state transition matrix of the second target and the preset process noise covariance matrix corresponding to the weather type at the current time, all initial first state estimates and initial first state transition matrices are corrected to obtain the target first state estimates and target first state transition matrices of the second target in all states.

[0112] In this embodiment, since weather conditions may affect state changes, it is necessary to correct the initial first state estimate and the initial first state transition matrix based on a preset process noise covariance matrix corresponding to the weather type. That is, the preset process noise covariance matrix is ​​used to measure the impact of weather conditions on the target's state changes. The specific correction process is as follows:

[0113] Step 1: For each state, based on the theoretical state transition matrix of the second target, correct the initial first state estimate of the second target in that state to obtain the target first state estimate of the second target in that state.

[0114] In this embodiment of the application, the theoretical state transition matrix of the second target and the initial first state estimate of the second target in that state are substituted into the following formula to obtain the target first state estimate of the second target in that state.

[0115]

[0116] in, For the first state estimation of the second objective in state m, F m The theoretical state transition matrix for the second objective. The initial first state estimate for the second objective in state m.

[0117] Step 2: For each state, based on the probability of transitioning from that state to each state in the theoretical state transition matrix of the second target and the preset process noise covariance matrix corresponding to the weather type at the current time, the probability of transitioning from that state to each state in the initial first state transition matrix is ​​corrected to obtain the probability of transitioning from that state to each state in the intermediate first state transition matrix of the second target.

[0118] Substituting the probability of transitioning from the current state to each state in the theoretical state transition matrix of the second target, the preset process noise covariance matrix corresponding to the weather type at the current time, and the probability of transitioning from the current state to each state in the initial first state transition matrix into the following formula, we obtain the probability of transitioning from the current state to each state in the intermediate first state transition matrix of the second target.

[0119]

[0120] in, For the second objective, F is a matrix containing the probabilities of transitioning from state m to each state in the intermediate first state transition matrix. m The theoretical state transition matrix for the second objective. Q is a matrix containing the probabilities of transitioning from state m to each state in the initial first state transition matrix. m T is the preset process noise covariance matrix corresponding to the weather type at the current moment, and T is the transpose.

[0121] Step 3: For each state, based on the observation data of the second target at the current time, the observation matrix of the second target at the current time, and the observation noise covariance matrix corresponding to the weather type at the current time, the intermediate first state estimate of the second target in that state and the probability of transitioning from that state to each state in the intermediate first state transition matrix are corrected to obtain the target first state estimate of the second target in that state and the probability of transitioning from that state to each state in the target first state transition matrix.

[0122] In this embodiment, since weather conditions may cause errors in the sensor observation data, it is necessary to correct the intermediate first state estimate and the intermediate first state transition matrix based on the observation noise covariance matrix corresponding to the weather type. That is, the observation noise covariance matrix is ​​used to measure the error caused by weather conditions in the observation data. Specifically, the correction process is as follows: Substitute the observation data of the second target at the current time, the observation matrix of the second target at the current time, the observation noise covariance matrix corresponding to the weather type at the current time, the intermediate first state estimate of the second target in that state, and the probability of transitioning from that state to various states in the intermediate first state transition matrix into the following formula to obtain the target first state estimate of the second target in that state and the probability of transitioning from that state to various states in the target first state transition matrix.

[0123]

[0124] Among them, K m The gain corresponding to state m as the second objective (used to balance the weights between the predicted intermediate state estimate and the observed state estimate), H is a matrix containing the probabilities of transitioning from state m to each state in the intermediate first state transition matrix, representing the second objective. m Let T be the observation matrix of the second target at the current time, T be the transpose, and R be the observation noise covariance matrix corresponding to the weather type at the current time. Estimate the first state of the second objective in state m. The first state estimate is given for the second objective in state m. It is a matrix containing the probabilities of transitioning from state m to each state in the first state transition matrix of the target.

[0125] It should be noted that K m This determines the weight used to correct the intermediate state estimate based on the observed data z, balancing the information content of the prediction and the observation. If K m If the value is large, the observed data has high reliability, and the model will use the current observed value. If the value is small, the reliability is low, and the model will rely on the predicted value.

[0126] H m This is the observation matrix of the second target, where H m It is a nonlinear function used to represent the distance and angle, r and θ, that map observation data to the observation matrix. z is the observation data obtained from sensors (such as radar), i.e., the actual data measured at the current moment; z can directly correspond to the observation matrix H. m The output dimension.

[0127] Here, in this embodiment of the application, the state estimate and state transition matrix of the second target can be obtained by interacting the vehicle under test with the first target and correcting it using a preset process noise covariance matrix and observation noise covariance matrix.

[0128] S103. Based on all the first predicted probabilities, the matching degree between the observation data of the second target at the current time and each first state estimate, the first state transition matrix, and all the first state estimates, the second state estimate and the second state transition matrix of the vehicle under test in each state are obtained.

[0129] In the embodiments of this application, it is assumed that the state change of the first target is completely correct. If there is an error between the observed data of the second target and the predicted first state estimate, it indicates that the first probability of the test vehicle at the current moment determined by the autonomous driving algorithm is wrong. Therefore, the second state estimate and the second state transition matrix of the test vehicle in each state are re-determined using the first state transition matrix of the second target and all first state estimates, so as to re-plan the vehicle trajectory for the test vehicle.

[0130] The specific process is as follows:

[0131] Step 1: Update each first prediction probability based on the matching degree between the observation data of the second target at the current time and all first state estimates, and obtain the third prediction probability of the vehicle under test in each state.

[0132] In this application embodiment, a third prediction probability of the vehicle under test being in any state is predicted through the following steps:

[0133] Substituting the matching degree between the observation data of the second target at the current moment and the first state estimate of the second target in that state, and the first predicted probability of the vehicle being tested in that state, into the following formula, we obtain the third predicted probability of the vehicle being tested in that state, including:

[0134]

[0135] Where, μ′ m Let be the third predicted probability of the vehicle under test being in state m. γ is the average probability that the vehicle under test is in state m. m Let M be the degree of matching between the observed data of the second target at the current moment and the first state estimate of the second target in state m (used to measure the consistency between the observed data and the first state estimate), where M is the number of states. γ is the average probability that the vehicle under test is in state n. n The degree of matching between the observation data of the second target at the current moment and the first state estimate of the second target in state n.

[0136] Furthermore, the degree of matching between the observation data of the second target at the current moment and the first state estimate of the second target in any state is calculated using the following formula:

[0137]

[0138] Where n′ represents the dimension of the observed data.

[0139] Step 2: Based on all the third predicted probabilities and all the first state estimates, predict the second state estimates of the vehicle under test in each state.

[0140] In the embodiments of this application, for each state, the third predicted probability of the vehicle under test being in that state and the first state estimate are substituted into the following formula to obtain the second state estimate of the vehicle under test being in that state.

[0141]

[0142] in, This is the second state estimate for the vehicle under test in state m.

[0143] Step 3: Based on all the third predicted probabilities, the first state transition matrix, all first state estimates, and all second state estimates, predict the second transition probabilities of the vehicle under test transitioning from each state to each state at the current time.

[0144] In the embodiments of this application, for each state, the third predicted probability of the vehicle under test being in that state, the probability of the vehicle under test transitioning from that state to various states in the first state transition matrix, the first state estimate of the second target being in that state, and the second state estimate of the vehicle under test being in that state are substituted into the following formula to obtain the second transition probability of the vehicle under test transitioning from that state to various states at the current time.

[0145]

[0146] in, Let μ′ be a matrix containing the second transition probabilities of the vehicle under test transitioning from state m to each state at the current time. m Let be the third predicted probability of the vehicle under test being in state m. Let be a matrix containing the probabilities of transitioning from state m to each state in the first state transition matrix of the target. Estimate the first state of the second objective in state m. This is the second state estimate for the vehicle under test in state m.

[0147] Furthermore, if there are multiple first targets, the vehicle under test will interact with each first target separately, which will result in multiple second state estimates of the vehicle under test in each state, as well as multiple second state transition matrices. It is then necessary to perform weighted fusion or average fusion of these multiple second state estimates or multiple second state transition matrices for each state.

[0148] S104. Input all second-state estimates and second-state transition matrices into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test.

[0149] In this embodiment, the core structure of the vehicle trajectory prediction model includes two parts: an encoder and a decoder. The encoder maps the input data and conditional inputs to the latent space, generating the distribution parameters of the latent variables. The decoder generates the driving trajectory based on the latent variables output by the encoder and the conditional inputs. The conditional inputs are further optimized through the preceding multi-objective interaction model, making the target trajectory generated by the decoder more realistic.

[0150] Specifically, the encoder's formula is as follows:

[0151] [u t ,σ t ] = Encoder(X t C t M t ,S t ,L t );

[0152] Among them, u t σ is the mean of the latent variable distribution of the target at time t; the target refers to a vehicle or pedestrian in the trajectory prediction task (this target is a traffic participant whose current trajectory needs to be traveled); latent variables are observations that cannot be directly obtained but can be inferred from observation data, such as the acceleration and velocity trends of target i; t X is the standard deviation of the latent variable distribution of the target at time t; t The historical trajectory data of the target at time t; C t At time t, the input traffic rules and road information are typically from GPS; M t At time t, the input map information, such as roads and bridges, is given; S t At time t, which is the State mentioned above, it is obtained after preprocessing, fusing, and feature extraction of sensor data. For example, the raw distance and angle information returned by the sensor, including radar, can be converted into the relative position and relative velocity of the target object after preprocessing, thus forming part of State t. t It refers to the prior knowledge input at time t, such as existing vehicle dynamics models.

[0153] Specifically, the decoder's formula is as follows:

[0154]

[0155] in, Z is the trajectory of the vehicle to be tested predicted at time t; t C′ is the latent variable sampled from the latent variable distribution of the target at time t. t At time t, the conditional input is based on the second state estimate and the optimized second state transition matrix; M t At time t, the input map information, such as roads and bridges, is given; S t It is at time t, which is the State mentioned above; L t Q is the prior knowledge input at time t, such as an existing vehicle dynamics model; t It refers to dynamic environmental factors at time t, such as the construction area.

[0156] The specific optimization formula based on the second state estimation and the second state transition matrix optimization conditions is as follows:

[0157] C′ t =Optimize(C t State t ,I t ,R t );

[0158] Among them, C′ t The optimized input conditions are derived from GPS; the original traffic rules are based on GPS, and the optimized input conditions will combine this information and be adjusted according to other factors, such as not being allowed to occupy the non-motorized vehicle lane, but violating the rules to avoid a target, etc.

[0159] The Optimize function can be represented by the function f in the following form:

[0160]

[0161] Among them, R t For risk assessment at time (t), safety distance, collision probability, etc. are considered. "arg min" is an abbreviation for "argument of the minimum," meaning "the independent variable that minimizes the objective function." In "arg{}", it represents finding the value among all possible values ​​that minimizes this expression. (C′) t ) is a structured constraint term. Find C′ when the formula reaches its minimum value.

[0162] In this embodiment, λ is the regularization coefficient, a hyperparameter used to control the strength of the structured constraint term. If λ is large, the optimization will be more inclined to satisfy the structured constraint term; if λ is small, the optimization will focus more on minimizing the loss function L′.

[0163] Specifically, the expression for the L′ loss function is:

[0164]

[0165] Specifically, (C′ t These are structured constraints, which can be adjusted later as needed.

[0166] In this application embodiment, if it is desired to be sparse, then (C′) is set. t ) = ||C′||1. If you want it to be in matrix form, then set (C′) = ||C′||1. t )=‖C′‖ * , where * is greater than 1.

[0167] In the embodiments of this application, the distribution of the latent variables of the model depends on (St), (Lt), and (Rt). The mean and variance of the latent variables are determined by these parameters. The mean of the latent variables is generated by the encoder function. Therefore, the distribution parameters of the latent variables, namely the mean and variance, are dynamically generated by the encoder based on the input data.

[0168] S105. Compare the driving trajectory with the actual trajectory to obtain the trajectory test results of the vehicle under test.

[0169] In this embodiment of the application, if the driving trajectory is consistent with the actual trajectory, the trajectory test result is a successful test; if the driving trajectory is inconsistent with the actual trajectory, the trajectory test result is a failed test.

[0170] This application provides a testing method for predicting vehicle trajectories based on multimodal data from autonomous driving. The method includes: obtaining a first state transition matrix and first state estimates of the second target after interaction between the vehicle under test and the first target, based on a first predicted probability of the vehicle under test being in each state at the current time and a second predicted probability of the first target being in each state at the current time; obtaining second state estimates and second state transition matrices of the vehicle under test in each state based on the first predicted probability, the matching degree between the observed data of the second target at the current time and each first state estimate, the first state transition matrix, and the first state estimates; predicting the driving trajectory of the vehicle under test based on the second state estimates and the second state transition matrix; and comparing the driving trajectory with the actual trajectory to obtain the trajectory test result of the vehicle under test. This method enables effective testing of vehicle trajectories.

[0171] Based on the same inventive concept, this application also provides a test device for predicting vehicle trajectories based on autonomous driving multimodal data, which corresponds to the test method for predicting vehicle trajectories based on autonomous driving multimodal data. Since the principle of the device in this application is similar to the test method for predicting vehicle trajectories based on autonomous driving multimodal data described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0172] Reference Figure 3 The diagram shown is a schematic of a test device for predicting vehicle trajectories based on multimodal data of autonomous driving, provided in an embodiment of this application. The test device for predicting vehicle trajectories based on multimodal data of autonomous driving includes:

[0173] The acquisition module 301 is used to acquire the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment, which are predicted by the autonomous driving algorithm when planning the trajectory for the vehicle under test in the driving scenario, and to acquire the actual trajectory of the vehicle under test at the current moment.

[0174] Interaction module 302 is used to interact with the vehicle under test and the first target based on all first prediction probabilities and all second prediction probabilities, and obtain the first state transition matrix of the second target after interaction and the first state estimate of each state.

[0175] The update prediction module 303 is used to obtain the second state estimate and the second state transition matrix of the vehicle under test in each state based on all first prediction probabilities, the matching degree between the observation data of the second target at the current time and each first state estimate, the first state transition matrix and all first state estimates.

[0176] The input module 304 is used to input all the second state estimates and the second state transition matrix into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test.

[0177] The comparison module 305 is used to compare the driving trajectory with the actual trajectory to obtain the trajectory test result of the vehicle under test.

[0178] This device enables effective testing of vehicle trajectories.

[0179] like Figure 4As shown in the embodiment of this application, an electronic device 400 includes a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device is running, the processor 401 communicates with the memory 402 via the bus. The processor 401 executes the machine-readable instructions to perform the steps of the test method for predicting vehicle trajectories based on autonomous driving multimodal data as described above.

[0180] Specifically, the memory 402 and processor 401 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 401 runs the computer program stored in the memory 402, it can execute the above-mentioned test method for predicting vehicle trajectory based on autonomous driving multimodal data.

[0181] Corresponding to the above-described test method for predicting vehicle trajectories based on multimodal data of autonomous driving, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described test method for predicting vehicle trajectories based on multimodal data of autonomous driving.

[0182] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0183] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0185] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the information processing methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0186] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A test method for predicting vehicle trajectories based on multimodal data from autonomous driving, characterized in that, The method includes: The system obtains the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment when the autonomous driving algorithm plans the trajectory for the vehicle under test in the driving scenario, and obtains the actual trajectory of the vehicle under test at the current moment. Based on all first predicted probabilities and all second predicted probabilities, the test vehicle and the first target are interacted to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state. Based on all the first predicted probabilities, the matching degree between the observation data of the second target at the current time and each of the first state estimates, the first state transition matrix, and all the first state estimates, the second state estimates and the second state transition matrix of the vehicle under test in each state are obtained. All second-state estimates and the second-state transition matrix are input into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test; By comparing the driving trajectory with the actual trajectory, the trajectory test results of the vehicle under test are obtained.

2. The test method for predicting vehicle trajectories based on multimodal data of autonomous driving according to claim 1, characterized in that, The step of interacting the vehicle under test and the first target based on all first predicted probabilities and all second predicted probabilities to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state includes: For each state, the average probability of the vehicle under test being in that state is calculated based on the first predicted probability of the vehicle under test being in each state and the theoretical probability of the vehicle under test transitioning from each state to that state. Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, and the theoretical probability of the vehicle under test transitioning from each state to each state, calculate the initial first state estimate of the second target being in the state after interaction and the probability of transitioning from the state to each state in the initial first state transition matrix. Based on the theoretical state transition matrix of the second target and the preset process noise covariance matrix corresponding to the weather type at the current moment, all initial first state estimates and the initial first state transition matrices are corrected to obtain the target first state estimates and target first state transition matrices of the second target in all states.

3. The test method for predicting vehicle trajectories based on multimodal data of autonomous driving according to claim 2, characterized in that, Based on the average probability of the vehicle under test being in the stated state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the stated state, the theoretical state estimate of the vehicle under test being in each state at the current moment, and the theoretical probability of the vehicle under test transitioning from each state to each state, the initial first state estimate of the second target being in the stated state after the interaction, and the probabilities of transitioning from the stated state to each state in the initial first state transition matrix are calculated, including: Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, the initial first state estimate of the second target being in the state after the interaction is calculated. Based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, the theoretical state estimate of the vehicle under test being in each state at the current moment, the theoretical probability of the vehicle under test being in each state, and the initial first state estimate of the second target being in the state, the probability of the second target transitioning from the state to each state in the initial first state transition matrix after interaction is calculated.

4. The test method for predicting vehicle trajectory based on multimodal data of autonomous driving according to claim 3, characterized in that, The step of calculating the initial first state estimate of the second target in the state after interaction, based on the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment, includes: Substituting the average probability of the vehicle under test being in the state, the second predicted probability of the first target being in each state, the theoretical probability of the first target transitioning from each state to the state, and the theoretical state estimate of the vehicle under test being in each state at the current moment into the following formula, we obtain the initial first state estimate of the second target being in the state after the interaction. in, For the initial first state estimate of the second objective i in state m. Let μ be the average probability that the vehicle a is in state m, M be the number of states, and μ be the average probability that the vehicle a is in state m. (b) n p represents the second predicted probability that the first objective is in state n. (b) nm Let m be the theoretical probability that the first objective is in a state transitioning from state n to state m. This is the theoretical state estimate of the vehicle under test, a, in state n at the current moment.

5. The test method for predicting vehicle trajectories based on multimodal data of autonomous driving according to claim 1, characterized in that, The step of obtaining the second state estimate and second state transition matrix of the vehicle under test in each state based on all first predicted probabilities, the matching degree between the observation data of the second target at the current time and each of the first state estimates, the first state transition matrix, and all first state estimates includes: The first predicted probabilities are updated based on the matching degree between the observation data of the second target at the current time and all first state estimates, to obtain the third predicted probabilities of the vehicle under test being in each state. Based on all third predicted probabilities and all first state estimates, predict the second state estimates of the test vehicle in each state; Based on all the third predicted probabilities, the first state transition matrix, all the first state estimates, and all the second state estimates, predict the second transition probabilities of the vehicle under test transitioning from each state to each state at the current time.

6. The test method for predicting vehicle trajectory based on multimodal data of autonomous driving according to claim 5, characterized in that, The third prediction probability of the vehicle under test being in any state is predicted by the following steps: Substituting the matching degree between the observation data of the second target at the current moment and the first state estimate of the second target in the stated state, and the first predicted probability of the vehicle under test being in the stated state, into the following formula, a third predicted probability of the vehicle under test being in the stated state is obtained, including: Where, μ ′ m Let be the third predicted probability of the vehicle under test being in state m. γ is the average probability that the vehicle under test is in state m. m The degree of matching between the observed data of the second target at the current moment and the first state estimate of the second target in state m, where M is the number of states. γ is the average probability that the vehicle under test is in state n. n The degree of matching between the observation data of the second target at the current moment and the first state estimate of the second target in state n.

7. The test method for predicting vehicle trajectories based on multimodal data of autonomous driving according to claim 1, characterized in that, The comparison of the driving trajectory and the actual trajectory to obtain the trajectory test result of the vehicle under test includes: If the driving trajectory matches the actual trajectory, then the trajectory test result is a pass. If the driving trajectory is inconsistent with the actual trajectory, the trajectory test result is a test failure.

8. A testing device for predicting vehicle trajectories based on multimodal data from autonomous driving, characterized in that, The device includes: The acquisition module is used to acquire the first predicted probability of the vehicle under test being in each state at the current moment and the second predicted probability of the first target being in each state at the current moment, which are predicted by the autonomous driving algorithm when planning the trajectory for the vehicle under test in the driving scenario, and to acquire the actual trajectory of the vehicle under test at the current moment. An interaction module is used to interact the vehicle under test and the first target based on all first prediction probabilities and all second prediction probabilities to obtain the first state transition matrix of the second target after the interaction and the first state estimate of each state. The update prediction module is used to obtain the second state estimate and the second state transition matrix of the vehicle under test in each state based on all first prediction probabilities, the matching degree between the observation data of the second target at the current time and each first state estimate, the first state transition matrix and all first state estimates. The input module is used to input all the second state estimates and the second state transition matrix into the vehicle trajectory prediction model to obtain the driving trajectory of the vehicle under test; The comparison module is used to compare the driving trajectory with the actual trajectory to obtain the trajectory test results of the vehicle under test.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the test method for predicting vehicle trajectories based on autonomous driving multimodal data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the test method for predicting vehicle trajectories based on autonomous driving multimodal data as described in any one of claims 1 to 7.

Images