Traffic state determination method and device, equipment and storage medium

Abstract

The invention provides a traffic state determination method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical fields of artificial intelligence, big data, deep learning, automatic driving and the like. According to the specific implementation scheme, initial state information is obtained, the initial state information at least can represent the initial traffic state of each traffic participant in a plurality of traffic participants contained in each frame of point cloud data in T frames of point cloud data, the initial traffic state of each traffic participant comprises a plurality of state values, and the state values represent the values of traffic parameters; determining a target factor graph; the target factor graph can constrain the state value of the state variable based on each preset constraint factor in a plurality of preset constraint factors; the state variable is determined from the plurality of traffic parameters based on a constraint target needing to be realized by a preset constraint factor; and obtaining a target traffic state of each traffic participant in the plurality of traffic participants by using the target factor graph.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to technical fields such as artificial intelligence, big data, deep learning, and autonomous driving. Background Art

[0002] With the widespread adoption of autonomous driving, the requirements for the accuracy and stability of autonomous driving system decisions are becoming increasingly stringent. The key to accurate decision-making lies in the quality of data samples. However, existing data samples often contain abnormal traffic conditions and are expensive to collect. Therefore, optimizing the traffic conditions in data samples and improving their accuracy have become key issues that need to be addressed. Summary of the Invention

[0003] The present disclosure provides a traffic status determination method, apparatus, device, and storage medium.

[0004] According to one aspect of the present disclosure, a method for determining a traffic state of a traffic participant is provided, comprising:

[0005] Acquiring initial state information, wherein the initial state information can at least represent an initial traffic state of each of a plurality of traffic participants included in each frame of point cloud data in T frames, the initial traffic state of the traffic participant including a plurality of state values, each state value representing a value of a traffic parameter; and T being an integer greater than 1;

[0006] Determining a target factor graph; wherein the target factor graph is capable of constraining a state value of a state variable based on each of a plurality of preset constraint factors; the state variable is determined from the plurality of traffic parameters based on a constraint target to be achieved by the preset constraint factors;

[0007] The target traffic state of each traffic participant among the multiple traffic participants is obtained by using the target factor graph.

[0008] According to another aspect of the present disclosure, a device for determining a traffic state of a traffic participant is provided, comprising:

[0009] an information acquisition unit, configured to acquire initial state information, wherein the initial state information can at least represent an initial traffic state of each of a plurality of traffic participants included in each frame of point cloud data in T frames, the initial traffic state of the traffic participants including a plurality of state values, each state value representing a value of a traffic parameter; and T being an integer greater than 1;

[0010] A factor graph determining unit, configured to determine a target factor graph; wherein the target factor graph is capable of constraining a state value of a state variable based on each of a plurality of preset constraint factors; and the state variable is determined from the plurality of traffic parameters based on a constraint target to be achieved by the preset constraint factors;

[0011] The state optimization unit is used to obtain the target traffic state of each traffic participant in the multiple traffic participants by using the target factor graph.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising the electronic device described above.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements any one of the methods according to the embodiments of the present disclosure.

[0019] In this way, the disclosed solution can optimize the traffic status of traffic participants contained in each frame of point cloud data with the help of the determined target factor graph, and thus obtain the optimized traffic status. In this way, the data quality of the traffic status can be significantly improved, thereby laying the foundation for providing rich and high-quality sample data for subsequent model training, and thus laying the foundation for improving the accuracy and stability of the decision-making of the autonomous driving system.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0022] Figure 1 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 1 ;

[0023] Figure 2 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 2 ;

[0024] Figure 3 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 3 ;

[0025] Figure 4 is a flow chart of a method for determining a traffic state of a traffic participant in a specific example according to an embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of the processing flow of a method for determining the traffic status of a traffic participant in a specific example according to an embodiment of the present application. Figure 1 ;

[0027] Figure 6 This is a schematic diagram of the processing flow of a method for determining the traffic status of a traffic participant in a specific example according to an embodiment of the present application. Figure 2 ;

[0028] Figure 7 is a schematic structural diagram of a device for determining a traffic state of a traffic participant according to an embodiment of the present application;

[0029] Figure 8 It is a block diagram of an electronic device used to implement the method for determining the traffic status of a traffic participant according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them, and do not mean to limit the order or to limit to only two. For example, the first feature and the second feature refer to two categories / two features. The first feature can be one or more, and the second feature can also be one or more.

[0032] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0033] The disclosed solution provides a method for determining the traffic state of traffic participants. After obtaining initial state information, that is, the initial traffic state of each traffic participant, the method can optimize the initial traffic state of the traffic participants using a constructed target factor graph, thereby improving the accuracy of the traffic state of the traffic participants. Here, due to the use of the factor graph, the relationship between state variables and constraint factors can be used to achieve global optimization of the traffic state of the traffic participants, thereby effectively improving the accuracy of the traffic state. This lays the foundation for providing rich and high-quality sample data for subsequent model training, thereby also laying the foundation for improving the accuracy and stability of the decision-making of the autonomous driving system.

[0034] Specifically, Figure 1 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0035] Furthermore, the method includes at least part of the following contents. Figure 1 Shown, including:

[0036] Step S101: Acquire initial state information.

[0037] Here, the initial state information can at least represent the initial traffic state of each traffic participant among the multiple traffic participants contained in each frame of point cloud data of T (T is an integer greater than 1) frames of point cloud data.

[0038] Furthermore, the initial traffic state of the traffic participant includes multiple state values, where the state value represents the value of the traffic parameter. In other words, the multiple state values ​​included in the initial traffic state of the traffic participant can be represented by the specific numerical value of each traffic parameter in the multiple traffic parameters.

[0039] Furthermore, the number of state values ​​included in the initial traffic state of the traffic participant is related to the number of traffic parameters. For example, the state values ​​and the traffic parameters are the same in number, that is, the two have a one-to-one correspondence.

[0040] Furthermore, in one example, the multiple traffic parameters include, but are not limited to: the size of the traffic participant (for example, it can be represented by the size of the detection box in which the traffic participant is located, such as by the length and width of the detection box), position, speed (for example, lateral speed and longitudinal speed), acceleration (for example, lateral acceleration and longitudinal acceleration), heading angle, etc. Furthermore, in one example, the multiple traffic parameters may also include category information (for example, the category attributes of the traffic participant, such as whether it is a large truck or a small passenger car, etc.) and semantic information (for example, whether the vehicle is a disabled vehicle or an unmanned vehicle, etc.).

[0041] It can be understood that the above-mentioned multiple traffic parameters can be used to describe the status of traffic participants in the traffic scene from different dimensions, thus laying the foundation for subsequent improvement of the accuracy and stability of the autonomous driving system's decision-making.

[0042] Here, the "traffic participants" referred to in the present disclosure may include but are not limited to vehicles, pedestrians, traffic lights, obstacles, etc. In other words, the present disclosure does not impose specific restrictions on traffic participants.

[0043] Step S102: Determine the target factor graph.

[0044] Here, the objective factor graph can constrain the state value of the state variable based on each of the plurality of preset constraint factors. In other words, the objective factor graph can constrain the values ​​of the state variable (i.e., the state value) using different preset constraint factors, thereby providing a quantifiable optimization solution and providing strong support for obtaining the optimal state value.

[0045] Furthermore, the state variable is determined from the plurality of traffic parameters based on the constraint target to be achieved by the preset constraint factor. In other words, the state variable is at least one of the plurality of traffic parameters.

[0046] It should be noted that, in the disclosed solution, different preset constraint factors need to achieve different constraint targets, thereby constraining the values ​​of state variables from different dimensions, thereby further providing support for obtaining the optimal state value.

[0047] Furthermore, the state variables constrained by different preset constraint factors may be the same or different. The present disclosure does not impose any specific restrictions on this, as long as the selected state variables can achieve the constraint objectives required by the preset constraint factors.

[0048] Step S103: using the target factor graph, obtaining a target traffic state of each of the multiple traffic participants.

[0049] That is to say, the disclosed solution can utilize the target factor graph to optimize the initial traffic state of each traffic participant among multiple traffic participants to obtain the optimized target traffic state of each traffic participant. In this way, the accuracy of the traffic state of each traffic participant contained in each frame of point cloud data can be effectively improved, thereby effectively improving the data quality.

[0050] In this way, the disclosed solution can optimize the traffic status of traffic participants contained in each frame of point cloud data with the help of the determined target factor graph, and thus obtain the optimized traffic status. In this way, the data quality of the traffic status can be significantly improved, thereby laying the foundation for providing rich and high-quality sample data for subsequent model training, and thus laying the foundation for improving the accuracy and stability of the decision-making of the autonomous driving system.

[0051] Furthermore, because the disclosed solution leverages the constraint relationships between state variables and constraint factors in the objective factor graph to globally optimize the traffic states of traffic participants, it effectively suppresses noise that may exist in single-frame point cloud data, resulting in a more accurate optimized traffic state for traffic participants. Furthermore, the solution is simple, practical, and easy to implement, providing strong support for efficiently and cost-effectively obtaining high-quality traffic states.

[0052] Figure 2 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters and other electronic devices. It is understood that the above Figure 1 The relevant contents of the method shown can also be applied to this example, and this example will not elaborate on the relevant contents.

[0053] Furthermore, the method includes at least part of the following contents. Figure 2 Shown, including:

[0054] Step S201: Acquire initial state information.

[0055] Here, the initial state information can at least represent the initial traffic state of each of the multiple traffic participants contained in each frame of T (T is an integer greater than 1) frames of point cloud data. Furthermore, the initial traffic state of the traffic participants includes multiple state values, each of which represents a value of a traffic parameter.

[0056] It should be noted that the relevant content about traffic participants and initial traffic status can be referred to the above examples and will not be repeated here.

[0057] Step S202: Determine the target factor graph.

[0058] Here, the target factor graph can constrain the state value of the state variable based on each preset constraint factor among a plurality of preset constraint factors, and the state variable is determined from the plurality of traffic parameters based on the constraint target to be achieved by the preset constraint factor.

[0059] Here, the relevant content about the target factor graph can be referred to the above example and will not be repeated here.

[0060] Step S203: using the target factor graph, obtaining a target traffic state of each of the multiple traffic participants.

[0061] Step S204: obtaining a target trajectory of each traffic participant based on the target traffic state of each traffic participant among the multiple traffic participants.

[0062] That is to say, after the target traffic state of each traffic participant is obtained by optimizing the target factor graph, the disclosed solution can further use the target traffic state of each traffic participant to obtain the target trajectory of each traffic participant; for example, for the same traffic participant, a continuous path of the traffic participant can be determined based on the target traffic state of the traffic participant contained in each frame of the T frame, and the continuous path is the target trajectory of the traffic participant.

[0063] It should be noted that the target trajectory of each traffic participant obtained by the disclosed solution can be the movement trajectory of the traffic participant in the T-frame point cloud data, or the predicted movement trajectory of the traffic participant at a future moment. In other words, the target traffic state of the traffic participant obtained by the disclosed solution can not only accurately obtain the movement trajectory of the traffic participant in the historical period, but also predict the movement trajectory of the traffic participant in the future period.

[0064] In this way, the disclosed solution can utilize the optimized target traffic state of each traffic participant to efficiently obtain the target trajectory of each traffic participant, thereby effectively improving the accuracy and reliability of the target trajectory, laying the foundation for subsequent improvements in the accuracy and stability of the autonomous driving system's decision-making.

[0065] Figure 3 This is a schematic flow chart of a method for determining the traffic status of a traffic participant according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters and other electronic devices. It is understood that the above Figure 1 and Figure 2 The relevant contents of the method shown can also be applied to this example, and this example will not elaborate on the relevant contents.

[0066] Furthermore, the method includes at least part of the following contents. Figure 3 As shown, including:

[0067] Step S301: Input the target multimodal spatiotemporal data into the initial prediction model to obtain the model output result.

[0068] Here, the model output result at least includes the initial traffic state of each traffic participant.

[0069] Furthermore, the target multimodal spatiotemporal data includes the T-frame point cloud data corresponding to the historical period.

[0070] In other words, the initial prediction model can determine the traffic status of each traffic participant based on the target multimodal spatiotemporal data.

[0071] Step S302: Post-process the model output result to obtain initial state information.

[0072] Here, the initial state information can at least represent the initial traffic state of each of the multiple traffic participants contained in each frame of T (T is an integer greater than 1) frames of point cloud data. Furthermore, the initial traffic state of the traffic participants includes multiple state values, each of which represents a value of a traffic parameter.

[0073] It should be noted that the relevant content about traffic participants and initial traffic status can be referred to the above examples and will not be repeated here.

[0074] That is to say, in this example, the initial state information is obtained after post-processing the model output results, so as to improve the data accuracy of the initial state information.

[0075] Step S303: Determine the target factor graph.

[0076] Here, the target factor graph can constrain the state value of the state variable based on each preset constraint factor among a plurality of preset constraint factors, and the state variable is determined from the plurality of traffic parameters based on the constraint target to be achieved by the preset constraint factor.

[0077] Here, the relevant content about the target factor graph can be referred to the above example and will not be repeated here.

[0078] Step S304: using the target factor graph, obtaining the target traffic state of each of the multiple traffic participants.

[0079] It should be noted that after the target traffic state of each traffic participant is obtained by optimizing the target factor graph, the target traffic state of each traffic participant can be further used to obtain the target trajectory of each traffic participant.

[0080] In this way, the disclosed solution provides a specific solution for obtaining initial state information, that is, the target multimodal spatiotemporal data is processed using the initial prediction model to obtain the model output result, and then the model output result is post-processed to obtain the initial state information. In this way, the traffic status of each traffic participant in the target multimodal spatiotemporal data is labeled using the model, which can effectively shorten the labeling time and thus greatly improve the processing efficiency; moreover, the disclosed solution also uses the target factor graph to optimize the initial state information, effectively avoiding the problem of inaccurate model labeling, thereby significantly improving the accuracy and robustness of traffic status labeling, thereby laying the foundation for subsequent provision of high-quality sample data for model training.

[0081] Furthermore, in a specific example, after obtaining the target traffic state of each of the multiple traffic participants, the target traffic state of each traffic participant may be used to perform model training on the initial prediction model; for example, in one example, after step S304, the method further includes:

[0082] Step S305: Based on the initial traffic state of each traffic participant and the target traffic state of each traffic participant, obtain state difference information of each traffic participant.

[0083] Step S306: using the status difference information of each traffic participant to train the initial prediction model to obtain a target prediction model.

[0084] That is to say, in one example, after obtaining the target traffic state of each traffic participant, the state difference information of each traffic participant is obtained based on the initial traffic state of each traffic participant and the target traffic state of each traffic participant, and then the state difference information of each traffic participant is used to adjust some adjustable parameters in the initial prediction model to obtain the target prediction model, thereby effectively improving the accuracy of the model output results.

[0085] In this way, the disclosed solution provides a specific solution for reversely optimizing the initial prediction model based on the post-processing results, that is, utilizing the state difference information between the initial traffic state of the traffic participants and the target traffic state obtained by post-processing to optimize the initial prediction model to obtain the target prediction model. In this way, the accuracy of the model's labeling results is effectively improved, making the model's output results more accurate and reliable, thereby laying the foundation for improving the accuracy and stability of the autonomous driving system's decision-making.

[0086] Furthermore, in a specific example, the post-processing of the model output result to obtain the initial state information (e.g., step S302) may specifically include:

[0087] Step S302-1: using the output result of the model, the traffic participants between different frames of point cloud data are associated (for example, the same traffic participant in different frames of point cloud data is associated) to obtain an initial association result.

[0088] Here, the initial association result may at least characterize the relationship between traffic participants between different frames of point cloud data. For example, the initial association result may mark the same traffic participant in different frames of point cloud data.

[0089] Step S302 - 2 : performing trajectory filtering processing on the initial traffic state of each traffic participant in the output result of the model to obtain the target modified traffic state of each traffic participant.

[0090] Step S302 - 3 : Based on the target-corrected traffic state of each traffic participant, the initial association result is fine-tuned to obtain the target association result, so as to obtain initial state information including the target association result.

[0091] That is to say, in this example, the trajectory filtering processing results, that is, the corrected traffic status, can be used to optimize the initial association results. In this way, the initial association results can be further corrected to make the association information in the association results more accurate, thereby effectively improving the data quality of the obtained initial state information, laying the foundation for providing rich and high-quality sample data for subsequent model training, and thus laying the foundation for improving the accuracy and stability of the autonomous driving system's decision-making.

[0092] In this way, the disclosed solution can use the target-corrected traffic status of each traffic participant obtained after trajectory filtering processing to fine-tune the initial association result to obtain the target association result. In this way, the accuracy and robustness of the association information between traffic participants between different frame point cloud data can be significantly improved. Moreover, the disclosed solution uses the target-corrected traffic status of traffic participants as the basis for fine-tuning, which can effectively correct obvious association errors. In this way, the rationality and reliability of the association result are enhanced, and a more accurate and more practical target association result is obtained, which lays the foundation for providing rich and high-quality sample data for subsequent model training, and thus lays the foundation for improving the accuracy and stability of the decision-making of the autonomous driving system.

[0093] Furthermore, in a specific example, the target modified traffic state of each traffic participant may be obtained in the following manner. Specifically, the above-mentioned trajectory filtering process of the initial traffic state of each traffic participant in the model output result to obtain the target modified traffic state of each traffic participant (e.g., step S302-2) may specifically include:

[0094] Step S302-2-1: Based on the time sequence in the T-frame point cloud data, forward filtering is performed on the initial traffic state of each traffic participant in the model output result to obtain the initial corrected traffic state of each traffic participant.

[0095] For example, in one example, a standard Kalman filter method may be used to perform forward filtering on the initial traffic state of each traffic participant in the output result of the model.

[0096] Step S302-2-2: Based on the time sequence in the T-frame point cloud data, perform backward smoothing on the initial corrected traffic state of each traffic participant to obtain the target corrected traffic state of each traffic participant.

[0097] It should be noted that, since each frame in the T-frame point cloud data corresponds to a time point, the processing direction of the trajectory filtering processing of the T-frame point cloud data can be determined according to the specified time sequence. For example, if the T-frame point cloud data is processed in the first order from the first time point to the last time point, the processing direction can be called "forward", or if the T-frame point cloud data is processed in the second order from the last time point to the first time point, the processing direction can be called "backward".

[0098] In other words, the "forward" in the forward filtering process can specifically refer to the direction from the starting point to the end point. Correspondingly, the "backward" in the backward smoothing process can specifically refer to the direction from the end point to the starting point. Here, the first frame is the starting point and the last frame is the end point.

[0099] In this way, the disclosed solution provides a specific solution for trajectory filtering processing, that is, using a bidirectional mechanism of "forward filtering-backward smoothing" to complete the trajectory filtering processing of the initial traffic state of each traffic participant. The "forward filtering" in this solution can effectively suppress the error accumulation in the traffic state estimation process, while the "backward smoothing" can smooth the traffic state obtained by forward filtering and reduce the impact of random noise. In this way, the traffic state obtained by using the bidirectional processing mechanism not only takes into account the causality of the time series, but also uses global information for optimization, thereby providing a strong basis for the subsequent accurate correction of the erroneous associations in the initial association results.

[0100] Furthermore, in a specific example, forward filtering may be performed in the following manner to obtain the initial revised traffic state of each traffic participant. Specifically, the aforementioned forward filtering of the initial traffic state of each traffic participant in the model output result to obtain the initial revised traffic state of each traffic participant (e.g., step S302-2-1) may specifically include:

[0101] Based on the initial corrected traffic state x of traffic participants in the k-1th frame point cloud data k―1 , predict the predicted traffic state x of the traffic participants in the k-th frame point cloud data k|k―1 ; k is an integer greater than or equal to 1 and less than or equal to T;

[0102] Predicted traffic status x for traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k .

[0103] In this way, the disclosed solution provides a specific solution for forward filtering, that is, first predicting the predicted traffic status of traffic participants in the current frame, and then correcting the obtained predicted traffic status to obtain the forward filtering result of the current frame (that is, the initial corrected traffic status). In this way, the impact of noise and errors in the prediction process can be effectively reduced, and a correction result that is closer to the actual traffic status can be obtained, further improving the data quality.

[0104] Furthermore, in a specific example, the following method can be used to predict the traffic state x of traffic participants: k|k―1 Make corrections to obtain the initial corrected traffic state x of the traffic participants k Specifically, the above-mentioned predicted traffic state x of traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k , which may specifically include:

[0105] Based on the observed position z of the traffic participant in the k-th frame point cloud data k (observable by observation devices), and predicted traffic status x k|k―1 , and obtain the residual ε corresponding to each traffic participant in the k-th frame point cloud data k ;

[0106] Based on the residual ε corresponding to the traffic participants in the k-th frame point cloud data k , the predicted traffic state x of traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the kth frame k .

[0107] In this way, the disclosed solution can use the residual between the observed position of the traffic participant and the predicted traffic state to correct the predicted traffic state and obtain the initial corrected traffic state of the traffic participant. In this way, the noise and errors brought about by the prediction process can be effectively avoided, and the data quality can be further improved.

[0108] In a specific example, backward smoothing can be performed in the following manner to obtain the target corrected traffic state of each traffic participant. Specifically, the aforementioned backward smoothing of the initial corrected traffic state of each traffic participant based on the time sequence in the T-frame point cloud data to obtain the target corrected traffic state of each traffic participant can specifically include:

[0109] Correct the traffic state x according to the target of the traffic participant in the k+1 frame point cloud data k+1|T And the predicted traffic state x in the k+1 frame point cloud data k+1|k , the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k Correction is performed to obtain the target corrected traffic state x of the traffic participants in the k-th frame point cloud data k|T ; Wherein, k is an integer less than T-1.

[0110] Here, in one example, when k is T-1, the target modified traffic state x of the traffic participant in the T-th frame point cloud data is T|T is the initial corrected traffic state x T .

[0111] In this way, the disclosed solution provides a specific solution for backward smoothing, that is, the initial corrected traffic state of the traffic participants in each frame is corrected in a "backward" manner to obtain the target traffic state of the traffic participants in each frame. In this way, the traffic state obtained by forward filtering can be effectively smoothed, and the influence of random noise can be reduced, thereby providing a strong basis for the subsequent precise correction of erroneous associations in the initial association results.

[0112] The following is a detailed description of the forward filtering and backward smoothing process. Specifically,

[0113] (1) Forward filtering

[0114] Input: Observation sequence of traffic participants in T frames {z1,z2,…,z T}, z k (k=1,2,…,T) represents the observed position of the traffic participant in the kth frame, for example, it can be recorded as Initialize the initial traffic state of traffic participants in frame 0 Initialize the initial error covariance matrix P0; and input the transfer matrix F corresponding to the traffic participants, the process noise covariance matrix Q, the observation matrix H, and the observation noise covariance matrix R.

[0115] here, They represent the horizontal and vertical coordinates of the center of the traffic participant in the 0th frame in the world coordinate system respectively; They represent the speed of the traffic participant in the 0th frame along the horizontal axis and the vertical axis of the world coordinate system respectively; They represent the acceleration of the traffic participant in the 0th frame along the horizontal axis and the vertical axis of the world coordinate system respectively.

[0116] Furthermore, the process noise covariance matrix Q is used to describe the uncertainty of the movement of traffic participants; the transfer matrix F is used to describe the degree of change of the traffic state of traffic participants over time; the observation matrix H is used to describe whether the traffic parameters in the traffic state of traffic participants are observed by observation equipment; the observation noise covariance matrix R is used to describe the uncertainty of observation equipment (such as sensors, etc.); the initial error covariance matrix P0 is used to describe the uncertainty of each traffic parameter in the initial traffic state.

[0117] Output: The initial modified traffic state {x k}, k=1,2,…,T, and the initial correction error covariance matrix {P k},k=1,2,…,T.

[0118] The specific processing flow is as follows:

[0119] Step 11: For the kth frame (k is 1, 2, 3, ..., T), the prediction process is as follows: Specifically, according to the transfer matrix F and the initial modified traffic state x of the traffic participant in the k-1th frame, k―1 , predict the predicted traffic state x of the traffic participants in the kth frame k|k―1; and, based on the transfer matrix F, the observation noise covariance matrix R, and the initial correction error covariance matrix P corresponding to the traffic participants in the k-1 frame k―1 , predict the prediction error covariance matrix P corresponding to the traffic participants in the kth frame k|k―1 .

[0120] Here, the predicted traffic state x k|k―1 and the prediction error covariance matrix P k|k―1 The expression is as follows:

[0121] x k |k―1=F×x k―1 ,

[0122]

[0123] here, Represents the transpose operator.

[0124] When k is 1, the initial modified traffic state x of the traffic participant in the k-1th frame is k―1 is the initial traffic state x0 of the traffic participant in the 0th frame after initialization, and the initial correction error covariance matrix P corresponding to the traffic participant in the k-1th frame k―1 is the initial error covariance matrix P0 corresponding to the traffic participants in the 0th frame after initialization.

[0125] Step 12: Update process; Specifically, according to the observation matrix H, the observation position z of the traffic participant in the kth frame k , and predicted traffic status x k|k―1 , and obtain the residual ε corresponding to the traffic participant in the kth frame k ; and according to the observation matrix H, the observation noise covariance matrix R, and the prediction error covariance matrix P corresponding to the traffic participants in the kth frame k|k―1 , get the observation covariance matrix S corresponding to the traffic participants in the kth frame k .

[0126] Here, the residual ε corresponding to the traffic participant in the kth frame is k and the observation covariance matrix S k The specific calculation expression is as follows:

[0127] ε k =z k ―H×x k|k―1 ,

[0128]

[0129] Step 13: Based on the observation matrix H and the prediction error covariance matrix P corresponding to the traffic participants in the kth frame k|k―1, and the corresponding observation covariance matrix S k , get the Kalman gain information K corresponding to the traffic participant in the kth frame k ; Here, the Kalman gain information K corresponding to the traffic participant in the kth frame k The specific calculation expression is as follows:

[0130]

[0131] Step 14: Based on the Kalman gain information K corresponding to the traffic participant in the kth frame k , residual ε k , the predicted traffic state x of the traffic participants in the kth frame k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the kth frame k , its specific expression is as follows:

[0132] x k =x k|k―1 +K k ×ε k .

[0133] Step 15: Based on the observation matrix H and the Kalman gain information K corresponding to the traffic participants in the kth frame k , the prediction error covariance matrix P corresponding to the traffic participants in the kth frame k|k―1 Correction is performed to obtain the initial error covariance matrix P corresponding to the traffic participants in the kth frame k , its specific expression is as follows:

[0134] P k =(I-K k ×H)×P k|k―1

[0135] Here, I represents the identity matrix.

[0136] Step 16: Traverse all k values ​​and obtain the initial modified traffic state of the traffic participants in each frame of T frame (which can be recorded as {x k},k=1,2,…,T), and the initial error covariance matrix (which can be recorded as {P k},k=1,2,…,T).

[0137] (2) Backward Smoothing

[0138] Input: The initial corrected traffic state of traffic participants in each frame of T frame Predicting traffic conditions Initial correction error covariance matrix Forecast error covariance matrix The transfer matrix F corresponding to the traffic participants.

[0139] Output: The target-corrected traffic state of the traffic participants in each frame of T frame and the target-corrected error covariance

[0140] The specific processing flow is as follows:

[0141] Step 21: For the Tth frame, the initial modified traffic state x of the traffic participant in the Tth frame is T As the target, the traffic state x is corrected T|T ; The initial correction error covariance matrix P of the traffic participants in the Tth frame T As the target correction error covariance matrix P T|T .

[0142] Step 22: For the kth frame (k is T-1, ..., 1), according to the transfer matrix F, the initial correction error covariance matrix P of the traffic participants in the kth frame k , and the prediction error covariance matrix P of traffic participants in the k+1th frame k+1|k , get the smooth gain information C corresponding to the traffic participant in the kth frame k Here, the smoothing gain information C k The calculation expression is as follows:

[0143]

[0144] Step 23: According to the smoothing gain information C corresponding to the traffic participant in the kth frame k , the target corrected traffic state x of the traffic participant in the k+1th frame k+1|T , predict traffic status x k+1|k , the initial modified traffic state x of the traffic participants in the kth frame k Make corrections to obtain the target corrected traffic state x of the traffic participants in the kth frame k|T ; Here, the target corrects the traffic state x k|T The calculation expression is:

[0145] x k|T =x k +C k ×(x k+1|T ―x k+1|k )

[0146] Step 24: According to the smoothing gain information C corresponding to the traffic participant in the kth frame k , the target correction error covariance matrix P of traffic participants in the k+1th frame k+1|T , prediction error covariance matrix P k+1|k , the initial correction error covariance matrix P of the traffic participants in the kth framek Correction is performed to obtain the target correction error covariance matrix P of the traffic participants in the kth frame k|T ; Here, the target correction error covariance matrix P k|T The calculation expression is:

[0147]

[0148] Step 25: Traverse all k values ​​and obtain the target modified traffic state of the traffic participants in each frame of T frame (which can be recorded as ), and the target correction error covariance (which can be recorded as ).

[0149] In this way, the disclosed solution can utilize the bidirectional mechanism of "forward filtering-backward smoothing" to complete the trajectory filtering processing of the initial traffic state of each traffic participant. This solution not only takes into account the causality of the time series, but also utilizes global information for optimization, thereby providing a strong basis for the subsequent accurate correction of erroneous associations in the initial association results.

[0150] Furthermore, in a specific example, after obtaining the initial state information containing the association results of the traffic participants (such as the target association results), the target association results can be used to perform data enhancement on the point cloud data of the traffic participants in at least some frames of the T-frame point cloud data to obtain enhanced T-frame point cloud data. That is, the target association results are used to reconstruct the point cloud data of the traffic participants in at least some frames of the T-frame point cloud data (also referred to as target reconstruction), thereby improving the data quality of the point cloud data. Furthermore, the enhanced T-frame point cloud data is used to perform orientation consistency detection on the orientation of the same traffic participant between different frames to obtain a traffic participant orientation detection result; and then, based on the traffic participant orientation detection result, the orientation angle of the traffic participant in the initial state information is adjusted to obtain adjusted initial state information.

[0151] It should be noted that, in one example, the orientation angle of the traffic participant in the t-th frame point cloud data is denoted as θ t , the orientation angle of the traffic participant in the t-1 frame point cloud data is θ t―1 , let the direction detection result of the traffic participant be δ. At this time, the direction detection result δ of the traffic participant can be expressed as follows:

[0152]

[0153] Here, f(·) represents a quantization function. If the value of δ is greater than a preset threshold (for example, greater than 0.8), it means that the orientation angles of traffic participants are inconsistent. In this case, the following optimization method can be used:

[0154] For any t (t is 2,…,T) value, the orientation angle is optimized. For example, the orientation angle θ of the traffic participant in the t-1 frame point cloud data is t―1 As the orientation angle of the traffic participant in the t-th frame point cloud data, let θ t =θ t―1 , thus ensuring that traffic participants are facing the same direction.

[0155] In this way, the disclosed solution further optimizes the traffic status of traffic participants by enhancing their point cloud data and detecting consistent orientation in post-processing, thereby obtaining more accurate initial status information, thereby providing strong support for improving data quality.

[0156] Specifically, if Figure 4 and Figure 5 As shown, the processing steps of the disclosed solution include:

[0157] Step S401: Obtain the model output result output by the initial prediction model, wherein the model output result includes the attributes of each traffic participant (corresponding to the above-mentioned traffic status).

[0158] Step S402: Using the target association module and combining the model output results, the traffic participants between the point cloud data of different frames are associated to obtain the initial association results.

[0159] Here, in this example, in order to avoid possible association errors in the target association process, a forward filtering-backward smoothing module can be used instead of a standard Kalman filter module to re-estimate the traffic state with high confidence, thereby optimizing the association errors in the target association.

[0160] Step S403: Using the forward filtering-backward smoothing module, the initial traffic state of each traffic participant in the model output result is subjected to trajectory filtering processing, and the initial association result is fine-tuned using the result after trajectory filtering processing (corresponding to the target-corrected traffic state of each traffic participant mentioned above) to obtain the target association result, and then the initial state information containing the target association result is obtained.

[0161] Here, for relevant examples of forward filtering and backward smoothing, please refer to the above description and will not be repeated here.

[0162] Step S404: Using the target reconstruction module and combining the target association results, the point cloud data of the traffic participants in at least some frames of the T-frame point cloud data are reconstructed (corresponding to the above-mentioned data enhancement) to obtain the reconstructed T-frame point cloud data.

[0163] Step S405: Using the orientation consistency detection module and combining the reconstructed T-frame point cloud data, the orientation consistency detection is performed on the orientation of the same traffic participant between different frames to ensure that the orientation angle of the traffic participant between different frames remains consistent and obtain the adjusted initial state information.

[0164] Step S406: Determine the target factor graph.

[0165] Step S407: optimizing the initial traffic state of each of the multiple traffic participants included in the initial state information through the offline global optimization module and in combination with the target factor graph to obtain the target traffic state of each traffic participant.

[0166] Here, after obtaining the target traffic state of each traffic participant, non-maximum suppression (NMS) can be used to remove redundant traffic state frame values ​​in the target traffic state of each traffic participant to obtain an updated target traffic state of each traffic participant.

[0167] It should be noted that in practical applications, position differentials are often used to optimize the initial traffic states of traffic participants contained in the initial state information. However, the traffic states determined using this method are relatively unstable and difficult to use as a basis for trajectory estimation. In view of this, the disclosed solution provides an optimization scheme based on a factor graph. This scheme can achieve offline global optimization, effectively improving the accuracy of traffic state estimation and, in turn, the stability of the resulting trajectory.

[0168] Furthermore, in a specific example, the constraint targets of different preset constraint factors in the target factor graph of the disclosed solution are different. This makes it easy to use different preset constraint factors to constrain the state values ​​of state variables from different dimensions, thereby further improving the accuracy of the optimization results, laying the foundation for providing rich and high-quality sample data for subsequent model training, and thus laying the foundation for improving the accuracy and stability of autonomous driving system decision-making.

[0169] Furthermore, in a specific example, the target factor graph includes at least two of the following:

[0170] First factor graph, second factor graph, third factor graph, fourth factor graph, fifth factor graph, sixth factor graph.

[0171] Furthermore, the first factor graph is constructed using the position of a traffic participant (e.g., the center of the detection frame containing the traffic participant) as a state variable, with the constraint factor being the ability to prevent the center of the traffic participant from jumping between frames (e.g., constraining the center of the detection frame to prevent it from jumping). This first factor graph effectively prevents abnormal jumps in the center of the traffic participant between frames, thereby improving the reliability and rationality of the state values ​​of the positions in the traffic state.

[0172] It's important to note that in practical applications, the initial traffic states (e.g., position, size, speed, acceleration, and heading angle) of traffic participants vary between frames, leading to issues like flickering and jumping trajectories. Furthermore, fluctuations in the detection frames of traffic participants between frames can also cause slight displacements when participants transition from a stationary state to a moving state. Using the first factor graph for constraints can effectively avoid these issues.

[0173] Furthermore, the second factor graph is constructed using the acceleration of traffic participants between two consecutive frames as the state variable and the control of the traffic participants' target acceleration (e.g., uniform acceleration) as the preset constraint factor. This second factor graph ensures that traffic participants maintain a constant uniform acceleration across consecutive frames, effectively controlling their acceleration and improving the reliability and rationality of the acceleration state values ​​in traffic conditions.

[0174] Furthermore, the third factor graph is constructed using the position of traffic participants as the state variable and the matching of their speed and position jumps as the preset constraint. This effectively addresses the mismatch between speed estimates and displacement jumps.

[0175] Furthermore, the fourth factor graph is constructed using the speed of traffic participants between multiple frames as the state variable and preventing sudden acceleration changes as the preset constraint factor. This effectively suppresses sudden acceleration changes, thereby improving trajectory smoothness.

[0176] Furthermore, the fifth factor graph is a factor graph constructed by using the position of the traffic participant as a state variable to prevent the center position of the traffic participant in a stationary state from jumping between different frames to a preset constraint factor.

[0177] Furthermore, the sixth factor graph is constructed using the position of traffic participants as the state variable and the size variation of the detection frames corresponding to traffic participants between frames as a preset constraint factor. This effectively prevents abnormal size variations of traffic participant detection frames between frames, thereby improving detection frame quality. It also indirectly constrains the state value of the position in the traffic state, providing strong support for improving data quality.

[0178] In this way, the disclosed solution provides a specific solution for constructing a target factor graph, so as to construct different factor graphs from different dimensions (position, speed, acceleration, size, etc.). This provides a theoretical basis for the subsequent optimization of the values ​​of state variables from different dimensions, and lays the foundation for providing rich and high-quality sample data for subsequent model training, thereby also laying the foundation for improving the accuracy and stability of autonomous driving system decision-making.

[0179] Furthermore, for the first factor graph, in one example, the preset constraint factor corresponding to the first factor graph can be represented by a first constraint function.

[0180] Furthermore, in one example, the first constraint function can represent at least one of the following:

[0181] The difference between the actual position of the traffic participant and the observed position (in practical applications, this can be obtained through observation equipment such as sensors);

[0182] The covariance of the residuals between the actual and observed positions of traffic participants.

[0183] Here, the actual position value may specifically be a state value of the position of the traffic participant in the point cloud data, in other words, a frame value of the position of the traffic participant in the point cloud data.

[0184] Furthermore, the position observation value can be obtained based on a preset observation device.

[0185] It should be noted that the first constraint function in the above example can be expressed from the position difference dimension, or from the covariance dimension of the position residual, or from the position difference dimension and the covariance dimension of the residual. The present disclosure does not make any specific limitations on this.

[0186] For example, in one example, the position of the traffic participant in the t-th frame point cloud data (that is, the actual position value, which can be specifically the value of the position in the initial traffic state) is recorded as The position observation value of the traffic participant in the t-th frame point cloud data is recorded as The first constraint function is Then the first constraint function The specific expression is:

[0187]

[0188] Here, ‖·‖ is the norm operator.

[0189] Alternatively, in another example, based on the residual between the actual position value of the point cloud data of the traffic participant in the tth frame and its position observation value, the initial covariance matrix (which can be recorded as ∑ init ), further, based on the initial covariance matrix ∑ init The observation covariance matrix can be obtained (for example, denoted as ∑ obs ), that is, ∑ obs =diag(∑ init ), at this time, the observation covariance matrix ∑ obs That is, it can be used as the covariance of the residual between the actual position value and the position observation value of the traffic participant in the first constraint function.

[0190] Specifically, the first constraint function The specific expression is:

[0191]

[0192] Here, diag(·) is an operator for obtaining the diagonal elements of a matrix, and the value of n is related to T.

[0193] Here, in a specific example, the observation covariance matrix ∑ obs The mid-diagonal element and the observed variance σ obs Whether it meets the preset requirements, if so, the diagonal element is not processed, otherwise the observation variance σ is used obs Here, the observed variance σ obs According to the detection confidence (which can be recorded as conf t ) is dynamically adjusted, for example, in one example, σ obs =0.5×(1-conf t )+0.1.

[0194] Furthermore, in a specific example, the first constraint function It can also be specifically expressed by the following formula:

[0195]

[0196] In this way, the disclosed solution provides a specific constraint method for the first factor graph, so that when the traffic status of traffic participants is optimized using the first factor graph, the position of the traffic participants can be constrained by relying on the first constraint function. In this way, the problem of position jumps of the center points of traffic participants can be effectively avoided, the robustness and stability of observations can be improved, and the data reliability of the obtained traffic status can be improved.

[0197] Furthermore, for the second factor graph, in one example, the preset constraint factor corresponding to the second factor graph can be represented by a second constraint function.

[0198] Furthermore, in one example, the second constraint function can represent at least one of the following:

[0199] The degree of difference between the estimated speed of the traffic participant in the t+1th frame point cloud data and the actual speed value of the t+1th frame point cloud data; wherein the estimated speed value of the t+1th frame point cloud data is based on the actual speed value and actual acceleration value of the tth frame point cloud data.

[0200] The degree of difference between the estimated position of the traffic participant in the t+1th frame of point cloud data and the actual position of the t+1th frame of point cloud data; the estimated position of the t+1th frame of point cloud data is based on the actual speed, position, and acceleration values ​​of the tth frame of point cloud data;

[0201] Here, the speed actual value, the position actual value and the acceleration actual value can be obtained based on the initial traffic state.

[0202] It should be noted that the actual value of speed, the actual value of position and the actual value of acceleration can all correspond to the state value of speed, the state value of position and the state value of acceleration of the traffic participants in the point cloud data. In other words, they correspond to the frame value of speed, the frame value of position and the frame value of acceleration of the traffic participants in the point cloud data.

[0203] It should be further explained that the second constraint function in the above example can be expressed from the dimension of the degree of difference between the speeds of traffic participants, or it can be expressed from the dimension of the degree of difference between the positions of traffic participants, or it can be expressed from the above two dimensions. The present disclosure does not make specific limitations on this.

[0204] For example, let the actual speed of the traffic participant in the t+1 frame point cloud data be The estimated speed of the traffic participant in the t+1 frame point cloud data is The actual position of the traffic participant in the t+1 frame point cloud data is The estimated position of the traffic participant in the t+1 frame point cloud data is The actual speed of the traffic participant in the t-frame point cloud data is The actual acceleration value of the point cloud data of the traffic participant in the t frame is The second constraint function is

[0205] At this time, in one example, the second constraint function Characterizes the difference between the speed estimate of the traffic participant in the t+1 frame point cloud data and the actual speed value of the t+1 frame point cloud data, then the second constraint function The specific expression of can be:

[0206]

[0207] Here, ∑ ca represents the acceleration noise covariance matrix. For example, in one example, it can be a preset value, such as ∑ ca =diag(0.1,0.1,0.5,0.5).

[0208] Or, in another example, the second constraint function Characterizes the degree of difference between the estimated position value of the traffic participant in the t+1 frame point cloud data and the actual position value of the t+1 frame point cloud data, then the second constraint function The specific expression of can be:

[0209]

[0210] Alternatively, in yet another example, the second constraint function At the same time, it characterizes the difference between the estimated speed value of the traffic participant in the t+1 frame point cloud data and the actual speed value of the t+1 frame point cloud data, as well as the difference between the estimated position value of the traffic participant in the t+1 frame point cloud data and the actual position value of the t+1 frame point cloud data. Then the second constraint function The specific expression of can be:

[0211]

[0212] In this way, the disclosed solution provides a specific constraint method for the second factor graph, which can constrain the state variables from the dimensions of the speed and / or position of the traffic participants, so that the traffic participants can perform target accelerated motion, such as uniform accelerated motion, thereby improving the data reliability of the obtained traffic state, and laying the foundation for improving the user experience in the subsequent autonomous driving scenario.

[0213] Furthermore, for the third factor graph, in one example, the preset constraint factor corresponding to the third factor graph can be represented by a third constraint function.

[0214] Furthermore, in one example, the third constraint function can represent:

[0215] The degree of difference between the vector modulus of the speed of the traffic participant in the t-th frame point cloud data and the displacement change rate corresponding to the t-th frame point cloud data.

[0216] Here, the displacement change rate corresponding to the t-th frame point cloud data is obtained based on the actual position value of the t-th frame point cloud data and the actual position value of the t-1-th frame point cloud data. Here, the actual position value can be obtained based on the initial traffic state of the traffic participant.

[0217] For example, let the actual speed of the traffic participant in the t-th frame point cloud data be v t (Also known as ), the actual position value of the traffic participant in the t-th frame point cloud data is c t (Also known as ), the actual position value of the traffic participant in the t-1 frame point cloud data is c t―1 (Also known as ), let the third constraint function be At this time, the third constraint function It can be specifically expressed as:

[0218]

[0219] Here, λ kin represents a tuning parameter, which is a preset value. For example, in one example, λ kin =10, ‖·‖2 represents the two-norm operator.

[0220] In this way, the disclosed solution provides a specific constraint method for the third factor graph, which makes it easy to make the velocity vector modulus and displacement change rate in the traffic state of traffic participants consistent, thereby effectively solving the problem of mismatch between the degree of speed and displacement jump, improving the robustness and stability of observation, and thus improving the data reliability of the obtained traffic state.

[0221] Furthermore, for the fourth factor graph, in one example, the preset constraint factor corresponding to the fourth factor graph can be represented by a fourth constraint function.

[0222] Furthermore, in one example, the fourth constraint function can represent: a degree of difference between a position difference value corresponding to the point cloud data of the t+1th frame and a position difference value corresponding to the point cloud data of the tth frame of the traffic participant;

[0223] Among them, the position difference corresponding to the t+1th frame point cloud data represents the position difference between the actual position value of the t+1th frame point cloud data and the actual position value of the tth frame point cloud data, and the position difference corresponding to the tth frame point cloud data represents the position difference between the actual position value of the tth frame point cloud data and the actual position value of the t-1th frame point cloud data.

[0224] For example, let the actual position of the traffic participant in the t+1 frame point cloud data be c t+1 (Also known as The actual position value of the traffic participant’s point cloud data in frame t is c t (Also known as ), the actual position value of the traffic participant in the t-1 frame point cloud data is c t―1 (Also known as ), let the fourth constraint function be At this time, the fourth constraint function It can be specifically expressed as:

[0225]

[0226] Here, λ smooth represents the acceleration smoothing term, which is a preset value. For example, in one example, λ smooth =0.5.

[0227] In this way, the disclosed solution provides a specific constraint method for the fourth factor graph, which can effectively suppress acceleration mutations, thereby improving trajectory smoothness and thus improving the data reliability of the obtained traffic status.

[0228] Furthermore, for the fifth factor graph, in one example, the preset constraint factor corresponding to the fifth factor graph can be represented by a fifth constraint function.

[0229] Furthermore, in one example, the fifth constraint function can represent: when the traffic participant is determined to be a static target, the vector modulus of the velocity and / or the vector modulus of the acceleration of the traffic participant.

[0230] For example, in one example, the static flag determination condition can be expressed by the following formula:

[0231]

[0232] Here, c k represents the actual position value of the traffic participant in the k-th frame point cloud data; μ c Represents the sum of the squares of the residuals of the actual position value; in this way, the position changes of multiple consecutive frames (for example, five frames in this example) are used to determine whether the traffic participant is a static target.

[0233] Furthermore, in one example, the fifth constraint function It can be expressed by the following formula:

[0234]

[0235] Here, x t represents the actual position value of the traffic participant in the tth frame; v t represents the actual speed value of the traffic participant in the tth frame, a t represents the actual acceleration value of the traffic participant in the tth frame. ‖·‖2 represents the two-norm operator.

[0236] In practical applications, the fifth constraint function may also be adjusted based on actual scenarios, and the present disclosure does not limit the specific adjustment method.

[0237] In this way, the disclosed solution provides a specific constraint method for the fifth factor graph, which can prevent the center position of a stationary traffic participant from jumping between different frames, thereby improving the trajectory smoothness and thus improving the data reliability of the obtained traffic status.

[0238] For the sixth factor graph, the preset constraint factor corresponding to the sixth factor graph is represented by a sixth constraint function.

[0239] Furthermore, in one example, the sixth constraint function can represent: the degree of difference between the size (such as length and width) of the detection box where the traffic participant is located and the preset size;

[0240] Here, the size of the detection frame where the traffic participant is located can be obtained through the actual position value of the traffic participant; the actual position value can be obtained based on the initial traffic state; the preset size is obtained based on the size of the detection frame where the traffic participant is located in different frames.

[0241] For example, let the length of the detection frame of the traffic participant in the t-th frame point cloud data be l t , the width of the detection box of the traffic participant in the t-th frame point cloud data is w t , the preset length of the detection frame of the traffic participant point cloud data in the tth frame is The preset width of the detection frame of the traffic participant point cloud data in the tth frame is The sixth constraint function is At this time, the sixth constraint function (c t ) can be specifically:

[0242]

[0243] Here, the preset length is based on the lengths {l1,l2,…,l T}, for example, the preset length For {l1,l2,…,l T}. Preset length is based on the lengths of traffic participants in different frames {w1,w2,…,w T}, for example, the preset length For {w1,w2,…,w T}'s median.

[0244] In this way, the disclosed solution can utilize the sixth constraint function to further optimize the position of traffic participants, that is, to constrain the position of traffic participants from the size of the detection frame of the traffic participants to obtain the optimal state value, thereby improving the accuracy of the position results in the traffic state while improving the quality of the detection frame, thereby improving the data reliability of the obtained traffic state.

[0245] Continue with Figure 5 For example, in one example, the target factor graph may include the above-mentioned first factor graph, third factor graph and sixth factor graph. In other words, in the process of optimizing the initial traffic state of each traffic participant, the first constraint function corresponding to the first factor graph in the target factor graph, the third constraint function corresponding to the third factor graph and the sixth constraint function corresponding to the sixth factor graph are used to perform constrained optimization on the state values ​​of the positions included in the initial traffic state of the traffic participants.

[0246] Alternatively, in another example, the target factor graph may include the first factor graph, the second factor graph, the third factor graph, the fourth factor graph, the fifth factor graph, and the sixth factor graph. In other words, in the process of optimizing the initial traffic state of each traffic participant, Figure 6 As shown, the state values ​​of some traffic parameters contained in the initial traffic state of traffic participants are constrainedly optimized by utilizing the first constraint function corresponding to the first factor graph in the target factor graph, the second constraint function corresponding to the second factor graph, the third constraint function corresponding to the third factor graph, the fifth constraint function corresponding to the fifth factor graph, and the sixth constraint function corresponding to the sixth factor graph.

[0247] At this point, the above six constraint functions can be used to optimize the following target loss function to obtain the target traffic state of traffic participants (which can be recorded as X * ):

[0248]

[0249] here, represents the first constraint function, represents the second constraint function, represents the third constraint function, represents the fourth constraint function, represents the fifth constraint function, Represents the sixth constraint function.

[0250] It should be noted that the specific factor graph included in the target factor graph can be one or more of the six factor graphs mentioned above, which are not exhaustively listed here. In practical applications, it can be set according to actual needs, and the present disclosure does not impose specific restrictions on this.

[0251] Furthermore, in one example, a new factor graph, referred to as a seventh factor graph, may be added to the target factor graph of the disclosed solution. This seventh factor graph may be a factor graph constructed with the size (or position) of the detection frame containing the traffic participant as a state variable and the size of the detection frame containing the traffic participant as a positive preset constraint factor. Furthermore, the preset constraint factor corresponding to the seventh factor graph is represented by a seventh constraint function. Here, the seventh constraint function can represent that the size (e.g., length and width) of the detection frame containing the traffic participant is greater than 0. In this way, the size of the detection frame containing the traffic participant is effectively constrained.

[0252] In summary, the disclosed solution can achieve significant results in practical application scenarios (e.g., scenarios where vehicles repeatedly start and stop in parking lots and scenarios where bicycles turn right at intersections), as follows:

[0253] Scenario of repeated vehicle starts and stops in a parking lot: Before optimizing the traffic conditions for starting and stopping vehicles, the speed estimation response delay reached 0.8 seconds, and the trajectory obtained using the traffic conditions before optimization exhibited "S"-shaped jitter. After optimizing the traffic conditions for starting and stopping vehicles using the disclosed solution, the speed estimation response delay was less than 0.2 seconds, and the resulting trajectory had an 82% improvement in smoothness.

[0254] The disclosed solution also provides a traffic status determination device for traffic participants, such as Figure 7 Shown, including:

[0255] An information acquisition unit 701 is configured to acquire initial state information, wherein the initial state information can at least represent the initial traffic state of each of the multiple traffic participants included in each frame of point cloud data in T frames, wherein the initial traffic state of the traffic participants includes multiple state values, each of which represents a value of a traffic parameter; and T is an integer greater than 1.

[0256] A factor graph determining unit 702 is configured to determine a target factor graph; wherein the target factor graph is capable of constraining a state value of a state variable based on each of a plurality of preset constraint factors; and the state variable is determined from the plurality of traffic parameters based on a constraint target to be achieved by the preset constraint factors.

[0257] The state optimization unit 703 is configured to obtain a target traffic state of each of the multiple traffic participants using the target factor graph.

[0258] In a specific example of the disclosed solution, the system further includes a trajectory prediction unit; wherein the trajectory prediction unit is configured to:

[0259] A target trajectory of each traffic participant is obtained based on the target traffic state of each traffic participant among the multiple traffic participants.

[0260] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0261] The target multimodal spatiotemporal data is input into the initial prediction model to obtain a model output result; the model output result at least includes the initial traffic state of each traffic participant; the target multimodal spatiotemporal data includes the T-frame point cloud data corresponding to the historical time period; the model output result is post-processed to obtain the initial state information.

[0262] In a specific example of the disclosed solution, the apparatus further includes: a model training unit; wherein the model training unit is configured to:

[0263] Based on the initial traffic state of each traffic participant and the target traffic state of each traffic participant, the state difference information of each traffic participant is obtained;

[0264] The initial prediction model is trained using the status difference information of each traffic participant to obtain a target prediction model.

[0265] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0266] Using the output of the model, traffic participants between point cloud data of different frames are associated to obtain an initial association result;

[0267] performing trajectory filtering on the initial traffic state of each traffic participant in the output result of the model to obtain a target modified traffic state of each traffic participant;

[0268] The traffic state is corrected based on the target of each traffic participant, and the initial association result is fine-tuned to obtain the target association result, so as to obtain the initial state information including the target association result.

[0269] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0270] Based on the time sequence in the T-frame point cloud data, forward filtering is performed on the initial traffic state of each traffic participant in the output result of the model to obtain the initial corrected traffic state of each traffic participant;

[0271] Based on the time sequence in the T-frame point cloud data, a backward smoothing process is performed on the initial corrected traffic state of each traffic participant to obtain a target corrected traffic state of each traffic participant.

[0272] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0273] Based on the initial corrected traffic state x of traffic participants in the k-1th frame point cloud data k―1 , predict the predicted traffic state x of the traffic participants in the k-th frame point cloud data k|k―1 ; k is an integer greater than or equal to 1 and less than or equal to T;

[0274] Predicted traffic status x for traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k .

[0275] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0276] Based on the observed position z of the traffic participant in the k-th frame point cloud data k , and predicted traffic status x k|k―1 , and obtain the residual ε corresponding to each traffic participant in the k-th frame point cloud data k ;

[0277] Based on the residual ε corresponding to the traffic participants in the k-th frame point cloud data k , the predicted traffic state x of traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the kth frame k .

[0278] In a specific example of the disclosed solution, the information acquisition unit is specifically configured to:

[0279] Correct the traffic state x according to the target of the traffic participant in the k+1 frame point cloud data k+1|TAnd the predicted traffic state x in the k+1 frame point cloud data k+1|k , the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k Correction is performed to obtain the target corrected traffic state x of the traffic participants in the k-th frame point cloud data k|T ; Wherein, k is an integer less than T-1.

[0280] In a specific example of the disclosed solution, when k is T-1, the target modified traffic state x of the traffic participant in the T-th frame point cloud data is T|T is the initial corrected traffic state x T .

[0281] In a specific example of the disclosed solution, different preset constraint factors have different constraint targets, so that the state value of the state variable can be constrained from different dimensions using different preset constraint factors.

[0282] In a specific example of the disclosed solution, the target factor graph includes at least two of the following:

[0283] The first factor graph is a factor graph constructed by using the position of the traffic participant as a state variable to prevent the center position of the traffic participant from jumping between frames to a preset constraint factor;

[0284] The second factor graph is a factor graph constructed by taking the acceleration of the traffic participant between two adjacent frames as the state variable and controlling the traffic participant to perform target acceleration motion as the preset constraint factor;

[0285] The third factor graph is a factor graph constructed with the position of the traffic participant as the state variable and the matching of the speed and position jump degree of the traffic participant as the preset constraint factor;

[0286] The fourth factor graph is a factor graph constructed by taking the speed of the traffic participant between adjacent frames as the state variable and preventing the traffic participant from performing a preset acceleration sudden movement as the preset constraint factor;

[0287] The fifth factor graph is a factor graph constructed by using the position of traffic participants as the state variable to prevent the center position of stationary traffic participants from jumping between different frames to a preset constraint factor.

[0288] The sixth factor graph is a factor graph constructed with the position of the traffic participant as the state variable and the size change information of the detection frame corresponding to the traffic participant between different frames meeting the preset requirements as the preset constraint factor.

[0289] In a specific example of the disclosed solution, the preset constraint factor corresponding to the first factor graph is represented by a first constraint function;

[0290] The first constraint function can represent at least one of the following:

[0291] The difference between the actual position and the observed position of the traffic participant;

[0292] The covariance of the residuals between the actual and observed positions of traffic participants;

[0293] The actual position value can be obtained based on the initial traffic state of the traffic participant, and the position observation value can be obtained based on a preset observation device.

[0294] In a specific example of the present disclosure, the preset constraint factor corresponding to the second factor graph is represented by a second constraint function;

[0295] The second constraint function can represent at least one of the following:

[0296] The degree of difference between the estimated speed of the traffic participant in the t+1 frame point cloud data and the actual speed value of the t+1 frame point cloud data; the estimated speed of the t+1 frame point cloud data is based on the actual speed value and acceleration value of the t frame point cloud data;

[0297] The degree of difference between the estimated position of the traffic participant in the t+1th frame of point cloud data and the actual position of the t+1th frame of point cloud data; where the estimated position of the t+1th frame of point cloud data is based on the actual speed, position, and acceleration values ​​of the tth frame of point cloud data;

[0298] The actual speed value, the actual position value, and the actual acceleration value can all be obtained based on the initial traffic state.

[0299] In a specific example of the disclosed solution, the preset constraint factor corresponding to the third factor graph is represented by a third constraint function;

[0300] The third constraint function can represent: the degree of difference between the vector modulus of the speed of the traffic participant in the t-th frame point cloud data and the displacement change rate corresponding to the t-th frame point cloud data;

[0301] The displacement change rate corresponding to the t-th frame point cloud data is obtained based on the actual position value of the t-th frame point cloud data and the actual position value of the t-1-th frame point cloud data; the actual position value can be obtained based on the initial traffic state.

[0302] In a specific example of the disclosed solution, the preset constraint factor corresponding to the fourth factor graph is represented by a fourth constraint function;

[0303] The fourth constraint function can represent: the degree of difference between the position difference corresponding to the point cloud data of the t+1th frame and the position difference corresponding to the point cloud data of the tth frame;

[0304] Among them, the position difference corresponding to the t+1th frame point cloud data represents the position difference between the actual position value of the t+1th frame point cloud data and the actual position value of the tth frame point cloud data, and the position difference corresponding to the tth frame point cloud data represents the position difference between the actual position value of the tth frame point cloud data and the actual position value of the t-1th frame point cloud data.

[0305] In a specific example of the disclosed solution, the preset constraint factor corresponding to the fifth factor graph is represented by a fifth constraint function;

[0306] The fifth constraint function may represent: when the traffic participant is determined to be a static target, the vector modulus of the velocity and / or the vector modulus of the acceleration of the traffic participant.

[0307] In a specific example of the disclosed solution, the preset constraint factor corresponding to the sixth factor graph is represented by a sixth constraint function;

[0308] The sixth constraint function can represent:

[0309] The degree of difference between the size of the detection box where the traffic participant is located and the preset size;

[0310] Among them, the size of the detection frame where the traffic participant is located can be obtained through the actual position value of the traffic participant; the actual position value can be obtained based on the initial traffic state; the preset size is obtained based on the size of the detection frame where the traffic participant is located in different frames.

[0311] For the description of specific functions and examples of each unit of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0312] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0313] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, an autonomous driving vehicle including the electronic device, a readable storage medium, and a computer program product.

[0314] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0315] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0316] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0317] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for determining the traffic state of a traffic participant. For example, in some embodiments, the method for determining the traffic state of a traffic participant can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method for determining the traffic state of a traffic participant described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the method for determining the traffic state of a traffic participant in any other appropriate manner (eg, by means of firmware).

[0318] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0319] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0320] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0321] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0322] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0323] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0324] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0325] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for determining a traffic state of a traffic participant, comprising: Acquiring initial state information, wherein the initial state information can at least represent an initial traffic state of each of a plurality of traffic participants included in each frame of point cloud data in T frames, the initial traffic state of the traffic participant including a plurality of state values, each state value representing a value of a traffic parameter; and T being an integer greater than 1; Determining a target factor graph; wherein the target factor graph is capable of constraining a state value of a state variable based on each of a plurality of preset constraint factors; the state variable is determined from the plurality of traffic parameters based on a constraint target to be achieved by the preset constraint factors; The target traffic state of each traffic participant among the multiple traffic participants is obtained by using the target factor graph.

2. The method according to claim 1, further comprising: A target trajectory of each traffic participant is obtained based on the target traffic state of each traffic participant among the multiple traffic participants.

3. The method according to claim 1, wherein obtaining initial state information comprises: Inputting the target multimodal spatiotemporal data into an initial prediction model to obtain a model output result; the model output result at least includes the initial traffic state of each traffic participant; The target multimodal spatiotemporal data includes the T-frame point cloud data corresponding to the historical period; The model output result is post-processed to obtain the initial state information.

4. The method according to claim 3, further comprising: Based on the initial traffic state of each traffic participant and the target traffic state of each traffic participant, the state difference information of each traffic participant is obtained; The initial prediction model is trained using the status difference information of each traffic participant to obtain a target prediction model.

5. The method according to claim 3 or 4, wherein: The post-processing of the model output result to obtain the initial state information includes: Using the output of the model, traffic participants between point cloud data of different frames are associated to obtain an initial association result; performing trajectory filtering on the initial traffic state of each traffic participant in the output result of the model to obtain a target modified traffic state of each traffic participant; The traffic state is corrected based on the target of each traffic participant, and the initial association result is fine-tuned to obtain the target association result, so as to obtain the initial state information including the target association result.

6. The method according to claim 5, wherein: The performing trajectory filtering on the initial traffic state of each traffic participant in the output result of the model to obtain the target corrected traffic state of each traffic participant includes: Based on the time sequence in the T-frame point cloud data, forward filtering is performed on the initial traffic state of each traffic participant in the output result of the model to obtain the initial corrected traffic state of each traffic participant; Based on the time sequence in the T-frame point cloud data, a backward smoothing process is performed on the initial corrected traffic state of each traffic participant to obtain the target corrected traffic state of each traffic participant.

7. The method according to claim 6, wherein: The forward filtering process is performed on the initial traffic state of each traffic participant in the output result of the model to obtain the initial corrected traffic state of each traffic participant, including: Based on the initial corrected traffic state x of traffic participants in the k-1th frame point cloud data k―1 , predict the predicted traffic state x of the traffic participants in the k-th frame point cloud data k|k―1 ; k is an integer greater than or equal to 1 and less than or equal to T; Predicted traffic status x for traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k .

8. The method according to claim 7, wherein: The predicted traffic state x of the traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k ,include: Based on the observed position z of the traffic participant in the k-th frame point cloud data k , and predicted traffic status x k|k―1 , and obtain the residual ε corresponding to each traffic participant in the k-th frame point cloud data k ; Based on the residual ε corresponding to the traffic participants in the k-th frame point cloud data k , the predicted traffic state x of traffic participants k|k―1 Correction is performed to obtain the initial corrected traffic state x of the traffic participants in the kth frame k .

9. The method according to any one of claims 6 to 8, wherein: The method of performing backward smoothing on the initial corrected traffic state of each traffic participant based on the time sequence in the T-frame point cloud data to obtain the target corrected traffic state of each traffic participant includes: Correct the traffic state x according to the target of the traffic participant in the k+1 frame point cloud data k+1|T And the predicted traffic state x in the k+1 frame point cloud data k+1|k , the initial corrected traffic state x of the traffic participants in the k-th frame point cloud data k Correction is performed to obtain the target corrected traffic state x of the traffic participants in the k-th frame point cloud data k|T ; Wherein, k is an integer less than T-1.

10. The method according to claim 9, wherein: When k is T-1, the target corrected traffic state x of the traffic participant in the T-th frame point cloud data is T|T is the initial corrected traffic state x T .

11. The method according to any one of claims 1 to 10, wherein: Different preset constraint factors have different constraint targets, so that the state values ​​of the state variables can be constrained from different dimensions using different preset constraint factors.

12. The method according to claim 11, wherein The target factor graph includes at least two of the following: The first factor graph is a factor graph constructed by using the position of the traffic participant as a state variable to prevent the center position of the traffic participant from jumping between frames to a preset constraint factor; The second factor graph is a factor graph constructed by taking the acceleration of the traffic participant between two adjacent frames as the state variable and controlling the traffic participant to perform target acceleration motion as the preset constraint factor; The third factor graph is a factor graph constructed with the position of the traffic participant as the state variable and the matching of the speed and position jump degree of the traffic participant as the preset constraint factor; The fourth factor graph is a factor graph constructed by taking the speed of the traffic participant between adjacent frames as the state variable and preventing the traffic participant from performing a preset acceleration sudden movement as the preset constraint factor; The fifth factor graph is a factor graph constructed by using the position of traffic participants as the state variable to prevent the center position of stationary traffic participants from jumping between different frames to a preset constraint factor; The sixth factor graph is a factor graph constructed with the position of the traffic participant as the state variable and the size change information of the detection frame corresponding to the traffic participant between different frames meeting the preset requirements as the preset constraint factor.

13. The method according to claim 12, wherein: The preset constraint factor corresponding to the first factor graph is represented by a first constraint function; The first constraint function can represent at least one of the following: The difference between the actual position and the observed position of the traffic participant; The covariance of the residuals between the actual and observed positions of traffic participants; The actual position value can be obtained based on the initial traffic state of the traffic participant, and the position observation value can be obtained based on a preset observation device.

14. The method according to claim 12, wherein: The preset constraint factor corresponding to the second factor graph is represented by a second constraint function; The second constraint function can represent at least one of the following: The degree of difference between the estimated speed of the traffic participant in the t+1 frame point cloud data and the actual speed value of the t+1 frame point cloud data; the estimated speed of the t+1 frame point cloud data is based on the actual speed value and acceleration value of the t frame point cloud data; The degree of difference between the estimated position of the traffic participant in the t+1th frame of point cloud data and the actual position of the t+1th frame of point cloud data; where the estimated position of the t+1th frame of point cloud data is based on the actual speed, position, and acceleration values ​​of the tth frame of point cloud data; The actual speed value, the actual position value, and the actual acceleration value can all be obtained based on the initial traffic state.

15. The method according to claim 12, wherein: The preset constraint factor corresponding to the third factor graph is represented by a third constraint function; The third constraint function can represent: the degree of difference between the vector modulus of the speed of the traffic participant in the t-th frame point cloud data and the displacement change rate corresponding to the t-th frame point cloud data; The displacement change rate corresponding to the t-th frame point cloud data is obtained based on the actual position value of the t-th frame point cloud data and the actual position value of the t-1-th frame point cloud data; the actual position value can be obtained based on the initial traffic state.

16. The method according to claim 12, wherein The preset constraint factor corresponding to the fourth factor graph is represented by a fourth constraint function; The fourth constraint function can represent: the degree of difference between the position difference corresponding to the point cloud data of the t+1th frame and the position difference corresponding to the point cloud data of the tth frame; Among them, the position difference corresponding to the t+1th frame point cloud data represents the position difference between the actual position value of the t+1th frame point cloud data and the actual position value of the tth frame point cloud data, and the position difference corresponding to the tth frame point cloud data represents the position difference between the actual position value of the tth frame point cloud data and the actual position value of the t-1th frame point cloud data.

17. The method according to claim 12, wherein: The preset constraint factor corresponding to the fifth factor graph is represented by a fifth constraint function; The fifth constraint function may represent: when the traffic participant is determined to be a static target, the vector modulus of the velocity and / or the vector modulus of the acceleration of the traffic participant.

18. The method according to claim 12, wherein: The preset constraint factor corresponding to the sixth factor graph is represented by a sixth constraint function; The sixth constraint function can represent: The degree of difference between the size of the detection box where the traffic participant is located and the preset size; Among them, the size of the detection frame where the traffic participant is located can be obtained through the actual position value of the traffic participant; the actual position value can be obtained based on the initial traffic state; the preset size is obtained based on the size of the detection frame where the traffic participant is located in different frames.

19. A device for determining a traffic state of a traffic participant, comprising: an information acquisition unit, configured to acquire initial state information, wherein the initial state information can at least represent an initial traffic state of each of a plurality of traffic participants included in each frame of point cloud data in T frames, the initial traffic state of the traffic participants including a plurality of state values, each state value representing a value of a traffic parameter; and T being an integer greater than 1; A factor graph determining unit, configured to determine a target factor graph; wherein the target factor graph is capable of constraining a state value of a state variable based on each of a plurality of preset constraint factors; and the state variable is determined from the plurality of traffic parameters based on a constraint target to be achieved by the preset constraint factors; The state optimization unit is used to obtain the target traffic state of each traffic participant in the multiple traffic participants by using the target factor graph.

20. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 18.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-18.

22. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 18.

23. An autonomous driving vehicle comprising the electronic device according to claim 20.

Images