A method for generating potential unsafe scenarios in automotive dynamic interaction testing based on POMDP

Abstract

This invention relates to a method for generating potential unsafe scenarios in vehicle dynamic interaction testing based on POMDP, belonging to the field of intelligent connected vehicle testing technology. The method includes constructing a natural driving model based on a Copula function to achieve joint distribution modeling of multi-dimensional state variables, ensuring that the evolution of scenario behavior conforms to the statistical laws of real driving. Then, under the realism constraints of the natural driving model, an adversarial search mechanism based on partially observable Markov decision processes is introduced. By simulating an information interference environment and introducing risk-driven rewards, the scenario is pushed towards the system functional boundary, causing low-probability, high-risk long-tail interaction processes to occur more frequently. This invention constructs a complete methodological chain for proactively detecting potential unsafe scenarios from two levels: ensuring realism and enhancing risk exposure capabilities.

Description

Technical Field

[0001] This invention belongs to the field of intelligent connected vehicle testing technology, and relates to a method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on Partially Observable Markov Decision Process (POMDP). Background Technology

[0002] Against the backdrop of the rapid development of intelligent connected vehicles, Intelligent Vehicle Cyber-Physical Systems (IVCPS) are becoming a key technology for solving future transportation system problems. High-coverage, high-reliability scenario construction is a crucial technical support for scenario-based testing. A deeper challenge facing the Safety of the Intended Functionality (SOTIF) verification system lies in the fact that the potential hazards arising from continuous interactions among multiple stakeholders in expressway merging zones are often outside the scope of any pre-analytical framework. The danger of these scenarios does not stem from extreme values ​​of a single factor, but from the coupled behaviors of multiple traffic participants in continuous time evolution—the superposition of factors such as subtle changes in speed differences and brief disturbances in communication delays at specific time sequences can push a seemingly normal interaction process to the system's functional boundaries. Since such events occur with extremely low probability in natural driving data, directly discretizing continuous variables and performing Cartesian product combinations, while rapidly constructing a large number of scenarios, often leads to generated samples deviating from the real driving distribution, resulting in unnatural trajectories, abrupt behavioral changes, and other problems, thus weakening the realism of the test scenarios. Furthermore, for IVCPS, multi-scale scenarios and multi-agent interaction and collaboration are the core content of testing. If we follow the traditional method of only considering the combination of a single element and time, we may easily overlook the interaction and influence between different agents, thereby weakening the scenario's ability to represent risky behavior and losing its testing value.

[0003] Therefore, it is necessary to dynamically extend the static and parameterized descriptions of the dynamic entity layer and the digital information layer. The key challenge in constructing dynamic scenarios lies in maintaining the unbiasedness and naturalness of driving behavior while also focusing on the ability to proactively induce risks, thereby generating dynamic interactive scenarios that combine realism and testing value, and enhancing the effectiveness of the generated test scenarios. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for generating potential unsafe scenarios in automotive dynamic interaction testing based on POMDP, which constructs a complete method chain for proactive detection of potential unsafe scenarios from two levels: ensuring authenticity and enhancing risk exposure capabilities.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for generating potential unsafe scenarios in vehicle dynamic interaction testing based on POMDP is proposed. This method includes a natural driving model construction process based on Copula functions and an adversarial dynamic element scenario generation process based on POMDP. In the natural driving model based on the Copula function, for the dynamic interaction scenario of the merging zone of the expressway, the constituent units of the dynamic scenario are uniformly described, so that the random variables in the Copula model have clear physical orientation; at the same time, the vehicle interaction stage is determined by combining the road spatial structure and vehicle state, and the continuous state variables involved in the vehicle interaction process are regarded as random vectors, and their statistical dependency structure is modeled. The natural driving trajectory is gradually generated through Monte Carlo sampling. In the process of generating adversarial dynamic element scenarios based on POMDP, an adversarial dynamic element scenario construction mechanism is introduced on the basis of the distribution constraints of the natural driving model. The scenario generation process is described as a risk-induced decision problem of the test vehicle under incomplete information conditions. The risk-induced strategy is obtained through adversarial strategy iteration and a set of dynamic interactive scenario trajectories with high risk exposure capability are generated accordingly.

[0006] Furthermore, in the natural driving model based on Copula functions, let the first... Each traffic participant at time The state vector is Control or behavioral variables are Then its instantaneous interaction unit is defined as:

[0007] Let the time window be Then by A dynamic scene segment consisting of multiple traffic participants is represented as follows:

[0008] in, Describe the interactive evolution process of all subjects within a given time window; In the test environment of the merging zone of an expressway, the focus is on the game relationship between the test vehicle and adjacent background vehicles, while also considering road set and traffic flow. Let the test vehicle be denoted as... e The target background car is recorded as b Then construct the scene layer state vector:

[0009] in, Indicates the relative distance in the longitudinal direction. Indicates the relative horizontal distance. and These represent the longitudinal and lateral relative velocity differences, respectively. , This indicates the speed of the test vehicle relative to the adjacent background vehicle. , This represents the longitudinal acceleration of the corresponding vehicle.

[0010] Furthermore, in the natural driving model based on Copula functions, for any vehicle Its state vector is defined as:

[0011] in, , These are the position coordinates in the global coordinate system. For velocity scalar, For heading angle, For longitudinal acceleration, This refers to the yaw rate; For any vehicle Its action vector is defined as:

[0012] in, Indicates longitudinal control input. This indicates input for lateral steering or attitude adjustment; Based on the vehicle's planar kinematics, the state update process is represented as follows:

[0013]

[0014]

[0015]

[0016] With a fixed sampling frequency, the longitudinal acceleration and yaw rate of the vehicle are estimated using discrete difference, i.e.:

[0017]

[0018] in, That is, the sampling time interval.

[0019] Furthermore, in the natural driving model based on the Copula function, let the reference coordinates of the starting point of the merging zone along the main line be... The test vehicle's current location is The longitudinal distance of the test vehicle relative to the merging point is expressed as:

[0020] Meanwhile, let the lateral offset of the test vehicle relative to the centerline of the target lane be... and with Indicates the lane keeping error threshold, in The threshold for determining the speed difference is used to divide the interaction phase into:

[0021] in, and The longitudinal segmentation threshold is set based on road geometry and statistical data characteristics to meet the following requirements. Steady-state following The stage refers to the period before the vehicle enters the actual merging preparation area; merging is executed. The stage indicates that the vehicle has exhibited significant lateral movement during lane change; conflict avoidance. The phase corresponds to the risk response phase triggered by a sharp compression of the gap, a sudden change in the velocity difference, or a rapid decrease in the collision time; Based on this, the feature vector used for joint distribution modeling is defined as

[0022] in, For defined continuous variables, This is a label for the interaction stage.

[0023] Furthermore, in the natural driving model based on the Copula function, let the feature random vector extracted during the merging zone interaction be:

[0024] in, Indicates the first The dynamic variables include relative longitudinal spacing, relative lateral offset, main vehicle speed, background vehicle speed, longitudinal acceleration, lateral speed, and interaction stage indicators; Let the marginal distribution functions of each component be denoted as . According to Sklar's theorem, the joint distribution of the random vector z can be expressed as:

[0025] In the formula, The parameter is Copula functions; If each marginal distribution has a probability density function The joint probability density function is then expressed as:

[0026] in, ( · ) is the Copula density function; We use Gaussian Copula to model the variable dependency structure, let ,in, Let be the inverse distribution function of the standard normal distribution, then the Gaussian Copula is expressed as:

[0027] in, The correlation matrix is ​​represented as R The multivariate normal distribution function; Correspondingly, its density is expressed as:

[0028] in, , It is an identity matrix.

[0029] Furthermore, in the parameter estimation stage of the natural driving model based on the Copula function, the original samples are first subjected to an empirical distribution transformation: Let the first... The observed samples are The total sample size is Then, a pseudo-observation is defined as:

[0030] Based on this, mapping it to Gaussian space yields:

[0031] Then transform the joint distribution model into a conditional sampling form; let... For a moment t State feature vector, Given the feature vector from the previous time step, the natural driving state transition can be abstractly represented as:

[0032] Introducing interaction mode variables ,in For the set of interaction phases, given an initial state Then, natural driving trajectories are gradually generated through Monte Carlo sampling, namely:

[0033] And by combining the time-series update mechanism, a complete dynamic interaction sequence is obtained.

[0034] Furthermore, in the process of generating adversarial dynamic element scenarios based on POMDP, the partially observable Markov decision process (POMDP) ​​is used to model the process, which is represented as follows:

[0035] in, Represents the state space of the real environment. Represents the action space. Represents the observation space. Represents the state transition function. Represents the observation probability function. Represents the reward function, Indicates the discount factor; While maintaining consistency with the state parameters in the natural driving model, the real-world environment state in POMDP is defined as follows:

[0036] in, , , and These represent the relative position difference and relative speed difference between the test vehicle and the target vehicle, respectively. , This represents the longitudinal acceleration of the test vehicle and the background vehicle. This indicates the yaw rate of the test vehicle. Indicates the interaction stage label; Indicates communication delay. Indicates perceived confidence level. Indicates the status of the sensing or communication link.

[0037] Furthermore, when the actual state cannot be directly observed, observation mode variables are defined. And the observation vector is represented as:

[0038] in, () represents the mapping function related to the observation mode. This is random observation noise; Indicates the delay step size. This represents an observation operator that masks or maps out some state variables; Based on the aforementioned settings, the longitudinal and lateral control variables of the test vehicle are discretized, and the motion space is constructed using a combination of the two, namely:

[0039] The action at any given moment is represented as:

[0040] in, and These represent the test vehicle at time [time]. Longitudinal and lateral control inputs; In terms of state transition modeling, an adversarial perturbation term is introduced based on the natural driving model, resulting in:

[0041] in, This represents the state evolution function constrained by the Copula natural driving distribution. The state transition probability is represented by the adversarial disturbance term formed by the random response of the background vehicle, the anomaly in perception communication, and the disturbance in the local environment.

[0042] Regarding the reward function, the single-step reward is designed as follows:

[0043] in, , , These are non-negative weighting coefficients, which respectively control the relative importance of risk approximation, terminal collision, and action smoothing. This is a risk item. For explicit collision rewards, To control the smoothing penalty term.

[0044] Furthermore, in the designed reward function, the risk term is defined in TTC form as follows:

[0045] in, This is a scale parameter; this allows the test vehicle to continue receiving feedback as it approaches the high-risk boundary even if a collision does not occur immediately. Explicit collision rewards are triggered only at the terminal moment and are defined as follows:

[0046] This feature enhances the ability to search for high-risk endgame scenarios; The control smoothing penalty term is expressed as:

[0047] in, This is a weight for horizontal control changes; this item can suppress abrupt changes in action between adjacent time steps.

[0048] Furthermore, since the agent cannot directly obtain the true state, it uses short-term historical observations to stack the data to obtain the true state. To approximate it, that is:

[0049] in, L Indicates the length of the historical observation window; intelligent agents at all times t based on The strategy for selecting an action is represented as follows:

[0050] in, For online Q network, For network parameters; In DDQN, the objective Q-value is determined by the online network selecting actions and the objective network evaluating those actions. The updated objective is represented as follows:

[0051] in, Indicates the target network parameters; Based on this, the loss function is defined as:

[0052] By minimizing this loss function, the online network parameters are continuously updated, enabling it to gradually learn the long-term risk-reward under different observation conditions and action combinations. In the training process, the online network, target network, and experience replay pool are first initialized, and the exploration rate is set. Discount Factor Synchronize with the target network cycle; subsequently, in each round of scene generation, the agent... The mechanism selects a combination of lateral and longitudinal control from the action space and applies it to the adversarial environment to obtain the next moment's observation. Instant rewards and round termination flag; state transition sample After being stored in the experience replay pool, the online network parameters are updated using a small-batch random sampling method, and the online network weights are periodically copied to the target network.

[0053] The beneficial effects of this invention are as follows: 1. Regarding authenticity assurance, this invention establishes a multi-dimensional joint distribution model of natural driving state variables based on the Copula function. By introducing merging interaction stage labels, it accurately characterizes the dependency structure between parameters such as relative distance, speed difference, and acceleration. Experimental verification shows that the generated samples maintain a high degree of consistency with real driving data at multiple levels, including edge distribution and two-dimensional and three-dimensional joint distribution, providing statistical assurance for the effectiveness of subsequent adversarial scenarios.

[0054] 2. Regarding the enhancement of risk exposure capabilities, this invention designs an adversarial scenario generation scheme based on POMDP under Copula prior constraints. By incorporating information layer variables such as communication latency and perceived confidence into the state space, the model can perform adversarial search on a fully coupled cyber-physical link. Experimental results show that the incidence rate of dangerous events stabilizes at a high level in the later stages of training, and the generated samples significantly migrate to high-risk areas while maintaining the overall distribution pattern, verifying the effectiveness of the method in targeted compression of potentially unsafe areas in SOTIF.

[0055] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to an embodiment of the present invention. Figure 2 This is a schematic diagram of some observable Markov processes under an embodiment of the present invention; Figure 3 This is a schematic diagram comparing the marginal distribution data of the following behavior variable in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the marginal distribution data of the entry behavior variables in an embodiment of the present invention; Figure 5 This is a schematic diagram comparing the distribution data of the binary behavioral variable of following vehicles under an embodiment of the present invention, wherein, Figure 5 (a) shows the speed and distance distribution of the vehicle in front. Figure 5 (b) shows the speed and acceleration distribution of the vehicle in front; Figure 6 This is a schematic diagram comparing the distribution data of the binary behavioral variables in an embodiment of the present invention, wherein, Figure 6(a) shows the speed and distance distribution of the vehicle in front. Figure 6 (b) shows the speed and acceleration distribution of the vehicle in front; Figure 7 This is a schematic diagram comparing the distribution data of the ternary behavioral variables in an embodiment of the present invention; Figure 8 This is a schematic diagram comparing the distribution of decision-making actions under an embodiment of the present invention; Figure 9 This is a schematic diagram of the training process according to an embodiment of the present invention, wherein, Figure 9 (a) is a schematic diagram of the reward convergence. Figure 9 (b) is a diagram illustrating the events that occurred; Figure 10 This is a schematic diagram of the joint sampling distribution of the key ternary variables in an embodiment of the present invention. Detailed Implementation

[0057] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0058] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0059] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0060] Please see Figures 1-10This is a method for generating potential unsafe scenarios for automotive dynamic interaction testing based on POMDP.

[0061] This embodiment provides a specific implementation process for a dynamic interactive test scenario generation method based on POMDP, such as... Figure 1 As shown, it includes the following steps: S1. Construction of a natural driving model based on the Copula function: The Copula function is used to model the joint distribution relationship of multidimensional state variables in natural driving data to ensure that the behavioral evolution of the generated scene conforms to the statistical laws of real driving. To give the random variables in the Copula model a clear physical orientation, it is necessary to first uniformly describe the constituent units of the dynamic scenario. This invention treats the dynamic interaction scenario in the merging zone of an expressway as a temporal evolution system. Its basic object is not a single discrete instant, but rather the state-action coupling process of multiple traffic participants over continuous time. Let the... Each traffic participant at time The state vector is Control or behavioral variables are Then its instantaneous interaction unit is defined as:

[0062] If the research time window is Then by A dynamic scene segment consisting of multiple traffic participants can be represented as:

[0063] in, It describes the interactive evolution process of all agents within a given time window. This definition emphasizes that the essence of a scene is a joint expression of "multi-agent + continuous time + behavioral coupling", which is consistent with the cyber-physical synergy characteristics exhibited by IVCPS during operation.

[0064] For the test problem in the merging zone of expressways, this invention focuses on the game relationship between the test vehicle and adjacent background vehicles, while also taking into account road geometry and traffic flow constraints. Let the test vehicle be denoted as... e The target background car is recorded as b Then the scene layer state vector can be constructed:

[0065] in, Indicates the relative distance in the longitudinal direction. Indicates the relative horizontal distance. and These represent the longitudinal and lateral relative velocity differences, respectively. , Indicates the speed of the test vehicle and the background vehicle. , This represents the longitudinal acceleration of the corresponding vehicle. Compared to descriptions using only single-vehicle state parameters, this relative state expression is more suitable for characterizing the proximity of collisions and gap competition in the merging region, and is also more beneficial for measuring hazard trigger boundaries in subsequent expected functional safety analyses.

[0066] In this embodiment, considering the significant lateral and longitudinal coupling motion characteristics of vehicles in the merging zone, the present invention employs an extended parameterized representation of the single-vehicle state. For any vehicle... Its state vector is defined as:

[0067] in, , These are the position coordinates in the global coordinate system. For velocity scalar, For heading angle, For longitudinal acceleration, ω represents the yaw rate.

[0068] Correspondingly, the vehicle motion vector can be represented as:

[0069] in, Indicates longitudinal control input. This represents the input for lateral steering or attitude adjustment. By modeling state variables separately from action variables, the hierarchical relationship between observable environmental conditions and control decisions can be more clearly distinguished during subsequent POMDP modeling.

[0070] Based on the vehicle's planar kinematics, the state update process can be written as:

[0071]

[0072]

[0073]

[0074] If the sampling frequency is fixed, the longitudinal acceleration and yaw rate of the vehicle can be further estimated using discrete difference, i.e.:

[0075]

[0076] The above relationship ensures that the generated scene has basic motion continuity during the time progression, thereby avoiding the trajectory abrupt change problem caused by independent splicing of parameters.

[0077] To avoid the interactive stage labels remaining only at the level of empirical description, the road spatial structure of the expressway merging area should be considered. Perform rule-based extraction. Let the reference coordinates of the starting point of the merging zone along the main line be... The test vehicle's current location is Then the longitudinal distance of the test vehicle relative to the merging point can be expressed as: .

[0078] Meanwhile, let the lateral offset of the test vehicle relative to the centerline of the target lane be... and with Indicates the lane keeping error threshold, in This represents the threshold for determining the speed difference. Therefore, the interaction phase can be divided into...

[0079] in, and The longitudinal segmentation threshold is set based on road geometry and statistical data characteristics to meet the following requirements. Among them, steady-state following mainly corresponds to the stage where the vehicle has not yet entered the actual merging preparation area; executing merging indicates that the vehicle has already shown obvious lateral lane-changing movement; and conflict avoidance corresponds to the risk response stage triggered by a sharp compression of the distance, a sudden change in speed difference, or a rapid decrease in collision time.

[0080] Based on this, the feature vector used for joint distribution modeling is defined as

[0081] The first seven items are continuous variables, and the last item is the interaction stage label. For discrete variables... This invention employs a phased modeling approach, first dividing the samples according to the interaction phase, and then applying Copula modeling to the continuous variables within each phase. This approach preserves the phased differences in the confluence behavior while avoiding estimation biases caused by directly applying a uniform fit to the confluence variables.

[0082] Let the feature random vector extracted during the merging zone interaction be:

[0083] in, Indicates the first Each of the dynamic variables corresponds to the relative longitudinal spacing, relative lateral offset, main vehicle speed, background vehicle speed, longitudinal acceleration, lateral speed, and interaction stage indicators, etc.

[0084] Let the marginal distribution functions of each component be denoted as . According to Sklar's theorem, the joint distribution of the random vector z can be expressed as:

[0085] In the formula, The parameter is The Copula function. The significance of this expression lies in decoupling the two questions of how each variable follows its own distribution and how the variables are related, thus making it easier to characterize the dependency structure in vehicle interaction behavior without destroying marginal statistical features.

[0086] In this embodiment, if each edge distribution has a probability density function Then the joint probability density function can be written as:

[0087] in, ( · Let be the Copula density function. It can be seen that the joint probability density of multidimensional variables consists of two parts: one part comes from the marginal probability densities of each variable, and the other part comes from the Copula density kernel describing the correlation. For strong interaction scenarios such as merging zones on expressways, this form can more naturally preserve statistical correlation characteristics such as the decrease in speed difference accompanied by spacing compression, and the enhancement of merging intention accompanied by changes in lateral displacement.

[0088] Gaussian Copula models the structure of variable dependencies. Let:

[0089] in, Let be the inverse distribution function of the standard normal distribution, then the Gaussian Copula can be expressed as:

[0090] in, The correlation matrix is ​​represented as R The multivariate normal distribution function.

[0091] Correspondingly, its density can be written in the form of:

[0092] in, , The above form transforms the joint distribution modeling problem into two sub-problems: marginal distribution identification and correlation matrix estimation, facilitating a step-by-step solution based on real driving data.

[0093] In the parameter estimation stage, the original sample is first subjected to an empirical distribution transformation. Let the first... The observed samples are The total sample size is Then, a pseudo-observation can be defined as:

[0094] Based on this, mapping it to Gaussian space yields:

[0095] To support the recursive generation of natural driving sequences, this invention further transforms the joint distribution model into a conditional sampling form. For a moment t State feature vector, If the feature vector is from the previous time step, then the natural driving state transition can be abstractly represented as:

[0096] If we further introduce interaction mode variables ,in This represents a set of patterns such as carousel and forced merge, ultimately, given an initial state. Then, natural driving trajectories are gradually generated through Monte Carlo sampling, namely:

[0097] Furthermore, a complete dynamic interaction sequence is obtained by combining a temporal update mechanism. The natural driving model constructed based on the above method can, on the one hand, maintain the joint distribution characteristics in real data, and on the other hand, provide a controllable initial behavioral prior for the subsequent generation of POMDP adversarial scenarios, thereby providing a reliable baseline for risk scenario search in IVCPS expected functional safety testing.

[0098] S2. Construction of adversarial dynamic element scenarios based on POMDP: Under the constraint of realism, the POMDP adversarial search mechanism is introduced. By simulating the information interference environment and introducing risk-driven rewards, the scenario is pushed to the boundary of system function, so that the low-probability, high-risk long-tail interaction process appears more frequently. The problem is modeled using POMDP, and the principle is as follows: Figure 2 As shown. The scene generation model is defined as follows:

[0099] in, Represents the state space of the real environment. Represents the action space. Represents the observation space. Represents the state transition function. Represents the observation probability function. Represents the reward function, This represents the discount factor. The core objective of this model is to proactively search for high-risk interaction trajectories that are more likely to expose system vulnerabilities, while maintaining the statistical accuracy of the scenarios.

[0100] To maintain consistency with the aforementioned natural driving state parameters, this invention defines the real-world environmental state in POMDP as follows:

[0101] in, , , and These represent the relative position difference and relative speed difference between the test vehicle and the target vehicle, respectively. , This represents the longitudinal acceleration of the test vehicle and the background vehicle. This indicates the yaw rate of the test vehicle. Indicates the interaction stage label; Indicates communication delay. Indicates perceived confidence level. This represents the state identifier of the sensing or communication link. By incorporating information layer variables into the state space, the model can more accurately reflect the coupling relationship between "vehicle motion state - sensing and communication quality - risk evolution outcome" in the IVCPS in the merging zone of expressways.

[0102] Since the system cannot directly observe the true state, this invention further defines observation mode variables. The observation vector is represented as:

[0103] in, () represents the mapping function related to the observation mode. This represents random observation noise. Furthermore, the observation function can be written according to different information anomaly forms as follows:

[0104] in, Indicates the delay step size. This represents an observation operator that masks or maps out certain state variables. Therefore, the test vehicle makes decisions based on observation information affected by noise, delay, and missing data, rather than an idealized full-state input.

[0105] Based on the aforementioned settings, the longitudinal and lateral control variables of the test vehicle are discretized, and the motion space is constructed using a combination of the two, namely:

[0106] The action at any given moment is represented as:

[0107] in, and These represent the test vehicle at time [time]. The longitudinal and lateral control inputs are used. The discrete combination form can not only retain the longitudinal and lateral coupling characteristics in the merging decision, but also provide a finite action space basis for subsequent deep reinforcement learning solutions.

[0108] In terms of state transition modeling, this invention does not directly adopt a completely manually set dynamic recursion, but introduces an adversarial perturbation term based on the natural driving model, thereby obtaining:

[0109] in, This represents the state evolution function constrained by the Copula natural driving distribution. This represents the adversarial disturbance term formed by the random response of the background vehicle, abnormal sensing communication, and local environmental disturbances.

[0110] Equivalently, the state transition probability can be expressed as:

[0111] This modeling approach ensures that the generated scenarios do not completely deviate from the statistical patterns of real driving, while also leaving a controllable disturbance space for proactively searching for edge-risk behaviors.

[0112] The reward function is crucial for adversarial scenario generation. To simultaneously consider risk exposure capability, action feasibility, and scenario naturalness, this invention designs the single-step reward as follows:

[0113] in, , , These are non-negative weighting coefficients, controlling the relative importance of risk approximation, terminal collision, and action smoothing, respectively. Specifically, the risk term is defined in TTC form:

[0114] in, This is a scale parameter. This parameter allows the test vehicle to continue receiving feedback as it approaches the high-risk boundary even if a collision does not occur immediately.

[0115] Explicit collision rewards are triggered only at the terminal moment and can be defined as:

[0116] This feature enhances the search capability for high-risk endings, but does not continuously induce the model to use collisions as the sole objective throughout the trajectory generation process.

[0117] To prevent the agent from learning control sequences that frequently jitter and clearly do not conform to vehicle dynamics constraints, this invention further introduces a control smoothing penalty term:

[0118] in, This is the weight for lateral control changes. This term can suppress excessively large abrupt changes in action between adjacent time steps, thereby ensuring that the generated trajectory has basic continuity in dynamics.

[0119] Considering that agents cannot directly obtain the true state under POMDP conditions This invention uses a stack of short-term historical observations to approximate the belief state, that is:

[0120] in, L This indicates the length of the historical observation window. The agent at time [time value missing]. t based on The strategy for selecting an action can be represented as follows:

[0121] in, For online Q network, For network parameters. This processing method can partially recover the hidden environment state using observation time series information without explicitly constructing the belief state update equation, thereby enhancing the decision stability under time delay, noise, and missing conditions.

[0122] In DDQN, the objective Q-value is determined by the online network selecting actions and the objective network evaluating those actions. The updated objective can be written as:

[0123] in, This represents the target network parameters. Based on this, the loss function is defined as:

[0124] By minimizing this loss function, the online network parameters can be continuously updated, allowing it to gradually learn the long-term risk-reward ratio under different observation conditions and action combinations.

[0125] In the training process, the online network, target network, and experience replay pool are first initialized, and the exploration rate is set. Discount Factor Synchronize with the target network cycle. Subsequently, during each round of scene generation, the agent... The mechanism selects a combination of lateral and longitudinal control from the action space and applies it to the adversarial environment to obtain the next moment's observation. Instant rewards and round termination flag. State transition sample. After being stored in the experience replay pool, the system updates the online network parameters using a mini-batch random sampling method and periodically copies the online network weights to the target network. This mechanism can weaken the correlation between samples and improve the stability of the training process.

[0126] like Figure 3 The comparison results of the edge distributions shown indicate that the probability distributions of the generated and real scenarios for key variables such as the speed of the vehicle in front, the speed of the vehicle behind, the following distance, and the acceleration of the vehicle behind are highly consistent. The speed distributions of both the vehicle in front and the vehicle behind exhibit a relatively smooth, single-peaked shape. The generated samples are largely consistent with the real samples within the main probability mass region, with only a slight deviation in the high-speed tail section. The following distance distribution also well reproduces the characteristics of the real samples, where small and medium distances dominate and the probability of long distances decays rapidly. The acceleration distribution of the vehicle behind forms a clear peak near zero, and the diffusion ranges of the two types of samples are quite similar on both the acceleration and deceleration sides. This demonstrates that the constructed model can well reproduce the edge statistical characteristics of real driving data at the univariate level, without significant mean drift or distortion of dispersion.

[0127] like Figure 4 As shown in the data comparison results, the joint distribution of variables for virtual scene entry behavior and real scene entry behavior is basically consistent.

[0128] At the level of two-dimensional joint distribution, the generated scene also maintains the correlation structure between variables quite well, such as Figure 5 , Figure 6 As shown, the joint scatter distribution of the speed of the vehicle in front and the following distance reveals a significant positive correlation between the two types of samples. The central region, diffusion direction, and density distribution of the generated sample point cloud are highly consistent with those of the real samples, indicating that the model can effectively reproduce the coupling relationship between speed and distance. Regarding the joint distribution of the speed and acceleration of the vehicle in front, both the real and generated samples are concentrated in regions where acceleration is close to zero, and exhibit high density within the moderate speed range. This demonstrates that the model not only maintains the marginal distribution of the vehicle in front's speed but also stably characterizes the correlated, non-independent statistical relationship between speed and acceleration.

[0129] From the three-dimensional joint distribution results, in the ternary state space composed of the speed of the preceding vehicle, the following distance, and the speed of the following vehicle, the generated scene sample point cloud is highly similar to the real scene sample point cloud in overall morphology. Both exhibit a banded distribution characteristic that gradually expands along the speed and spacing directions, indicating that the generated samples can well maintain the cooperative change trend among multiple variables, such as... Figure 7 As shown, compared to methods that fit only at the univariate level, this result further demonstrates that Copula joint distribution modeling has a significant advantage in preserving multidimensional dependency structures and can effectively avoid combinatorial distortion caused by independent parameter sampling.

[0130] In addition to the overall joint distribution, the unbiasedness of natural behavior models is also reflected in their decision-making, such as... Figure 8 As shown, conditional probability-based slicing was performed according to the ranges of the remaining three parameters. Specifically, the distribution of rear vehicle acceleration was grouped and compared under different distance, preceding vehicle speed, and following vehicle speed ranges. The results show that, under most conditional combinations, the peak position, distribution width, and skewness trend of the generated samples and the real samples are basically consistent, with only slight deviations in a few sparse sample intervals. This indicates that the constructed natural driving model can not only maintain the consistency of data distribution overall, but also reproduce the conditional statistical regularity of the following vehicle's behavioral response well under given interaction conditions, thus verifying its ability to describe local behavioral mechanisms.

[0131] like Figure 9 The training curves shown indicate that in the initial stage, due to the agent's random exploration and strategy trial phase, both the cumulative reward and the event occurrence rate fluctuate. As the training rounds increase, the reward value generally shows a continuous upward trend, gradually stabilizing in the later stages, indicating that the agent has learned relatively stable high-risk induced strategies under the constraints of natural driving distribution. Corresponding to the reward trend, the dangerous event occurrence rate also gradually increases from a low level in the early training stage, stabilizing at a high level in the later stages, indicating that the learned strategy can continuously push the scenario towards a higher-risk interactive state. Although there are slight differences in the initial local fluctuation amplitude among the different training curves, they all show a consistent upward trend, indicating that the adversarial search mechanism constructed in this paper has good training stability.

[0132] A comparative analysis was conducted on the distribution characteristics of real samples and adversarial generated samples in the key ternary variable space, such as... Figure 10As shown, the three-dimensional joint distribution of the speed of the preceding vehicle, the following distance, and the speed of the following vehicle reveals that the generated samples generally follow the main distribution manifold of the real samples, indicating that the adversarial scenario has not deviated from the basic behavioral patterns of the merging zone on expressways. At the same time, the generated samples exhibit more pronounced diffusion and aggregation characteristics in some high-risk areas, especially in areas where vehicle spacing is smaller and the speed coupling between the preceding and following vehicles is tighter, resulting in a relatively higher sample density. This suggests that the adversarial strategy does not simply replicate real driving data, but rather, while maintaining statistical accuracy, actively propels the scenario towards states more prone to conflict and dangerous approach events.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for generating potential unsafe scenarios for automotive dynamic interaction testing based on POMDP, characterized in that: The method includes a natural driving model construction process based on Copula functions and an adversarial dynamic element scene generation process based on POMDP, wherein, In the natural driving model based on the Copula function, for the dynamic interaction scenario of the merging zone of the expressway, the constituent units of the dynamic scenario are uniformly described, so that the random variables in the Copula model have clear physical orientation; at the same time, the vehicle interaction stage is determined by combining the road spatial structure and vehicle state, and the continuous state variables involved in the vehicle interaction process are regarded as random vectors, and their statistical dependency structure is modeled. The natural driving trajectory is gradually generated through Monte Carlo sampling. In the process of generating adversarial dynamic element scenarios based on POMDP, an adversarial dynamic element scenario construction mechanism is introduced on the basis of the distribution constraints of the natural driving model. The scenario generation process is described as a risk-induced decision problem of the test vehicle under incomplete information conditions. The risk-induced strategy is obtained through adversarial strategy iteration and a set of dynamic interactive scenario trajectories with high risk exposure capability are generated accordingly.

2. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 1, characterized in that: In the natural driving model based on Copula functions, let the first... Each traffic participant at time The state vector is Control or behavioral variables are Then its instantaneous interaction unit is defined as: Let the time window be Then by A dynamic scene segment consisting of multiple traffic participants is represented as follows: in, Describe the interactive evolution process of all subjects within a given time window; In the test environment of the merging zone of an expressway, the focus is on the game relationship between the test vehicle and adjacent background vehicles, while also considering road set and traffic flow. Let the test vehicle be denoted as... e The target background car is recorded as b Then construct the scene layer state vector: in, Indicates the relative distance in the longitudinal direction. Indicates the relative horizontal distance. and These represent the longitudinal and lateral relative velocity differences, respectively. , This indicates the speed of the test vehicle relative to the adjacent background vehicle. , This represents the longitudinal acceleration of the corresponding vehicle.

3. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 2, characterized in that: In the natural driving model based on Copula functions, for any vehicle Its state vector is defined as: in, , These are the position coordinates in the global coordinate system. For velocity scalar, For heading angle, For longitudinal acceleration, This refers to the yaw rate; For any vehicle Its action vector is defined as: in, Indicates longitudinal control input. This indicates input for lateral steering or attitude adjustment; Based on the vehicle's planar kinematics, the state update process is represented as follows: With a fixed sampling frequency, the longitudinal acceleration and yaw rate of the vehicle are estimated using discrete difference, i.e.: in, That is, the sampling time interval.

4. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 3, characterized in that: In the natural driving model based on the Copula function, let the reference coordinates of the starting point of the merging zone along the main line be... The test vehicle's current location is The longitudinal distance of the test vehicle relative to the merging point is expressed as: Meanwhile, let the lateral offset of the test vehicle relative to the centerline of the target lane be... and with Indicates the lane keeping error threshold, in The threshold for determining the speed difference is used to divide the interaction phase into: in, and The longitudinal segmentation threshold is set based on road geometry and statistical data characteristics to meet the following requirements. Steady-state following The stage refers to the period before the vehicle enters the actual merging preparation area; merging is executed. The stage indicates that the vehicle has exhibited significant lateral movement during lane change; conflict avoidance. The phase corresponds to the risk response phase triggered by a sharp compression of the gap, a sudden change in the velocity difference, or a rapid decrease in the collision time; Based on this, the feature vector used for joint distribution modeling is defined as in, For defined continuous variables, This is a label for the interaction stage.

5. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 4, characterized in that: In the natural driving model based on the Copula function, let the feature random vector extracted during the merging zone interaction be: in, Indicates the first The dynamic variables include relative longitudinal spacing, relative lateral offset, main vehicle speed, background vehicle speed, longitudinal acceleration, lateral speed, and interaction stage indicators; Let the marginal distribution functions of each component be denoted as . According to Sklar's theorem, the joint distribution of the random vector z can be expressed as: In the formula, The parameter is Copula functions; If each marginal distribution has a probability density function The joint probability density function is then expressed as: in, ( · ) is the Copula density function; We use Gaussian Copula to model the variable dependency structure, let ,in, Let be the inverse distribution function of the standard normal distribution, then the Gaussian Copula is expressed as: in, The correlation matrix is ​​represented as R The multivariate normal distribution function; Correspondingly, its density is expressed as: in, , It is an identity matrix.

6. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 5, characterized in that: In the parameter estimation stage of the Copula-based natural driving model, the original samples are first subjected to an empirical distribution transformation: Let the first... The observed samples are The total sample size is Then, a pseudo-observation is defined as: Based on this, mapping it to Gaussian space yields: Then transform the joint distribution model into a conditional sampling form; let... For a moment t State feature vector, Given the feature vector from the previous time step, the natural driving state transition can be abstractly represented as: Introducing interaction mode variables ,in For the set of interaction phases, given an initial state Then, natural driving trajectories are gradually generated through Monte Carlo sampling, namely: And by combining the time-series update mechanism, a complete dynamic interaction sequence is obtained.

7. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 1, characterized in that: In the adversarial dynamic element scene generation process based on POMDP, the partially observable Markov decision process (POMDP) ​​is used to model the adversarial dynamic element scene generation process, which is represented as follows: in, Represents the state space of the real environment. Represents the action space. Represents the observation space. Represents the state transition function. Represents the observation probability function. Represents the reward function, Indicates the discount factor; While maintaining consistency with the state parameters in the natural driving model, the real-world environment state in POMDP is defined as follows: in, , , and These represent the relative position difference and relative speed difference between the test vehicle and the target vehicle, respectively. , This represents the longitudinal acceleration of the test vehicle and the background vehicle. This indicates the yaw rate of the test vehicle. Indicates the interaction stage label; Indicates communication delay. Indicates perceived confidence level. Indicates the status of the sensing or communication link.

8. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 7, characterized in that: When the actual state cannot be directly observed, define observation mode variables. And the observation vector is represented as: in, () represents the mapping function related to the observation mode. This is random observation noise; Indicates the delay step size. This represents an observation operator that masks or maps out some state variables; Based on the aforementioned settings, the longitudinal and lateral control variables of the test vehicle are discretized, and the motion space is constructed using a combination of the two, namely: The action at any given moment is represented as: in, and These represent the test vehicle at time [time]. Longitudinal and lateral control inputs; In terms of state transition modeling, an adversarial perturbation term is introduced based on the natural driving model, resulting in: in, This represents the state evolution function constrained by the Copula natural driving distribution. The state transition probability is represented by the adversarial disturbance term formed by the random response of the background vehicle, the anomaly in perception communication, and the disturbance in the local environment. Regarding the reward function, the single-step reward is designed as follows: in, , , These are non-negative weighting coefficients, which respectively control the relative importance of risk approximation, terminal collision, and action smoothing. This is a risk item. For explicit collision rewards, To control the smoothing penalty term.

9. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 8, characterized in that: In the designed reward function, the risk term is defined in TTC form as follows: in, This is a scale parameter; this allows the test vehicle to continue receiving feedback as it approaches the high-risk boundary even if a collision does not occur immediately. Explicit collision rewards are triggered only at the terminal moment and are defined as follows: This feature enhances the search capabilities for high-risk endgame scenarios; The control smoothing penalty term is expressed as: in, This is a weight for horizontal control changes; this term can suppress abrupt changes in action between adjacent time steps.

10. The method for generating potential unsafe scenarios for vehicle dynamic interaction testing based on POMDP according to claim 8, characterized in that: Since the agent cannot directly obtain the true state, it uses short-term historical observations to stack the true state. To approximate it, that is: in, L Indicates the length of the historical observation window; intelligent agents at all times t based on The strategy for selecting an action is represented as follows: in, For online Q network, For network parameters; In DDQN, the objective Q-value is determined by the online network selecting actions and the objective network evaluating those actions. The updated objective is represented as follows: in, Indicates the target network parameters; Based on this, the loss function is defined as: By minimizing this loss function, the online network parameters are continuously updated, enabling it to gradually learn the long-term risk-reward under different observation conditions and action combinations. In the training process, the online network, target network, and experience replay pool are first initialized, and the exploration rate is set. Discount Factor Synchronize with the target network cycle; subsequently, in each round of scene generation, the agent... The mechanism selects a combination of lateral and longitudinal control from the action space and applies it to the adversarial environment to obtain the next moment's observation. Instant rewards and round termination flag; state transition sample After being stored in the experience replay pool, the online network parameters are updated using a small-batch random sampling method, and the online network weights are periodically copied to the target network.

Images