Dynamic test scene confrontation generation method based on multi-stage spatial-temporal feature integration

Abstract

The invention relates to a dynamic test scene confrontation generation method based on multi-stage spatial-temporal feature integration, and belongs to the technical field of automatic driving automobile testing. The method comprises the following steps: S1, coding high-frequency dynamic characteristics according to traffic flow trajectory data, and additionally generating explicit interaction graph characteristics; s2, combining high-frequency dynamic features with explicit interaction graph features, and extracting traffic flow spatial features by using a secondary interaction feature module; s3, realizing spatio-temporal feature fusion by using a lightweight time integration module and a secondary interaction feature extraction module; s4, constructing a track decoding module and completing the generation of a test scene; and S5, constructing a trajectory discriminator and designing an overall network loss function for adversarial training. Trajectory characteristic extraction is realized through combination of deep extraction of interactive characteristics and lightweight time sequence characteristic integration, and generator trajectory generation is better realized. Compared with a traditional scene generation method, the method can generate a more real and accurate driving scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous vehicle testing, and relates to a dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration. Background Art

[0002] The test scenario generation technology is a key link in the verification and optimization of autonomous driving systems. By constructing diverse and high-fidelity dynamic scenarios to simulate the behaviors of traffic participants in complex interaction environments, it provides reliable test data for downstream modules to ensure the safety and robustness of autonomous vehicles in real dynamic environments.

[0003] Currently, test scenario generation methods mainly include rule-based manual design methods, data-based statistical modeling methods, and artificial intelligence-based generation methods. Traditional manual design methods rely on expert experience and are suitable for constructing simple scenarios, but it is difficult to cope with real traffic environments with high dynamics and strong interactivity; data-based statistical modeling methods generate scenarios through historical data analysis, but they have a strong dependence on data distribution and it is difficult to cover long-tail scenarios. In recent years, artificial intelligence-based generation methods, especially those based on deep learning and generative adversarial networks, have gradually become a research hotspot due to their strong feature extraction capabilities and data-driven characteristics. These methods can be further refined into specific implementations based on convolutional neural networks, graph neural networks, Transformer architectures, and generative adversarial networks. In the test scenario generation task, the spatial interaction characteristics and temporal evolution laws of traffic flow are core elements, and the dynamic generation method based on multi-stage spatio-temporal feature integration can effectively fuse multi-source information to generate test scenarios with high authenticity and diversity.

[0004] However, existing test scenario generation technologies still face several challenges. First, the dynamic characteristics of traffic flow have multi-frequency and high-dimensional spatio-temporal complexity, and it is difficult to comprehensively capture with a single feature extraction method; second, the explicit interaction relationships and implicit spatio-temporal coupling characteristics between traffic participants need to be comprehensively modeled; in addition, the generated scenarios need to meet the diversity and uncertainty requirements of downstream tasks (such as trajectory prediction and decision-making planning). Therefore, there is an urgent need for a new dynamic test scenario adversarial generation method to solve the above problems. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration. By encoding high-frequency trajectory information, extracting explicit interaction features, fusing spatio-temporal dynamic characteristics, and combining an adversarial training mechanism, it realizes the generation of high-fidelity and multi-modal test scenarios, providing solid support for the safety verification and performance optimization of autonomous driving systems.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration, specifically including the following steps:

[0008] S1: Encode high-frequency dynamic features according to traffic flow trajectory data, and additionally generate explicit interaction graph features;

[0009] S2: Combine high-frequency dynamic features with explicit interaction graph features, and use a quadratic interaction feature module to extract traffic flow spatial features;

[0010] S3: Use a lightweight time integration module and a quadratic interaction feature extraction module to achieve spatio-temporal feature fusion;

[0011] S4: Construct a trajectory decoding module and complete the generation of the test scenario;

[0012] S5: Construct a trajectory discriminator and design an overall network loss function for adversarial training.

[0013] Further, in step S1, encoding high-frequency dynamic features according to traffic flow trajectory data specifically includes the following steps:

[0014] S11: The scenario data is represented as G, where the content sources used to construct the scenario data G include the historical motion states H of the agents t :

[0015] G = [H1, H2,..., H t ,...]

[0016]

[0017] where n represents the number of vehicle nodes in the scenario, represents the position information of node i at time t and the heading angle information

[0018] S12: For traffic flow trajectories with complex dynamics, use multi-frequency sine-cosine encoding to map the data to obtain multi-frequency dynamic features F, and encode the high-frequency dynamic features F through a linear layer emb , in order to ensure that the influence of the dimension between different information is mitigated during encoding, a scaling factor is used to stabilize the encoded values:

[0019] F = [H t , f0, f1,..., f L

[0020]

[0021] F emb ​= Linear(F)

[0022] where α p represents the position information scaling factor, represents the heading angle information scaling factor, l represents the frequency series, and L is the maximum series;

[0023] S13: Extract the relative relationship between vehicle nodes according to the historical motion state H t ; by splicing the relative position relationship P relative and the relative velocity V relative , and then encoding and outputting the node explicit interaction graph feature through the linear layer;

[0024] H relative = [P relative , V relative

[0025]

[0026] F relative = Linear(H relative )

[0027] where H relative is the complete relative feature between nodes, P i t is the position information of node i at time t, is the relative position relationship between two nodes at time t; V i t is the velocity information of node i at time t, is the relative velocity between two nodes at time t; F relative is the explicit graph interaction feature between nodes at all times.

[0028] Furthermore, in step S1, an explicit interaction graph feature is additionally generated, specifically including: To achieve the extraction of important features, a two-layer vehicle mask matrix M is constructed through the inter-vehicle distance mask M dis and the dummy node mask M fake to remove irrelevant explicit interaction features:

[0029] M dis = if(dis i-＞j < dis thre )

[0030] M fake = if(SUM(h i ) == nan)

[0031] M = M dis ⊙ M fake

[0032] where dis​i-＞j represents the relative distance between node i and node j at a certain moment; dis thre represents the threshold of vehicle distance; SUM(h i ) == nan represents the judgment condition of virtual nodes; ⊙ represents element-wise multiplication of matrices.

[0033] Furthermore, in step S2, the quadratic interaction feature module is used to extract traffic flow spatial features, which specifically includes the following steps:

[0034] S21: According to the high-frequency dynamic feature F emb Use the quadratic interaction feature module to extract vehicle interaction features; Pass the dynamic feature through the self-attention mechanism of the explicit interaction graph feature F relative to perform the first-stage spatial feature extraction, and output the attention matrix α, so as to extract the interaction feature F act :

[0035] F0 = F emb ·W0

[0036]

[0037] α = softmax(((F relative ·W relative )⊙F1)·W atten )

[0038] F act = αF emb

[0039] Among them, F0 is the high-dimensional feature of vehicle state, F1 is the implicit vehicle interaction feature, is the ego-vehicle state feature in the interaction, is the other-vehicle state feature in the interaction, W0 and W1 are state mapping weight matrices, W relative is the relative feature fusion weight matrix, W atten is the importance mapping weight matrix, softmax(·) represents the activation function softmax processing, ⊙ represents element-wise multiplication of matrices, and · represents matrix multiplication;

[0040] S22: Integrate the spatial features received by the vehicle according to the high-frequency dynamic feature F emb and the interaction feature F act ; Pass the dynamic feature and the interaction feature through the linear gating mechanism, linear layer, linear layer and residual connection to realize the controllable integration of the interaction feature and the original node feature, and output the first-stage spatial feature F spatial of the vehicle: spatial :

[0041] F spatial = (F emb ·W2 × sigmoid(Femb ·W3)+F act )·W spatial

[0042] Among them, W2 is the vehicle state mapping weight matrix, W3 is the vehicle state gating mapping weight matrix, and W spatial is the integration weight matrix of vehicle interaction features and state features;

[0043] S23: Perform second-stage depth extraction based on the first-stage spatial feature F spatial ; update the deep interaction relationship through the self-attention mechanism to integrate the spatial feature;

[0044] Q = F spatial ·W Q , K = F spatial ·W K , V = F spatial ·W V

[0045]

[0046] Among them, drop(·) represents the dropout layer, norm(·) represents the normalization layer, Q, K, and V respectively represent the query, key, and value of attention concentration, and W Q , W K and W V represent the standard qkv weight matrix, (·) T represents matrix transpose, and d k represents the dimension of the vector; is the first-stage spatial interaction feature,[[ID=-47]] is the second-stage initial spatial interaction feature;

[0047] In addition, in the second-stage spatial feature extraction, the same gating feature fusion mechanism as in the first stage is adopted to ensure the transmission of effective information;

[0048] S24: Based on the extracted spatial attention features, use the feed-forward network architecture in the transformer network to integrate the individual features of each node; pass the second-stage spatial features through a feed-forward network composed of a linear layer, an activation function, a linear layer, a normalization layer, and a residual connection to achieve non-linear expansion and compression of node-independent features and improve the feature expression ability;

[0049]

[0050] Among them, is the final second-stage spatial interaction feature, W4 and W5 are the vehicle state and vehicle state gating mapping weight matrices respectively, and gelu(·) represents the gelu operation of the activation function.

[0051] Furthermore, in step S3, a lightweight time integration module and a secondary interaction feature extraction module are used to achieve spatio-temporal feature fusion, which specifically includes the following steps:

[0052] S31: Implement the integration of time information using a lightweight time integration module; pass the hidden features through parallel multi-scale convolutional blocks, and initially integrate short-term time features at different fine-grained levels using convolutional architectures of different scales. Each convolutional block consists of two one-dimensional convolutional layers with the same convolutional kernel size and a linear layer. Different from the past, the time dimension is used as the channel dimension of the convolutional layer instead of the feature dimension in the convolutional layer:

[0053]

[0054] Among them, is the spatio-temporal feature integrated for branch i, Conv0 represents the first convolutional layer, Conv1 represents the second convolutional layer, and W6 is the spatial interaction feature mapping weight matrix;

[0055] S32: Add the spatio-temporal features initially integrated by different branches to achieve feature unification; pass the branch features through a structure of addition, normalization layer, residual connection, and dropout layer to achieve simple feature fusion:

[0056]

[0057] Among them, represents the initial spatio-temporal feature, represents the feature output by branch one, represents the feature output by branch two, represents the feature output by branch three, W7 is the feature fusion weight matrix, and || represents the concatenation operation;

[0058] S33: After the parallel convolutional blocks, tile the features along the time dimension, implement high-dimensional - low-dimensional - high-dimensional feature transformation through a linear layer, and then combine the normalization layer, residual connection, and dropout layer to achieve the overall architecture:

[0059]

[0060] Among them, represents the initial low-dimensional spatio-temporal feature, F s-t represents the final spatio-temporal feature, Flatten(·) represents the tiling operation, W8 and W9 are the weight matrices for high-dimensional - low-dimensional - high-dimensional transformation, and dim represents the dimension indication of the tiling operation;

[0061] The quadratic interaction feature module implements spatial feature extraction without considering temporal variations, while the lightweight temporal integration module implements temporal feature integration based on spatial features. A feature extraction module with the same structure as the quadratic interaction feature module extracts coupled spatial features from spatiotemporal features. Although the lightweight temporal integration module flattens the features along the temporal dimension through linear layers to integrate temporal features, the residual connection structure still leaves subtle characteristics at each time step. Therefore, the encoded explicit interaction features are still used in the first-stage feature extraction of the feature extraction module to supplement the learning of interactive attention.

[0062] Furthermore, in step S4, a trajectory decoding module is constructed, which specifically includes the following steps:

[0063] S41: In the overall model, due to the choice of data format and the special nature of the task, each scene node is selected to output the subsequent trajectory separately. The fully extracted features are flattened in the time dimension and dimensionality is compressed through a linear layer. The implicit features of each node are integrated into independent features for subsequent output decoding:

[0064] F out =norm(Flatten(F s-t ,dim=-2)·W 10 )

[0065] Among them, F out Represents the complete independent characteristics of the vehicle node, W 10 is the weight matrix of information compression;

[0066] S42: Based on the independent features of each node, the subsequent trajectory content is output through the parallel linear layer, and the output of the subsequent trajectory is divided into two parts: the subsequent heading angle output and the subsequent position information output; the features are directly decoded through the linear layer to obtain the sine and cosine values of the subsequent heading angle

[0067]

[0068] Among them, L out Indicates the output trajectory length, W 11 is the heading angle information decoding weight matrix;

[0069] S43: Model the position information output as an output form that takes uncertainty into account, and establish a single-peak Gaussian distribution to fit the position possibility of each step; the features are directly decoded through the linear layer to directly decode the subsequent trajectory speed information and the standard deviation of the horizontal and vertical coordinates of each step:

[0070] σ=softplus(F out W 12 )

[0071] V out = F out · W 13

[0072] where softplus(·) represents the activation function; W 12 , W 13 are the position error prediction weight matrix and the speed information decoding weight matrix; σ is the standard deviation value of the predicted subsequent position error of the vehicle, and V out is the subsequent speed of the vehicle;

[0073] S44: Generate the position information by using the speed information and the standard deviation; the position mean value is output by accumulating the speed and the position information at the last moment of the input trajectory, and then the final subsequent position information is generated by sampling.

[0074] Furthermore, in step S5, construct a trajectory discriminator, which specifically includes the following steps:

[0075] S51: Design a discriminator model in the MLP architecture according to the trajectory content and the capacity of the generator network; similar to the overall architecture of the generator, the discriminator is also composed of a multi-frequency encoder and the MLP network itself; the multi-frequency encoder of the discriminator and the multi-frequency encoder of the generator adopt the same number of frequency levels and scaling factors:

[0076] H emd = Embedding(H)

[0077] H flag = MLP(H emd )

[0078] where H represents the input trajectory, H emd represents the encoded data, and H flag represents the discriminator judgment result;

[0079] S52: In the discriminator with the MLP architecture, encode the features of all channels of the trajectory of each node into a hidden feature, and finally output it by a process of gradually compressing from high dimension to low dimension. In addition, an activation function is added between each linear layer to introduce non-linearity and improve the feature expression ability.

[0080] Furthermore, in step S5, the loss function of the overall network includes the generator loss and the discriminator loss; the generator loss L G includes the position loss, the heading angle loss, the trajectory deviation loss, and the adversarial cross-entropy loss of the trajectory reconstruction; the discriminator loss L D includes the adversarial cross-entropy loss;

[0081] L G = α0 × log(1 - D(G(H))) + α1 × (H real-(G(H)) / (B + n)

[0082] L D = logD(H real ) + log(1 - D(G(H)))

[0083] α0 and α1 represent the weights of each loss, G(·) represents the generator, D(·) represents the discriminator, H real represents the true trajectory, H represents the input trajectory of the generator, and B and n represent the batch size of the input data and the number of nodes when calculating the loss each time.

[0084] The beneficial effects of the present invention are as follows: The present invention realizes the extraction of trajectory characteristics through the deep extraction of interaction features combined with the lightweight integration of temporal features, and better realizes the generation of generator trajectories. Compared with traditional scene generation methods, the present invention can generate more realistic and accurate driving scenes.

[0085] (1) The present invention proposes a dynamic test scene generation method based on multi-stage spatio-temporal feature integration, designs a multi-frequency sine-cosine encoding and explicit interaction graph feature extraction module, comprehensively considers the high-frequency dynamic characteristics of traffic flow trajectories and the interaction relationship between vehicles, stabilizes the encoded values through a scaling factor and combines a two-layer vehicle mask matrix to remove irrelevant features, and the constructed interaction graph can efficiently and accurately express the spatio-temporal characteristics of traffic flow, improving the authenticity and diversity of test scene generation.

[0086] (2) The present invention designs a spatio-temporal feature extraction module based on the self-attention mechanism and lightweight separable convolution. Through multi-stage spatial feature extraction and temporal feature integration, it captures the deep interaction relationship and short-term dynamic evolution law of traffic flow respectively. The gated feature fusion mechanism is adopted to ensure the effective integration of features, and progressive feature extraction realizes the effective decoupling and coupling of spatial features and temporal features, ensuring the spatio-temporal consistency of the generated test scenes in a high-frequency dynamic environment.

[0087] (3) The present invention constructs a multi-modal trajectory decoding module and an adversarial training framework, combines the unimodal Gaussian distribution to model the uncertainty of the trajectory, and outputs a multi-modal test scene including heading angle and position information. A comprehensive loss function including position loss, heading angle loss, trajectory deviation loss and adversarial cross-entropy loss is designed, and the performance of the generator is optimized through the dynamic game of the trajectory discriminator, realizing the efficient generation and training of high-fidelity test scenes.

[0088] Other advantages, objectives and features of the present invention will be elaborated to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. Brief Description of the Drawings

[0089] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in detail and preferably below in conjunction with the accompanying drawings, where:

[0090] Figure 1 is the overall flowchart of the dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration proposed by the present invention;

[0091] Figure 2 is the flowchart of the two-stage spatial feature extraction and temporal feature integration module;

[0092] Figure 3 is the training flowchart of the overall adversarial generation network architecture. Detailed Embodiments

[0093] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0094] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as limiting the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0095] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0096] Please refer to Figures 1 to 3, the present invention provides a dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration. First, high-frequency transformation information is extracted from the existing traffic flow trajectory data of the scenario, and explicit vehicle interaction information features are constructed in the form of a graph and encoded as auxiliary features. Secondly, local and global spatial features between different nodes of the traffic flow are extracted through a two-stage interaction module by combining explicit and implicit interaction features. Temporal features are integrated based on the traffic flow spatial features using multi-scale lightweight convolution, and spatio-temporal features are deeply extracted again using the interaction module. Finally, a unit Gaussian distribution is constructed to decode and output subsequent test scenarios, and a trajectory reconstruction and generative adversarial multi-task loss function is designed for adversarial training. The present invention realizes trajectory feature extraction through the deep extraction of interaction features combined with lightweight temporal feature integration, and better realizes generator trajectory generation. Compared with traditional scenario generation methods, the present invention can generate more realistic and accurate driving scenarios.

[0097] The method specifically includes the following steps:

[0098] S1: Encode high-frequency dynamic features based on traffic flow trajectory data and additionally generate explicit interaction graph features;

[0099] S2: Combine high-frequency dynamic features and explicit interaction graph features, and use a quadratic interaction feature module to extract traffic flow spatial features;

[0100] S3: Use a lightweight time integration module and a quadratic interaction feature extraction module to achieve spatio-temporal feature fusion;

[0101] S4: Construct a trajectory decoding module and complete the generation of test scenarios;

[0102] S5: Construct a trajectory discriminator and design an overall network loss function for adversarial training.

[0103] Embodiment:

[0104] In step S1, in this example, traffic flow historical trajectory information is used to construct richer data features through multi-frequency encoding. In addition, explicit vehicle interaction relationships are constructed through historical trajectory information, and a masking mechanism is adopted to screen explicit interaction features. Both the encoded data and the explicit interaction features are used for feature extraction of subsequent models. Specifically, it includes the following steps:

[0105] S11: The scenario data is represented as G, where the content source for constructing the scenario data G includes the historical motion states H of the agents t :

[0106] G = [H1, H2,..., H t ,...]

[0107]

[0108] Among them, n represents the number of vehicle nodes in the scenario, represents the position information of node i at time t and the heading angle information

[0109] S12: For traffic flow trajectories with complex dynamics, multi-frequency sine-cosine coding is used to map data to obtain multi-frequency dynamic features F, and the high-frequency dynamic features F are encoded through a linear layer emb , in order to ensure that the influence brought by the dimension between different information is slowed down during coding, a scaling factor is used to stabilize the coding value:

[0110] F = [H t , f0, f1,..., f L

[0111]

[0112] F emb = Linear(F)

[0113] Among them, α p represents the position information scaling factor, represents the heading angle information scaling factor, l represents the frequency series, and L is the maximum series.

[0114] S13: According to the historical motion state H t extract the relative relationship between vehicle nodes; by concatenating the relative position relationship P relative and the relative velocity relationship V relative , and then encoding through a linear layer to output the explicit interaction graph features of the nodes:

[0115] H relative = [P relative , V relative

[0116]

[0117] F relative = Linear(H relative )

[0118] Among them, P i t is the position information of a certain node at time t; V i t is the velocity information of a certain node at time t; F relative serves as the explicit relative feature encoding between nodes at all times.

[0119] S14: In order to achieve the extraction of important features, through the inter-vehicle distance mask M dis and the virtual node mask M fakeConstruct a two-layer workshop mask matrix M to remove irrelevant explicit interaction features:

[0120] M dis = if(dis i-＞j < dis thre )

[0121] M fake = if(SUM(h i ) == nan)

[0122] M = M dis ⊙M fake

[0123] where dis i-＞j represents the relative distance between node i and node j at a certain moment; dis thre represents the workshop distance threshold; SUM(h i ) == nan represents the virtual node judgment condition.

[0124] In step S2, with explicit interaction features as an aid, use the self-attention mechanism combined with gated feature fusion to deeply extract spatial features. The self-attention mechanism is used to learn the mutual attention relationships between different vehicle nodes, while the gating mechanism ensures the enhanced retention of its own features. It specifically includes the following steps:

[0125] S21: According to the high-frequency dynamic feature F emb Use the quadratic interaction feature module to extract vehicle interaction features; Pass the dynamic features through the self-attention mechanism considering the explicit interaction graph feature F relative to perform the first-stage spatial feature extraction, output the attention matrix α, and thus extract the interaction feature F act :

[0126] F0 = F emb ·W0

[0127]

[0128] α = softmax(((F relative ·W relative )⊙F1)·W atten )

[0129] F act = α·F emb

[0130] where W0 and W1 are state mapping weight matrices, W relative is the relative feature fusion weight matrix, W atten is the importance mapping weight matrix, ⊙ represents element-wise matrix multiplication, and · represents matrix multiplication.

[0131] S22: According to the high-frequency dynamic feature F emb and the interaction feature F act integrate the spatial feature F received by the vehicle spatial ; Through the linear gating mechanism, linear layers, linear layers and residual connections for the dynamic feature interaction feature, realize the controllable integration of the interaction feature and the original node feature, and output the first-stage spatial feature of the vehicle:

[0132] F spatial =(F emb ·W2×sigmoid(F emb ·W3)+F act )·W spatial

[0133] S23: Perform second-stage depth extraction according to the first-stage spatial feature F spatial ; Update the deep interaction relationship of the spatial feature through the self-attention mechanism to integrate the spatial feature.

[0134] Q = F spatial ·W Q , K = F spatial ·W K , V = F spatial ·W V

[0135]

[0136] where drop(·) represents the dropout layer, norm(·) represents the normalization layer, W Q , W K and W V represent the standard qkv weight matrices, (·) T represents the matrix transpose, and d k represents the dimension of the vector.

[0137] In addition, in the second-stage spatial feature extraction, the same gating feature fusion mechanism as in the first stage is adopted to ensure the transmission of effective information.

[0138] S24: Based on the extracted spatial attention feature, use the feed-forward network architecture in the transformer network to integrate the individual features of each node. Pass the second-stage spatial feature through the feed-forward network composed of a linear layer, activation function, linear layer, normalization layer and residual connection to realize the non-linear expansion and compression of the node independent feature and improve the feature expression ability:

[0139]

[0140] In step S3, the temporal features in the spatial features are integrated using the cross-channel feature of convolution, and then the spatio-temporal features are deeply integrated in combination with the linear layer. A module with the same architecture as the module for extracting spatial features is used to achieve deep coupling of spatio-temporal features.

[0141] Specifically, it includes the following steps:

[0142] S31: The trajectory information is high-frequency spatio-temporal dynamic data. However, due to the input of short-term information, a lightweight time integration module is used to integrate time information. The hidden features are passed through parallel multi-scale convolution blocks, and the short-term time features are initially integrated at different fine-grained levels using convolution architectures of different scales. Each convolution block consists of two one-dimensional convolution layers with the same kernel size and a linear layer. Different from the past, in the convolution layer, the time dimension is used as the channel dimension of the convolution layer instead of the feature dimension:

[0143]

[0144] Among them, Conv0 represents the first convolution layer, and Conv1 represents the second convolution layer.

[0145] S32: Add the spatio-temporal features initially integrated by different branches to achieve feature unification. The branch features are simply fused through a structure of addition, normalization layer, residual connection, and dropout layer:

[0146]

[0147] Among them, represents the features output by branch one, and || represents the concatenation operation.

[0148] S33: To achieve feature integration in the short term. After the parallel convolution blocks, the features are tiled along the time dimension, and a high-dimensional - low-dimensional - high-dimensional feature transformation is achieved through a linear layer to enrich the feature expression of each node in the time dimension, realizing the integration of time features. Then, in combination with the normalization layer, residual connection, and dropout layer, the overall architecture is realized:

[0149]

[0150] Among them, Flatten(·) represents the tiling operation.

[0151] S34: The secondary interaction feature module realizes the extraction of spatial features without considering temporal changes. The lightweight time integration module realizes the integration of temporal features based on spatial features. A feature extraction module with the same structure as the secondary interaction feature module is used to extract the coupled spatial features under spatio-temporal features. Although in the lightweight time integration module, in order to integrate the features in the time dimension, the features are tiled along the time dimension and passed through a linear layer, but due to the residual connection structure, there are still subtle characteristics at each time step. Therefore, in the first-stage feature extraction in the feature extraction module, the encoded explicit interaction features are still used as supplementary learning interaction attention.

[0152] In step S4, the position output of the subsequent traffic flow trajectory is modeled in the form of a low-dimensional Gaussian distribution, and a parallel linear layer is used as the output head to realize the generation of multi-dimensional information of the subsequent trajectory. Specifically, it includes the following steps:

[0153] S41: In the overall model, due to the choice of data format and the special nature of the task, each scene node is selected to output the subsequent trajectory separately. The features after sufficient extraction are tiled along the time dimension, and dimension compression is achieved through a linear layer. The implicit features of each node are integrated into independent features for subsequent output decoding:

[0154] F out = norm(Flatten(F s-t , dim=-2)·W 10 )

[0155] S42: According to the independent features of each node, the subsequent trajectory content is output through a parallel linear layer. The output of the subsequent trajectory is divided into two parts, the output of the subsequent heading angle and the output of the subsequent position information. The features are directly decoded through a linear layer to obtain the sine and cosine values of the subsequent heading angle

[0156]

[0157] Among them, L out represents the output trajectory length.

[0158] S43: The position information output is modeled as an output form considering uncertainty, and a unimodal Gaussian distribution is established to fit the possible positions at each step. The features are directly decoded through a linear layer to obtain the subsequent trajectory speed information and the standard deviation values of the horizontal and vertical coordinates at each step:

[0159] σ = softplus(F out ·W 12 )

[0160] V out = F out ·W 13

[0161] Among them, softplus(·) represents the activation function used to stabilize the value of the standard deviation to ensure rationality.

[0162] S44: Generate the position information by using the speed information and the standard deviation. The position mean is output by accumulating the speed and the position information at the last moment of the input trajectory, and then the final subsequent position information is generated by sampling.

[0163] In step S5, the adversarial generation network architecture is fully constructed. The discriminator is designed according to the generator model. At the same time, in order to achieve the authenticity and accuracy of trajectory generation, the generator loss function is designed by using the multi-task learning method. The specific steps are as follows:

[0164] S51: Design the discriminator model with the MLP architecture according to the trajectory content and the capacity of the generator network. Similar to the overall architecture of the generator, the discriminator is also composed of a multi-frequency encoder and the MLP network itself. The multi-frequency encoder of the discriminator and the multi-frequency encoder of the generator adopt the same number of frequency levels and scaling factors:

[0165] H emd = Embedding(H)

[0166] H flag = MLP(H emd )

[0167] Among them, H represents the input trajectory, H emd represents the encoded data, and H flag represents the discriminator judgment result.

[0168] S52: In the discriminator with the MLP architecture, the features of all channels of the trajectory of each node are encoded into a hidden feature, and a process of gradually compressing from high dimension to low dimension is adopted for the final output. In addition, an activation function is added between each linear layer to introduce non-linearity and improve the feature expression ability:

[0169] S53: In the whole task, the generator faces two sub-tasks, namely the trajectory reconstruction task and the adversarial generation task. Therefore, the loss function is also designed as the position loss, heading angle loss, trajectory deviation loss of trajectory reconstruction and the adversarial cross-entropy loss, while the discriminator loss only contains the adversarial cross-entropy loss term:

[0170] L G = α0×log(1 - D(G(H))) + α1×(H real - G(H)) / (B + n)

[0171] L D = logD(H real ) + log(1 - D(G(H)))

[0172] Among them, α represents the loss weight, H represents the input trajectory of the generator, and B and n represent the batch size of the input data and the number of nodes each time the loss is calculated.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A dynamic test scenario adversarial generation method based on multi-stage spatio-temporal feature integration, characterized in that, The method specifically includes the following steps: S1: Encode high-frequency dynamic features based on traffic flow trajectory data and additionally generate explicit interaction graph features; S2: Combine high-frequency dynamic features and explicit interaction graph features, and use a quadratic interaction feature module to extract traffic flow spatial features; S3: Use a lightweight time integration module and a quadratic interaction feature extraction module to achieve spatio-temporal feature fusion; S4: Construct a trajectory decoding module and complete the generation of test scenarios; S5: Construct a trajectory discriminator and design an overall network loss function for adversarial training.

2. The dynamic test scenario adversarial generation method according to claim 1, wherein In step S1, encoding high-frequency dynamic features based on traffic flow trajectory data specifically includes the following steps: S11: The scenario data is represented as G, where the content source for constructing the scenario data G includes the historical motion state H of the agent t : G = [H1, H2,..., H t ,...] where n represents the number of vehicle nodes in the scenario, represents the position information of node i at time t and the heading angle information S12: For traffic flow trajectories with complex dynamics, multi-frequency sine-cosine coding is used to map data to obtain multi-frequency dynamic features F, and the high-frequency dynamic features F are encoded through a linear layer. emb A scaling factor is used to stabilize the encoded values: F = [H t , f0, f1,..., f L ​ F emb = Linear(F) Among them, α p represents the scaling factor of the position information, represents the scaling factor of the heading angle information, l represents the frequency series, and L is the maximum series; S13: Extract the relative relationship between vehicle nodes according to the historical motion state H t Extract the relative relationship between vehicle nodes; by splicing the relative position relationship P relative and the relative speed V relative , and then output the explicit interaction graph features of the nodes through linear layer encoding; H relative = [P relative , V relative ​ F relative = Linear(H relative ) Among them, H relative is the complete relative feature between nodes, is the position information of node i at time t, is the relative position relationship between two nodes at time t; is the velocity information of node i at time t, is the relative velocity between two nodes at time t; F relative is the explicit graph interaction feature between nodes at all times.

3. The dynamic test scenario adversarial generation method according to claim 2, wherein In step S1, explicit interaction graph features are additionally generated, specifically including: constructing a two-layer workshop mask matrix M through the inter-vehicle distance mask M dis and the virtual node mask M fake to remove irrelevant explicit interaction features: M dis = if(dis i-＞j < dis thre ) M fake = if(SUM(h i ) == nan) M = M dis ⊙M fake Among them, dis i-＞j represents the relative distance between node i and node j at a certain moment; dis thre represents the threshold of the vehicle spacing; SUM(h i ) == nan represents the judgment condition of the virtual node; ⊙ represents element-by-element multiplication of the matrix.

4. The dynamic test scenario adversarial generation method according to claim 3, wherein In step S2, using a quadratic interaction feature module to extract traffic flow spatial features specifically includes the following steps: S21: According to the high-frequency dynamic feature F emb Use the quadratic interaction feature module to extract vehicle interaction features; Pass the dynamic feature through the self-attention mechanism of the explicit interaction graph feature F relative Perform a first-stage spatial feature extraction to output the attention matrix α, so as to extract the interaction feature F act : F0 = F emb ·W0 α = soft max(((F relative ·W relative ) ⊙ F1) · W atten ) Among them, F0 is the high-dimensional feature of the vehicle state, and F1 is the implicit vehicle-to-vehicle interaction feature. is the ego-vehicle state feature in the interaction. is the other-vehicle state feature in the interaction. W0 and W1 are state mapping weight matrices, and W relative is the relative feature fusion weight matrix, and W atten is the importance mapping weight matrix. softmax(·) represents the processing by the activation function softmax, ⊙ represents element-wise multiplication of matrices, and · represents matrix multiplication. S22: According to the high-frequency dynamic feature F emb and the interaction feature F act integrate the spatial feature F received by the vehicle spatial ; Through the linear gating mechanism, linear layers, linear layers and residual connections for the dynamic feature and the interaction feature, integrate the interaction feature and the original node feature, and output the first-stage spatial interaction feature F of the vehicle spatial : F spatial = (F emb · W2 × sigmoid(F emb · W3) + F act ) · W spatial Among them, W2 is the vehicle state mapping weight matrix, W3 is the vehicle state gating mapping weight matrix, and W spatial is the integration weight matrix of vehicle interaction features and state features; S23: Perform second-stage depth extraction based on the first-stage spatial feature F spatial Update the deep interaction relationship through the self-attention mechanism and integrate the spatial features Q = F spatial ·W Q , K = F spatial ·W K , V = F spatial ·W V Among them, drop(·) represents the dropout layer, norm(·) represents the normalization layer, Q, K, and V respectively represent the query, key, and value for attention concentration, W Q , W K and W V represent the standard qkv weight matrices, (·) T represents matrix transpose, d k represents the dimension of the vector; is the one-stage spatial interaction feature, is the two-stage initial spatial interaction feature; In addition, in the second-stage spatial feature extraction, the same gated feature fusion mechanism as in the first stage is adopted to ensure the transmission of effective information; S24: Based on the extracted spatial attention features, use the feed-forward network architecture in the transformer network to integrate the individual features of each node; Pass the second-stage spatial features through a feed-forward network composed of a linear layer, an activation function, a linear layer, a normalization layer, and a residual connection; Among them, is the final second-stage spatial interaction feature, W4 and W5 are the vehicle state and vehicle state gating mapping weight matrices respectively, and gelu(·) represents the activation function gelu operation.

5. The dynamic test scenario adversarial generation method according to claim 4, characterized in that, In step S3, using a lightweight time integration module and a quadratic interaction feature extraction module to achieve spatio-temporal feature fusion specifically includes the following steps: S31: Use a lightweight time integration module to integrate time information; Pass the hidden features through parallel multi-scale convolutional blocks, and use convolutional architectures of different scales to initially integrate short-term time features at different fine-grained levels. Each convolutional block consists of two one-dimensional convolutional layers with the same convolutional kernel size and a linear layer. Different from the past, the time dimension is used as the channel dimension of the convolutional layer instead of the feature dimension in the convolutional layer: Among them, are the spatio-temporal features integrated for branch i, Conv0 represents the first convolutional layer, Conv1 represents the second convolutional layer, and W6 is the weight matrix of the spatial interaction feature map; S32: Add the spatio-temporal features initially integrated by different branches to achieve feature unification; Pass the branch features through a structure of addition, normalization layer, residual connection, and dropout layer to achieve simple fusion of features: Among them, represents the initial spatio-temporal feature, represents the feature output by branch one, represents the feature output by branch two, represents the feature output by branch three, W7 is the feature fusion weight matrix, and || represents the concatenation operation; S33: After the parallel convolutional blocks, tile the features along the time dimension, use a linear layer to achieve high-dimensional - low-dimensional - high-dimensional feature transformation, and then combine the normalization layer, residual connection, and dropout layer to implement the overall architecture: Among them, represents the initial low-dimensional spatio-temporal feature, F s-t represents the final spatio-temporal feature, Flatten(·) represents the flattening operation, W8 and W9 are weight matrices for high-dimensional to low-dimensional to high-dimensional conversion, and dim represents the dimension indication of the flattening operation.

6. The dynamic test scenario adversarial generation method according to claim 5, characterized in that In step S4, constructing a trajectory decoding module specifically includes the following steps: S41: Tile the features after sufficient extraction along the time dimension, and use a linear layer to achieve dimensional compression, and integrate the hidden features of each node into independent features for subsequent output decoding: F out = norm(Flatten(F s-t , dim=-2)·W 10 ) Among them, F out represents the complete independent feature of the vehicle node, and W 10 is the weight matrix for information compression; S42: According to the independent features of each node, output the subsequent trajectory content through a parallel linear layer, and divide the output of the subsequent trajectory into two parts, the output of the subsequent heading angle and the output of the subsequent position information; directly decode the sine and cosine values of the subsequent heading angle by passing the features through the linear layer Among them, L out represents the output trajectory length, and W 11 is the decoding weight matrix of the heading angle information; S43: Model the output of position information as an output form considering uncertainty, and establish a unimodal Gaussian distribution to fit the possible positions at each step; Pass the features through a linear layer to directly decode the subsequent trajectory speed information and the standard deviation of each step's horizontal and vertical coordinates respectively: σ = softplus(F out ·W 12 ) V out = F out · W 13 Among them, softplus(·) represents the activation function; W 12 , W 13 are the position error prediction weight matrix and the speed information decoding weight matrix; σ is the standard deviation of the predicted position error of the vehicle in the future, and V out is the speed of the vehicle in the future; S44: Use the speed information and standard deviation to generate position information; Accumulate the speed and the position information at the last moment of the input trajectory to output the position mean, and then generate the final subsequent position information through sampling.

7. The dynamic test scenario adversarial generation method according to claim 6, wherein In step S5, constructing a trajectory discriminator specifically includes the following steps: S51: Design a discriminator model in the MLP architecture according to the trajectory content and the generator network capacity. Similar to the overall architecture of the generator, the discriminator is also composed of a multi-frequency encoder and the MLP network itself. The multi-frequency encoder of the discriminator uses the same number of frequency levels and scaling factors as that of the generator: H emd = Embedding(H) H flag = MLP(H emd ) Among them, H represents the input trajectory, H emd represents the encoded data, and H flag represents the discriminator's judgment result; S52: In the discriminator of the MLP architecture, encode the features of all channels of the trajectory of each node into a hidden feature, and finally output it through a process of gradually compressing from high dimension to low dimension. In addition, an activation function is added between each linear layer.

8. The dynamic test scenario adversarial generation method according to claim 7, wherein In step S5, the loss function of the overall network includes the generator loss and the discriminator loss; the generator loss L G includes the position loss, heading angle loss, trajectory deviation loss of trajectory reconstruction, and adversarial cross-entropy loss; the discriminator loss L D includes adversarial cross-entropy loss; L G = α0 × log(1 - D(G(H))) + α1 × (H real - G(H)) / (B + n) L D = logD(H real ) + log(1 - D(G(H))) Among them, α0 and α1 represent the weights of each loss, G(·) represents the generator, D(·) represents the discriminator, H real represents the true trajectory, H represents the input trajectory of the generator, and B and n represent the batch size of the input data and the number of nodes when calculating the loss each time.

Images