Method, device and equipment for predicting confrontation trajectory of automatic driving vehicle and medium

Abstract

The invention provides an automatic driving vehicle confrontation trajectory prediction method, apparatus and device, and a medium. The method comprises the steps of determining a dynamic interaction space of an automatic driving vehicle and an attack vehicle based on respective driving information of the automatic driving vehicle and the attack vehicle and common lane information of the automatic driving vehicle and the attack vehicle; converting the multi-dimensional information of the dynamic interaction space into a graph structure; extracting a high-dimensional feature vector in the graph structure; and inputting the high-dimensional feature vector into a pre-trained trajectory prediction model, mapping the high-dimensional feature vector into a low-dimensional latent space feature vector by the trajectory prediction model, and decoding the low-dimensional latent space feature vector into a confrontation trajectory of the autonomous vehicle. According to the invention, the precision of trajectory prediction is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method, device, equipment and medium for predicting adversarial trajectories of an autonomous driving vehicle. Background Art

[0002] With the rapid development of autonomous driving technology, ADS (Automotive Driving System) is gradually becoming a core technology in the field of intelligent transportation. However, the safety, reliability and stability of autonomous driving systems remain key issues, especially in complex and changing road environments. Trajectory prediction is an important function in autonomous driving, which involves predicting future trajectories based on information such as the current environment, traffic conditions and vehicle status.

[0003] Traditional trajectory prediction methods mostly rely on simplified models and assumptions, which cannot effectively cope with complex dynamic environments and potential adversarial attacks, resulting in poor robustness of existing adversarial trajectory prediction models and low accuracy of predicted adversarial trajectories. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a method, device, equipment and medium for predicting adversarial trajectories of autonomous driving vehicles to improve the accuracy of adversarial trajectory prediction.

[0005] In a first aspect, a method for predicting adversarial trajectories of an autonomous driving vehicle is provided, the method comprising: The dynamic interaction space between the autonomous driving vehicle and the attacking vehicle is determined based on their respective driving information and the common lane information of the two; the attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle; Convert the multi-dimensional information of dynamic interaction space into a graph structure; Extract high-dimensional feature vectors from graph structures; The high-dimensional feature vector is input into a pre-trained trajectory prediction model so that the trajectory prediction model maps the high-dimensional feature vector into a low-dimensional latent space feature vector, and decodes the low-dimensional latent space feature vector into an adversarial trajectory of the autonomous driving vehicle so that the autonomous driving vehicle adjusts its driving strategy according to the adversarial trajectory.

[0006] Optionally, determining the dynamic interaction space of the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information includes: Obtain driving information of the autonomous vehicle and the attack vehicle, as well as the common lane information of the two; Construct road constraints based on the common lane information of the two; Determine dynamic constraints of the autonomous driving vehicle based on driving information of the autonomous driving vehicle; and determine a dynamic drivable area of ​​the autonomous driving vehicle based on road constraints and dynamic constraints of the autonomous driving vehicle; Determine the intervention strategy constraint of the attacking vehicle based on the driving information of the attacking vehicle, and determine the intervention space of the attacking vehicle based on the road constraint and the intervention strategy constraint; The dynamic interaction space between the two is determined based on the dynamic drivable area of ​​the autonomous driving vehicle and the intervention space of the attacking vehicle.

[0007] Optionally, extracting a high-dimensional feature vector in the graph structure includes: Extract high-dimensional feature vectors from graph structures based on multi-aggregate message passing attention mechanism.

[0008] Optionally, the trajectory prediction model is composed of an encoder and a decoder; the encoder includes a first encoder, a second encoder and a third encoder, and the training process of the first to third encoders includes: The training samples of the time headway between the autonomous driving vehicle and several attacking vehicles are input into the first encoder for regression training, and the parameters of the first encoder are optimized by the first loss function until the preset number of iterations is reached; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; the first loss function is as follows:

[0009] in, is the predicted time headway, is the observed real time headway, is the number of training sessions; The training samples of the vehicle behavior pattern are input into the second encoder for classification training, and the parameters of the second encoder are optimized by the second loss function until the preset number of iterations is reached; the vehicle behavior pattern includes at least straight driving, left turn / left lane change, right turn / right lane change; the second loss function is as follows:

[0010] in, is the real behavior pattern label, is the probability of the predicted behavior pattern; The training samples of the low-level random features of the attack vehicle are input into the third encoder for Gaussian training until a preset number of training iterations is reached, and the low-level random features at least include driving details and environmental interference.

[0011] Optionally, the training process of the first to third encoders further includes: Performing discriminant training on the time headway data generated by the first encoder through the first discriminator; Performing discriminative training on the behavior pattern data generated by the second encoder through a second discriminator; The low-level random features generated by the third encoder are discriminated by the third discriminator.

[0012] Optionally, the training process of the decoder includes: Calculate the center of mass of the intersection space between the autonomous driving vehicle and the attacking vehicle based on the multi-dimensional information of the dynamic interaction space; The fourth loss function is calculated based on the decoded trajectory points and centroids until the preset number of iterations is reached. The fourth loss function is as follows:

[0013] in, represents the adversarial trajectory of the attacking vehicle, is the distance between the attack vehicle trajectory point and the center of mass of the dynamic interaction space.

[0014] In a second aspect, an autonomous driving adversarial trajectory prediction device is provided, the device comprising: a cross modeling module, a graph structure vectorization processing module, a latent space vector mapping module and a trajectory generation module; The cross modeling module is used to determine the dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information; the attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle; The graph structure vectorization processing module is used to convert the multi-dimensional information of the dynamic interaction space into a graph structure and extract the high-dimensional feature vector in the graph structure; The latent space vector mapping module is used to map the high-dimensional feature vector to a low-dimensional latent space vector; The trajectory generation module is used to decode the low-dimensional latent space vector into the adversarial trajectory of the autonomous driving vehicle so that the autonomous driving vehicle can adjust its driving strategy according to the adversarial trajectory.

[0015] Optionally, the latent space vector mapping module is composed of a first encoder, a second encoder and a third encoder; The first encoder is used to extract the distribution characteristics of the time headway between the autonomous driving vehicle and the attacking vehicle; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; The second encoder is used to extract the behavior pattern characteristics of the attacking vehicle, and the behavior pattern includes straight driving, left turn / left lane change, and right turn / right lane change; The third encoder is used to extract low-level random features of the attacking vehicle, where the low-level random features at least include driving details and environmental interference.

[0016] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0017] According to a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0018] The embodiments of the present invention provide a method, device, equipment and medium for predicting the adversarial trajectory of an autonomous driving vehicle. The present invention can extract rich semantic information by modeling the intersection space between the autonomous driving vehicle and the attacking vehicle, which can not only reflect the geometric characteristics of the road, but also describe the dynamic interaction relationship between vehicles; provide rich data support for subsequent trajectory prediction models; and by converting the multi-dimensional information of the dynamic interaction space into graph structured data, the data representation can take into account both local details and global semantics, while facilitating efficient processing of subsequent algorithms; and use an encoder to map high-dimensional feature vectors into low-dimensional latent space vectors, which can not only capture the high-level semantic features of the vehicle, but also include low-level random characteristics, thereby improving the understanding and generation capabilities of trajectory behavior and improving the robustness of the trajectory prediction model.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a method for predicting an adversarial trajectory of an autonomous driving vehicle provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of a device for predicting a confrontation trajectory of an autonomous driving vehicle provided by an embodiment of the present invention is shown; Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0023] Considering that most traditional trajectory prediction methods rely on simplified models and assumptions, they cannot effectively deal with complex dynamic environments and potential adversarial attacks, resulting in low robustness of existing adversarial trajectory prediction models, low precision and poor accuracy of predicted adversarial trajectories. The main problems are: 1. Insufficient use of semantic information: Although the existing adversarial trajectory prediction model has good performance in most scenarios, it does not make full use of semantic information in the algorithm's strategy guidance. As a result, it does not explicitly affect the actual drivable area of ​​the autonomous driving vehicle, weakening the effectiveness of the adversarial trajectory prediction model for autonomous driving.

[0024] 2. Redundancy of high-dimensional feature space: Currently, end-to-end adversarial trajectory prediction models usually use a learning-based approach to optimize model parameters. Existing models often process available trajectory, map and other information into feature vectors during training. However, in fact, high-dimensional feature vectors have more redundant dimensions in expressing significant features, that is, dimensional information with weak correlation also participates in the parameter iteration process. The accuracy of the expression of significant features directly affects the performance of the adversarial trajectory prediction model.

[0025] Based on this, an embodiment of the present invention provides a method and device for predicting adversarial trajectories of an autonomous driving vehicle, which is described below through an embodiment.

[0026] The present invention provides a method for predicting the trajectory of an autonomous driving vehicle. Figure 1 As shown, the method comprises the following steps: Step S101: Determine the dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information. The attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle.

[0027] In a feasible implementation, determining the dynamic interaction space of the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information includes: Step S101A: Acquire driving information of the autonomous driving vehicle and the attacking vehicle, as well as lane information common to both.

[0028] In this step, the posture information of the autonomous driving vehicle includes the position coordinates, heading angle, and speed of the autonomous driving vehicle. In one example, the driving information of the autonomous driving vehicle can be expressed as ;in, is the position coordinate of the autonomous driving vehicle, is the heading angle of the autonomous vehicle, is the speed of the autonomous vehicle. The driving information of the attacking vehicle can be expressed as ,in, is the attack vehicle position coordinate, is the heading angle of the attacking vehicle; is the speed of the attacking vehicle.

[0029] In one example, lane information includes road structure, traffic rules, etc.

[0030] Step S101B: constructing road constraint conditions based on the common lane information of both.

[0031] In this step, road constraints such as lane boundaries, no-drive areas, etc., in one example, the road constraints can be expressed as .

[0032] Step S101C: Determine the dynamic constraints of the autonomous driving vehicle based on the driving information of the autonomous driving vehicle; and determine the dynamic drivable area of ​​the autonomous driving vehicle based on the road constraints and the dynamic constraints of the autonomous driving vehicle.

[0033] In this step, the dynamic feasible area refers to the area where the autonomous vehicle can legally and safely drive at the current moment. This area is constrained by factors such as road structure, traffic rules, and vehicle dynamic performance. The dynamic constraints of the autonomous vehicle are, for example, turning radius, acceleration limit, etc. In one example, the dynamic constraints of the autonomous vehicle can be expressed as ; The dynamic drivable region (DVR) can be expressed as: (1); in, is the road space, including all lane boundaries.

[0034] Step S101D: determining the intervention strategy constraints of the attacking vehicle based on the driving information of the attacking vehicle, and determining the intervention space of the attacking vehicle based on the road constraints and the intervention strategy constraints.

[0035] The Adversarial Intervention Space (AIS) describes the area where the attacking vehicle can potentially interfere with the autonomous vehicle, involving intervention strategies such as lane changing behavior and speed changes.

[0036] The intervention strategy constraint is a lane-changing target area strategy constraint, which predicts the lane-changing intention of the attacking vehicle based on its current state. In one example, the intervention strategy constraint can be expressed as ; Continuing from the previous example, the intervention space can be expressed as: (2); Step S101E: Determine the dynamic interaction space of the two based on the dynamic drivable area of ​​the autonomous driving vehicle and the intervention space of the attacking vehicle.

[0037] Continuing from the previous example, the dynamic interaction space can be expressed as: (3).

[0038] The embodiment of the present invention generates a semantically rich feature representation by cross-modeling the dynamic viable region (DVR) of the autonomous driving vehicle and the adversarial intervention space (AIS) of the attacking vehicle. This representation can reflect the geometric characteristics of the road and describe the dynamic interaction relationship between vehicles, thus achieving full utilization of semantic information.

[0039] Step S102: converting the multi-dimensional information of the dynamic interaction space into a graph structure.

[0040] By converting the complex multidimensional information of the dynamic interaction space J into graph structure data, the graph structure enables the data representation to take into account both local details and global semantics, while facilitating efficient processing by subsequent algorithms.

[0041] In a feasible implementation, the conversion process of the graph structure is as follows: The first step is to build a vertex set in the graph structure , a set of vertices is a discretized position point or dynamic state point in the dynamic interaction space. Let the dynamic interaction space be , generate vertex sets by uniform sampling or semantic importance sampling : ; in Indicates the state of the autonomous vehicle and the attack vehicle status A semantic point generated by the combination of .

[0042] The second step is to build a set of edges between vertices. , if two vertices share the same semantic features (such as being in the same lane), an edge connection relationship is constructed In one example, the semantic dependency determination function can be used To build edge connection relationship ,when When true, The value is 1, otherwise The value is 0. A value of 1 indicates that the two nodes have the same semantic features and an edge connection relationship can be established between the two nodes; A value of 0 means that no edge relationship between two nodes is built.

[0043] If an edge connection is established between two nodes, you can also set a corresponding weight for each edge. Specifically, you can add the following three weights: Add a distance weight to each edge , the distance weight measures the direct distance between the two vehicles; the dynamic change weight , the dynamic change weight measures the change of the vehicle's position in different states; the semantic similarity weight ,when When true, The value is 1; otherwise The value is 0.

[0044] The third step is to obtain the attributes of the vertex, including spatial attributes, dynamic attributes, interference attributes, and semantic attributes. The position of the vertex in the physical space As spatial attributes; the heading and speed of the two vehicles as a dynamic attribute; the interference intensity is the adversarial interference function in the semantic space Indicator, interference intensity as interference attributes; lane number, priority mark, traffic rule status, etc. as semantic attributes.

[0045] Finally, the graph structure is formed ,in is a set of vertices, is a set of edges, with vertex attributes attached and edge attributes .

[0046] Step S103: extracting high-dimensional feature vectors from the graph structure.

[0047] In one feasible implementation, a high-dimensional feature vector is extracted from the graph structure based on the Mamba (Multi-Aggregation Message-BasedAttention) model. Semantic features are extracted from the graph structure to provide a powerful input representation for latent space optimization.

[0048] The Mamba model extracts significant features from the graph structure. Its core idea is to combine multiple aggregation strategies and attention mechanisms to dynamically learn the relationship between vertices and neighbors.

[0049] Specifically, the extraction process is as follows: in each round of propagation, each vertex Receive from neighbors Information , and updates its own status: (4); in, is the vertex In the The characteristics of the layer, is the trainable weight matrix, Vertex The neighbor set of , is the message generation function, is the attention weight, expressed as: (5); in, is the edge of the graph structure; is the number of propagation layers.

[0050] After the propagation is completed, the high-dimensional feature vector of the entire graph is extracted through a global pooling operation. : (6).

[0051] Step S104: Input the high-dimensional feature vector into a pre-trained trajectory prediction model so that the trajectory prediction model maps the high-dimensional feature vector into a low-dimensional latent space feature vector, and decodes the low-dimensional latent space feature vector into an adversarial trajectory of the autonomous driving vehicle, so that the autonomous driving vehicle adjusts its driving strategy according to the adversarial trajectory.

[0052] The low-dimensional latent space feature vector is the core part of the entire autonomous driving trajectory prediction, which aims to represent the high-dimensional information in the vehicle trajectory in a compact and interpretable way. These latent space feature vectors can not only capture the high-level semantic features of the vehicle, but also include low-level random characteristics by combining domain knowledge and statistical distribution modeling, thereby improving the understanding and generation capabilities of trajectory behavior.

[0053] The high-dimensional feature vector is mapped to a low-dimensional latent space feature vector, and the extraction is mainly carried out from three dimensions: the longitudinal dimension, the lateral dimension, and the detail dimension. The longitudinal dimension refers to the change information of the vehicle's position and time in the driving direction, the lateral dimension refers to the change information when the vehicle changes lanes to the left or right, and the detail dimension refers to the vehicle's driving details, surrounding environment and other information.

[0054] In an embodiment of the present invention, the trajectory prediction model is composed of an encoder and a decoder; the encoder includes a first encoder, a second encoder and a third encoder, and the first to third encoders are trained respectively, and the training process includes: Step S104A: Input the training samples of the time headway between the autonomous driving vehicle and the plurality of attacking vehicles into the first encoder for regression training, and optimize the parameters of the first encoder through the first loss function until a preset number of iterations is reached; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; the first loss function is as follows: (7); in, is the predicted time headway, is the observed real time headway, is the number of training times.

[0055] The first encoder is used to extract the longitudinal feature of the vehicle, which is represented by the time headway. In one example, the time headway in an urban driving scenario follows a log-normal distribution. Its probability density function is defined as: (8); in, is the vehicle position parameter, which represents the mean of the normal distribution; is the scale parameter, which represents the standard deviation of the distribution. This function can be used to predict the time headway of the vehicle. Step S104B: input the training samples of the vehicle behavior pattern into the second encoder for classification training, and optimize the parameters of the second encoder by the second loss function until the preset number of iterations is reached; the vehicle behavior pattern at least includes straight driving, left turn / left lane change, right turn / right lane change; the second loss function is as follows: (9); in, is the real behavior pattern label, is the probability of the predicted behavior pattern.

[0056] The second encoder is used to extract the lateral characteristics of the vehicle, which are represented by the vehicle behavior pattern. In one example, the lateral characteristic latent space vector can be extracted by the second encoder. ,in 、 、 Represent the probability of the vehicle being in the three behavior modes respectively.

[0057] Step S104C: input the training samples of the low-level random features of the attacking vehicle into the third encoder for Gaussian training until a preset number of training iterations is reached, wherein the low-level random features at least include driving details and environmental interference.

[0058] In addition to semantic features, there are many low-level random features in vehicle trajectories, such as driving details, environmental interference, etc. These features are often not captured by explicit semantic classification or regression modeling, so the embodiment of the present invention uses a Gaussian encoder G to extract them.

[0059] High-dimensional feature vector Through Gaussian encoder Mapping to a random latent space vector , the latent space vector satisfies the Gaussian distribution: ;in, and is the mean and covariance matrix of the Gaussian distribution, predicted by the Gaussian encoder G.

[0060] In order to ensure the authenticity of the latent space feature vector, an embodiment of the present invention introduces adversarial generation loss for regularization, that is, the false samples and true samples generated by each encoder are input into the discriminator for discrimination, and generative adversarial training is performed to force the latent space feature vector generated by the encoder to be close to the true value.

[0061] Therefore, based on the above embodiment, the training process of the first to third encoders further includes: Step S104D: Perform discrimination training on the time headway data generated by the first encoder through the first discriminator.

[0062] The first discriminator Mainly for time headway Verify that it follows a lognormal distribution: (10); in, Represents the probability density function of the lognormal distribution. The longitudinal latent space distribution generated by the first encoder is constrained by optimizing the following adversarial loss function: (11); in, represents the real distribution of time headway; Represents the distribution of the time headway predicted by the first encoder.

[0063] Step S104E: Perform discrimination training on the behavior pattern data generated by the second encoder through the second discriminator.

[0064] The second discriminator is mainly for the lateral characteristics Verify that it complies with the classification distribution : (12); The lateral characteristics are constrained by a similar adversarial loss function: (13); in, A categorical distribution representing the actual behavior pattern; Represents the categorical distribution of the behavior patterns predicted by the second encoder.

[0065] Step S104F: Perform discriminative training on the low-level random features generated by the third encoder through the third discriminator.

[0066] Random latent space It is a highly random feature extracted by the third encoder from the high-dimensional feature vector, and its target distribution is a standard Gaussian distribution . Discriminator To verify Whether it satisfies the standard Gaussian distribution: (14); The distribution alignment of random latent space vectors is optimized by the following adversarial loss function: (15); in, is the true random latent space vector distribution; Distribution of random latent space vectors predicted by the third encoder.

[0067] The encoder and discriminator of the embodiment of the present invention form an adversarial autoencoder, which is trained by generating an adversarial training mechanism. The trained adversarial autoencoder is used to optimize low-dimensional latent space variables, automatically extract low-dimensional significant features, and force the generated latent space vector to be close to the target distribution. The robustness of adversarial trajectory learning is improved. The trajectory prediction model can better adapt and defend against potential adversarial attacks, thereby enhancing the safety and stability of the autonomous driving system.

[0068] Moreover, by optimizing the low-dimensional latent space vector, not only the convergence speed of adversarial learning of the trajectory prediction model is improved, but also the prediction effect of the model is optimized, the computational cost in the training process is reduced, and the final trajectory prediction accuracy and reliability are improved.

[0069] The embodiment of the present invention uses less dimensional information to express more representative feature information by mapping high-dimensional feature vectors to low-dimensional latent space vectors, thereby reducing the redundancy of high-dimensional feature vectors.

[0070] Based on the above embodiment, the training process of the decoder includes: Step S104G: Calculate the center of mass of the intersection space between the autonomous driving vehicle and the attacking vehicle based on the multi-dimensional information of the dynamic interaction space.

[0071] The centroid position reflects the most representative area of ​​interaction between the dynamic drivable area of ​​the autonomous vehicle DVR and the intervention space AIS of the attacking vehicle, and is the key point that is most likely to interfere with the autonomous vehicle. is the area of ​​the dynamic interaction space, then the calculation formula of the center of mass is: (16); in, is the center of mass; The area is a dynamic interactive space.

[0072] Step S104H: Calculate a fourth loss function based on the decoded trajectory points and the centroid until a preset number of iterations is reached. The fourth loss function is as follows: (17); in, represents the adversarial trajectory of the attacking vehicle, is the distance between the attack vehicle trajectory point and the centroid of the dynamic interaction space. The smaller the distance, the smaller the loss, and the closer the optimized trajectory is to the dry centroid.

[0073] The trajectory points obtained by decoding are the trajectory points of the attacking vehicle. The trajectory optimization goal of the attacking vehicle is to make its trajectory points gradually approach the center of mass position, thereby enhancing the interference effect on the autonomous driving vehicle.

[0074] In another embodiment, after the training is completed, the effectiveness and naturalness of the trajectory predicted by the trajectory prediction model are optimized, and the generated trajectory can be iteratively optimized by a multi-objective optimization method. The optimization objectives include the interference effect of the adversarial trajectory on the autonomous driving vehicle and the physical feasibility, smoothness and human-like driving characteristics (naturalness) of the adversarial trajectory to ensure that the adversarial trajectory meets the actual application requirements. After the optimization is completed, the effectiveness and naturalness of the trajectory can be evaluated separately. Specifically, the evaluation of effectiveness includes quantitative indicators such as path deviation, speed change and safety impact. The evaluation of naturalness includes evaluating whether the generated adversarial trajectory complies with physical constraints and traffic rules to ensure that the trajectory is smooth, continuous and realistically executable.

[0075] By optimizing and evaluating the effectiveness of the trajectory, we ensure that the generated adversarial trajectory has attack effectiveness in practical applications, thereby improving the safety and robustness evaluation capabilities of the autonomous driving system. By optimizing and evaluating the naturalness of the trajectory, we ensure that the trajectory has natural driving characteristics in terms of rationality and reality. We ensure that the adversarial trajectory has both interference effects and is close to the real scene.

[0076] In addition, the virtual-reality combined evaluation system can be used to conduct simulation tests and real-vehicle verification on the adversarial trajectories after evaluation and screening, comprehensively evaluate the effectiveness and transferability of the adversarial testing method in safety-critical scenarios, and provide a reliable basis for the safety testing of autonomous driving systems.

[0077] It can be understood from the above embodiments that the present invention can extract rich semantic information by modeling the intersection space of the autonomous driving vehicle and the attacking vehicle, which can not only reflect the geometric characteristics of the road, but also describe the dynamic interaction relationship between vehicles; provide rich data support for the subsequent trajectory prediction model; and by converting the multi-dimensional information of the dynamic interaction space into graph structured data, the data representation can take into account both local details and global semantics, while facilitating efficient processing of subsequent algorithms; by using an encoder to map high-dimensional feature vectors into low-dimensional latent space vectors, it can not only capture the high-level semantic features of the vehicle, but also include low-level random characteristics, thereby improving the understanding and generation capabilities of trajectory behavior and improving the robustness of the trajectory prediction model. Based on the same inventive concept, an autonomous driving confrontation trajectory prediction device is provided, such as Figure 2 As shown, the device includes: a cross modeling module 201, a graph structure vectorization processing module 202, a latent space vector mapping module 203 and a trajectory generation module 204; The intersection modeling module 201 is used to determine the dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information; the attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle; The graph structure vectorization processing module 202 is used to convert the multi-dimensional information of the dynamic interaction space into a graph structure and extract the high-dimensional feature vector in the graph structure; The latent space vector mapping module 203 is used to map the high-dimensional feature vector to a low-dimensional latent space vector; The trajectory generation module 204 is used to decode the low-dimensional latent space vector into an adversarial trajectory of the autonomous driving vehicle so that the autonomous driving vehicle adjusts its driving strategy according to the adversarial trajectory.

[0078] Based on the above embodiment, the latent space vector mapping module 203 is composed of a first encoder, a second encoder and a third encoder; The first encoder is used to extract the distribution characteristics of the time headway between the autonomous driving vehicle and the attacking vehicle; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; The second encoder is used to extract the behavior pattern characteristics of the attacking vehicle, and the behavior pattern includes straight driving, left turn / left lane change, and right turn / right lane change; The third encoder is used to extract low-level random features of the attacking vehicle, where the low-level random features at least include driving details and environmental interference.

[0079] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Figure 3 As shown, it includes a processor 301 , a communication interface 302 , a memory 303 and a communication bus 304 , wherein the processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0080] Memory 303, used for storing computer programs; The processor 301 is used to implement the steps of the autonomous driving vehicle adversarial trajectory prediction method when executing the program stored in the memory 303.

[0081] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0082] The communication interface is used for communication between the above electronic device and other devices.

[0083] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0084] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0085] The computer program product for the method for predicting adversarial trajectories of autonomous driving vehicles provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0086] The device for countering trajectory prediction of an autonomous driving vehicle provided in an embodiment of the present invention may be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in an embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for predicting adversarial trajectories of an autonomous driving vehicle, characterized in that: The method comprises: Determining a dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information; the attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle; Converting the multidimensional information of the dynamic interaction space into a graph structure; Extracting high-dimensional feature vectors from the graph structure; The high-dimensional feature vector is input into a pre-trained trajectory prediction model so that the trajectory prediction model maps the high-dimensional feature vector into a low-dimensional latent space feature vector, and decodes the low-dimensional latent space feature vector into an adversarial trajectory of the autonomous driving vehicle, so that the autonomous driving vehicle adjusts its driving strategy according to the adversarial trajectory.

2. The method according to claim 1, characterized in that The determining of the dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and their common lane information includes: Obtain driving information of the autonomous vehicle and the attack vehicle, as well as the common lane information of the two; Construct road constraints based on the common lane information of the two; Determine a dynamic constraint condition of the autonomous driving vehicle based on the driving information of the autonomous driving vehicle; and determine a dynamic drivable area of ​​the autonomous driving vehicle based on the road constraint condition and the dynamic constraint condition of the autonomous driving vehicle; Determining an intervention strategy constraint condition of the attacking vehicle based on the driving information of the attacking vehicle, and determining an intervention space of the attacking vehicle based on the road constraint condition and the intervention strategy constraint condition; The dynamic interaction space of the two is determined based on the dynamic drivable area of ​​the autonomous driving vehicle and the intervention space of the attacking vehicle.

3. The method according to claim 2, characterized in that The extracting of the high-dimensional feature vector in the graph structure comprises: A high-dimensional feature vector is extracted from the graph structure based on a multi-aggregate message passing attention mechanism.

4. The method according to claim 1, characterized in that The trajectory prediction model is composed of an encoder and a decoder; the encoder includes a first encoder, a second encoder and a third encoder, and the training process of the first to third encoders includes: The training samples of the time headway between the autonomous driving vehicle and several attacking vehicles are input into the first encoder for regression training, and the parameters of the first encoder are optimized by the first loss function until a preset number of iterations is reached; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; the first loss function is as follows: in, is the predicted time headway, is the observed real time headway, is the number of training sessions; The training samples of the vehicle behavior pattern are input into the second encoder for classification training, and the parameters of the second encoder are optimized by the second loss function until a preset number of iterations is reached; the vehicle behavior pattern includes at least straight driving, left turn / left lane change, and right turn / right lane change; the second loss function is as follows: in, is the real behavior pattern label, is the probability of the predicted behavior pattern; The training samples of the low-level random features of the attack vehicle are input into the third encoder for Gaussian training until a preset number of training iterations is reached, and the low-level random features at least include driving details and environmental interference.

5. The method according to claim 4, characterized in that The training process of the first to third encoders also includes: Performing discriminant training on the time headway data generated by the first encoder through a first discriminator; Performing discriminant training on the behavior pattern data generated by the second encoder through a second discriminator; The low-level random features generated by the third encoder are discriminated and trained by a third discriminator.

6. The method according to claim 5, characterized in that The training process of the decoder includes: Calculate the center of mass of the intersection space between the autonomous driving vehicle and the attacking vehicle based on the multi-dimensional information of the dynamic interaction space; A fourth loss function is calculated based on the decoded trajectory points and the centroid until a preset number of iterations is reached. The fourth loss function is as follows: in, represents the adversarial trajectory of the attacking vehicle, is the distance between the attack vehicle trajectory point and the center of mass of the dynamic interaction space.

7. An autonomous driving confrontation trajectory prediction device, characterized in that: The device comprises: a cross modeling module, a graph structure vectorization processing module, a latent space vector mapping module and a trajectory generation module; The intersection modeling module is used to determine the dynamic interaction space between the autonomous driving vehicle and the attacking vehicle based on their respective driving information and the common lane information of the two; the attacking vehicle refers to a vehicle that poses a driving risk to the autonomous driving vehicle; The graph structure vectorization processing module is used to convert the multi-dimensional information of the dynamic interaction space into a graph structure and extract the high-dimensional feature vector in the graph structure; The latent space vector mapping module is used to map the high-dimensional feature vector into a low-dimensional latent space vector; The trajectory generation module is used to decode the low-dimensional latent space vector into an adversarial trajectory of the autonomous driving vehicle so that the autonomous driving vehicle adjusts its driving strategy according to the adversarial trajectory.

8. The autonomous driving confrontation trajectory prediction device according to claim 7, characterized in that: The latent space vector mapping module is composed of a first encoder, a second encoder and a third encoder; The first encoder is used to extract the distribution characteristics of the time headway between the autonomous driving vehicle and the attacking vehicle; the time headway refers to the time difference when the head of the autonomous driving vehicle and the attacking vehicle pass the same position point; The second encoder is used to extract the behavior pattern characteristics of the attacking vehicle, and the behavior pattern includes going straight, turning left / changing lanes left, and turning right / changing lanes right; The third encoder is used to extract low-level random features of the attacking vehicle, where the low-level random features at least include driving details and environmental interference.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

Images