A criticality quantification method for intelligent vehicle test scenarios based on spatiotemporal hypergraph learning

Abstract

The present invention relates to a method for quantifying the criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning, which uses a hypergraph model to construct the spatial characteristics of the intrinsic relationship between multi-dimensional scene elements, and uses a gated recurrent unit to learn the timing information of all elements in the intelligent vehicle test scenario, thereby realizing the criticality quantification of all-dimensional complex test scenario elements. It can improve test efficiency and reduce test costs. The hypergraph structure constructed by the present invention takes into account the dynamics, directionality, spatiotemporal nature and heterogeneity of the graph.

Description

Technical Field

[0001] The present invention relates to a method for screening intelligent vehicle test scenarios, and in particular to a criticality quantification method for complex intelligent vehicle test scenarios based on spatiotemporal hypergraph learning. Background Art

[0002] Smart cars are an important development direction of the future automotive industry. Major manufacturers have conducted in-depth research on smart cars. However, there are still a large number of traffic accidents involving smart cars. The reason is that smart cars have not been effectively and completely tested, which leads to their inability to handle complex road environments. Therefore, it is of great significance to fully test the functions of smart cars. Scenario-based testing methods have now become an important means of testing smart cars. The functional completeness of smart cars is determined by the responses of smart cars in specific test scenarios. However, the actual amount of test scenario data is extremely large. If the collected scenarios are fully used for testing, the test efficiency will be extremely low and the test cost will be high. Therefore, screening key test scenarios is essential for testing smart cars.

[0003] The existing test scenario criticality quantification methods have the following shortcomings: First, the existing methods cannot use a unified framework to comprehensively quantify the multi-dimensional scene elements of smart cars. For example, TTC lacks scene elements that consider weather and road traffic conditions; Second, the existing methods only take the feature values ​​of multi-dimensional scene elements as input, without specifically analyzing the intrinsic relationship between different scene element dimensions, which leads to the lack of effective data features. In addition, since the intrinsic relationship belongs to non-Euclidean data, it cannot be processed by traditional machine learning or convolutional neural networks; Finally, the graph structure corresponding to the test scenario is dynamic, directed, spatiotemporal and heterogeneous, and the existing graph neural network cannot model this type of graph data structure. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for quantifying the criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning, comprising the following steps:

[0005] (1) Collect complex meteorological environment data through meteorological data collection equipment; collect traffic participant characteristic data and road structure characteristic data through drone images;

[0006] (2) In the node feature parameter extraction module, the image Img collected by the drone is t The node feature parameters are extracted from the meteorological environment data collected by the meteorological collection equipment, where t represents the collected scene duration in seconds, and three different dimensions of node feature information are obtained, namely:

[0007] Meteorological dimension node feature N W ;

[0008] Road dimension node feature N R ;

[0009] Traffic dimension node feature N T,i ;

[0010] (3) The node characteristic parameters are input into the hyperedge characteristic parameter calculation module. The constructed hyperedges are divided into three types: meteorological, road, and traffic hyperedges. The hyperedge characteristic parameter calculation module calculates the three types of hyperedges corresponding to each time t. The values ​​are expressed as meteorological hyperedge SG W , Road over edge SG R and traffic super edge SG T ;

[0011] (3.1) The meteorological super-edge SG W It mainly describes the impact of complex meteorological environment on roads and traffic. The impact of weather on roads is mainly reflected in the change of road adhesion coefficient, and the impact on traffic is reflected in the functional level of intelligent vehicle perception system;

[0012] (3.2) The road superedge SG R Including the impact risk caused by static obstacles E sta and the risk E of the lane line affecting each position on the road R ;

[0013] (3.3) The traffic super-edge SG T Describe the interactive risk generated by different traffic participants, determine it according to the position (x, y) of different traffic participants, consider the motion field strength generated by all other traffic participants in front of the corresponding position of the traffic participant, and consider the motion parameters of the vehicle to output the risk value of the corresponding traffic participant. The risk values ​​of all traffic participants in the scene are calculated by the above method, and the summation result is recorded as the traffic hyperedge SG T The value of

[0014] (4) Using meteorological dimension node features, road dimension node features, traffic dimension node features, and meteorological hyperedge SG W , Road over edge SG R and traffic super edge SG T , a hypergraph structure is constructed in the hypergraph construction module; the hypergraph is defined as in represents the node feature set at time t, ε t represents the set of hyperedges at time t, W t The diagonal matrix representing the hyperedge weights at time t;

[0015] Defining the Incidence Matrix Represents the number of nodes in the scene, |εt | represents the number of hyperedges when constructing a hypergraph, and H is a matrix describing whether a node belongs to a specified hyperedge, where each element h is expressed as:

[0016]

[0017] In the formula, v represents a node and e represents a hyperedge. At the same time, the node degree matrix is ​​defined as The diagonal elements of for:

[0018]

[0019] At the same time, we define the hyperedge degree matrix D e The diagonal elements d(e) are:

[0020]

[0021] (5) The constructed hypergraph structure is input into the spatial feature information extraction module. This module is based on the spectral domain convolution theory of ordinary graphs. It integrates the information of neighboring nodes through hyperedges and updates the features of the nodes to achieve spatial feature extraction.

[0022] (6) The output of the spatial feature information extraction module is input to the temporal feature information extraction module. The temporal feature information extraction module uses a gated recurrent unit network (GRU) and takes the current input x as input. t and the hidden state H passed down from the previous node t-1 , this hidden state contains the relevant information of the previous node, combined with x t and H t-1 , the GRU network will get the output Y of the current hidden node t and the hidden state H passed to the next node t ;

[0023] (7) The input of the test scene classification module is Y t , and then output the category of the test scenario. The test scenario category is divided into high, medium and low risk labels. The true value of the label needs to be obtained through a driving simulator and driver experiments.

[0024] Furthermore, in step (1), during the image acquisition process by the drone, the pixel-distance mapping coefficient δ (unit: m / pixel) of the drone needs to be confirmed. The coefficient is determined by the drone flight altitude H, the camera longitudinal field of view fov and the image longitudinal resolution Pix. The expression of δ is:

[0025]

[0026] The video stream is converted into feature data of traffic participants and road structures through δ.

[0027] Furthermore, in step (2), the meteorological dimension node feature N W :[Rain,Vis], the nodes of the meteorological dimension will not change with time. Rain is the rainfall intensity in mm / h; Vis is the fog visibility in m;

[0028] Road dimension node feature N R :[n r ,RL,μ,ε l ,(x s ,y s )], the road dimension node features do not have time series, that is, the structural features of the road will not change with time; where n r is the number of lanes, in units of lanes; RL is the lane line curve expression; μ is the road adhesion coefficient; εl 为 Lane lines are linear, including dashed lines and solid lines; (x s ,y s ) is the static obstacle position information, in m;

[0029] Traffic dimension node feature N T,i :[x,y,v x ,v y ,a x ,a y ],T∈t,i∈(1,2,3…n v ),n v Represents the number of traffic participants in the traffic dimension, N T,i Describes the characteristic information of the i-th vehicle at time T; where (x, y) is the location information of the traffic participant, in units of m; v x ,v y are the lateral and longitudinal speeds of traffic participants, in m / s; a x ,a y are the lateral and longitudinal accelerations of traffic participants, in m / s 2 .

[0030] Furthermore, in step (3.1), the hyperedge feature modeling method for the weather-to-traffic dimension and the hyperedge feature modeling method for the weather-to-road dimension include:

[0031] 3.1.1 Hyperedge feature modeling for weather and traffic dimensions:

[0032] The hyperedge feature of the meteorological dimension of traffic mainly describes the changes in the perception and recognition errors of intelligent vehicles caused by complex meteorological environments, collects the accuracy of the corresponding perception results under the conditions of two vehicles, and then fits the three-dimensional surface f W (); f W The input of () is N T The vehicle position (x, y) and N W The complex meteorological parameters in the output are the accuracy of the perception function of the smart car under the distance and position conditions, f W () Parameter calibration is required for different perception schemes;

[0033] Meteorological super edge feature SG W The expression is:

[0034] SG W =exp[-f W (N T ,N W )]

[0035] 3.1.2 Hyperedge feature modeling for weather-related road dimensions:

[0036] In rainy conditions, the road adhesion coefficient μ in the road dimension will be affected. The value of the road adhesion coefficient after the change due to these two meteorological environments is defined as μ W In a rainy weather environment, water on the road creates a water film between the tire and the road surface, which reduces the distance μ between the vehicle and the road surface. The amount of reduction depends on the thickness of the water film h. R The size of h R The relationship with Rain is:

[0037] h R =(60·Rain) 0.7786

[0038] μ W It is expressed as:

[0039] μ W =0.6603-0.0036v y -0.0057h R

[0040] The maximum braking deceleration a under a specific rainfall intensity is r max Expressed as:

[0041] a r max =εμ W g

[0042] Where ε is the degree of tire wear relative to a r maxThe influence coefficient of g is gravitational acceleration.

[0043] Furthermore, in step (3.2), the road superedge SG R The impact on the vehicle includes two parts, one is the static obstacle and the other is the lane line;

[0044] Risk E for static obstacles sta It is expressed as:

[0045]

[0046] R is the road condition influencing factor; Dis ij is the Euclidean distance between vehicle j and static obstacle i; N sta Represents the number of static obstacles; N represents the number of vehicles in the scene;

[0047] The characteristics of the lane line will constrain the vehicle movement. Input N R , using the road potential field method to calculate the risk E of each position on the road R Describe, E R The specific expression is:

[0048] E R =ε l exp(-r l 2 / 2σ 2 )

[0049] In the formula, ε l According to the linear determination of different lane lines, the ε corresponding to the solid lane line l The value is greater than the value corresponding to the dotted line; r l For different vehicles in N R The lateral distance between the center axis of the vehicle and the lane line, in meters; σ is a constant that determines the speed at which the safety risk increases when the vehicle approaches the lane line;

[0050] SG R The value of is obtained by adding the above two parts to get the final value, which is expressed as:

[0051] SG R =E R +E sta .

[0052] Furthermore, in step (3.3), the traffic superedge SG T The calculation methods include:

[0053] The risk field E generated by vehicle i i The expression is:

[0054]

[0055] Where R is the road condition influencing factor; D ij is the Euclidean distance between vehicle j and vehicle i, D / |D| represents the direction of the Euclidean distance; v i is the speed of vehicle i, θ ij v represents vehicle i i The angle between the direction and the distance D between vehicles i and j; when there are N vehicles in the scene, SG T The calculation process is expressed as:

[0056]

[0057] Furthermore, in step (5), the spatial feature information extraction module is based on the spectral domain convolution theory of ordinary graphs, using spectral convolution The input X signal is processed and converted into the Fourier domain. In order to reduce the number of parameters to alleviate overfitting and reduce computational complexity, the formula is obtained:

[0058]

[0059] The spatial feature information extraction module transforms the input feature X into a set of new features Where F is the number of output features of each node, expressed as:

[0060]

[0061] Where Θ refers to the filter parameter matrix learned during training and is the matrix of parameters to be learned.

[0062] Furthermore, the process of step (6) includes:

[0063] First, by transmitting the state H t-1 and input x t To obtain the gate state, where R t is the reset gate, Z t Is the update gate:

[0064] R t =σ(x t W xr +H t-1 W hr +b r )

[0065] Z t =σ(x t W xz +Ht-1 W hz +b z )

[0066] Among them, σ( ) represents the sigmoid function, Transform the data to a value in the range of 0-1;

[0067] After getting the gate signal, use the reset gate to get the data h t-1' =H t-1 ⊙R t , the data is scaled to between -1 and 1 through tanh; ⊙ represents the multiplication of the corresponding elements in the matrix, where, ; The specific expression is:

[0068]

[0069] Finally use Z t Perform the "Update Memory" phase:

[0070]

[0071] Y t Represents the output vector at each moment, which not only integrates the spatial information of the hypergraph but also extracts the features of the dynamically changing timing information of each node.

[0072] Furthermore, in step (7), the test scene is divided into high, medium and low risk category labels. The true value of the label needs to be obtained through a driving simulator and driver experiments. The trajectory data collected by the drone is arranged in the virtual simulation software according to the true value, and the interactive vehicle is made to move according to the real trajectory. Then the image of the forward perspective is played back, and the scene is classified according to the driver's description in the scene, and the true value of the classification label is obtained by this method; in the test scene classification module, the fully connected layer is used to classify Y t After processing, the [3*1] vector is output and normalized to obtain the classification result of the model.

[0073] Beneficial effects of the present invention:

[0074] The test scenario criticality quantification method proposed in the present invention is a method for screening test scenarios, which can improve test efficiency and reduce test costs. The present invention uses a hypergraph model to construct the spatial characteristics of the intrinsic relationship between multi-dimensional scene elements, and uses a gated recurrent unit to learn the timing information of all elements in the intelligent vehicle test scenario, thereby realizing the criticality quantification of all-dimensional complex test scenario elements.

[0075] The hypergraph structure constructed by the present invention takes into account the dynamics, directionality, spatiotemporality and heterogeneity of the graph. The dynamics are mainly reflected in the fact that the connection structure of the graph will change with the spatial position of the traffic participants. For example, the vehicle in front will cause safety hazards to the vehicle behind. When the vehicle behind exceeds the vehicle in front, the direction of the safety hazard changes, and the corresponding constructed graph structure also changes; Directionality refers to the directionality of the impact of the scene elements in the test scene. For example, meteorological elements can only affect the characteristics of the road in one direction, while the linear characteristic changes of static obstacles and lane lines in the road will not affect the meteorological elements; Spatiotemporality refers to the fact that the elements in the entire test scene not only have intrinsic characteristics in space, but also these characteristics will change in time; Heterogeneity refers to the different types of impacts on each other produced by different scene elements. For example, the meteorological environment has different impacts on traffic participants and roads respectively. For traffic participants, it is mainly manifested in the impact on the perceived target object, while for roads, it is mainly manifested in the decrease in the adhesion coefficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0077] Figure 2 A schematic diagram of traffic participants and road information collected by the drone of the present invention;

[0078] Figure 3 is the f of different rainfall intensities and distances between two vehicles in the present invention. W () Surface schematic diagram;

[0079] Figure 4 This is a schematic diagram of the road super-edge of the present invention;

[0080] Figure 5 A schematic diagram of the impact direction of the traffic super edge of the present invention;

[0081] Figure 6 It is a schematic diagram of a hypergraph construction module of the present invention;

[0082] Figure 7 Schematic diagram of the structure of the GRU-gated recurrent unit of the present invention;

[0083] Figure 8 It is a schematic diagram of the overall framework of the model of the present invention. DETAILED DESCRIPTION

[0084] See also Figure 1 As shown, this embodiment provides a method for quantifying the criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning, comprising the following steps:

[0085] (1) Collect complex meteorological environment data through meteorological data collection equipment; collect traffic participant characteristic data and road structure characteristic data through drone images;

[0086] In this embodiment, the meteorological environment data is obtained by a graduated cylinder and a fog visibility measuring instrument. During the image acquisition process by a drone, the pixel-distance mapping coefficient δ (unit: m / pixel) of the drone needs to be confirmed. The coefficient is determined by the drone flight altitude H, the camera longitudinal field of view fov and the image longitudinal resolution Pix. The expression of δ is:

[0087]

[0088] The video stream is converted into feature data of traffic participants and road structures through δ; Figure 2 As shown, the captured images include information about traffic participants and road structure.

[0089] (2) In the node feature parameter extraction module, the image Img collected by the drone is t , where t represents the collected scene duration in seconds, and the meteorological environment data collected by the meteorological collection equipment are used to extract node feature parameters to obtain node feature information of three different dimensions, namely:

[0090] Meteorological dimension node feature N W :[Rain,Vis], the nodes of the meteorological dimension will not change with time. Rain is the rainfall intensity in mm / h; Vis is the fog visibility in m;

[0091] Road dimension node feature N R :[n r ,R L ,μ,ε l ,(x s ,y s )], the road dimension node features do not have time series, that is, the structural features of the road will not change with time; where n r is the number of lanes, in units of lanes; RL is the lane line curve expression; μ is the road adhesion coefficient; ε l Lane line linearity, including dotted lines and solid lines; (x s ,y s ) is the static obstacle position information, in m;

[0092] Traffic dimension node feature N T,i :[x,y,v x ,v y ,a x ,a y],T∈t,i∈(1,2,3…n v ),n v Represents the number of traffic participants in the traffic dimension, N T,i Describes the characteristic information of the i-th vehicle at time T; where (x, y) is the location information of the traffic participant, in units of m; v x ,v y are the lateral and longitudinal speeds of traffic participants, in m / s; a x ,a y are the lateral and longitudinal accelerations of traffic participants, in m / s 2 ;

[0093] In this embodiment, the features of traffic participants and road structure nodes are obtained through the GroundTruthLabeler toolkit in Matlab, which can realize self-tracking of target objects to ensure the temporal nature of feature data.

[0094] (3) The node feature parameters are input to the hyperedge feature parameter calculation module. The hyperedges constructed by the present invention are divided into three types of hyperedges: meteorological, road, and traffic. The hyperedges aggregate the features of different nodes and then describe the constructed graph structure in a higher dimension. The hyperedge feature parameter calculation module calculates the three types of hyperedges corresponding to each time t, and the values ​​are respectively expressed as meteorological hyperedge SG W , Road over edge SG R and traffic super edge SG T ;

[0095] (3.1) The meteorological super-edge SG W It mainly describes the impact of complex meteorological environment on roads and traffic. The impact of weather on roads is mainly reflected in the change of road adhesion coefficient, and the impact on traffic is reflected in the functional level of intelligent vehicle perception system;

[0096] 3.1.1 Hyperedge feature modeling for weather and traffic dimensions:

[0097] The hyperedge feature of the meteorological effect on the traffic dimension mainly describes the changes in the perception and recognition errors of intelligent vehicles caused by complex meteorological environments. The present invention collects the corresponding perception result accuracy under different distances between two vehicles and rainfall intensities, and then fits the three-dimensional surface f W (); f W The input of ( ) is N T The vehicle position (x, y) and N W The complex meteorological parameters in the output are the accuracy of the perception function of the smart car under the distance and position conditions, f W ()Parameter calibration is required for different perception schemes.

[0098] This embodiment uses a camera sensor as an example to analyze different rainfall intensities and distances between two vehicles. W () parameter calibration, the result is as follows Figure 3 As shown, the final meteorological hyperedge feature SG W The expression is:

[0099] SG W =exp[-f W (N T ,N W )] (2)

[0100] 3.1.2 Hyperedge feature modeling for weather-related road dimensions:

[0101] In rainy conditions, the road adhesion coefficient μ in the road dimension will be affected. In this embodiment, the value of the road adhesion coefficient after the change due to these two meteorological environments is defined as μ W In a rainy weather environment, water on the road creates a water film between the tire and the road surface, which reduces the distance μ between the vehicle and the road surface. The amount of reduction depends on the thickness of the water film h. R The size of h R The relationship with Rain is:

[0102] h R =(60·Rain) 0.7786 (3)

[0103] And μ W It is expressed as:

[0104] μ W =0.6603-0.0036v y -0.0057h R (4)

[0105] The maximum braking deceleration a under a specific rainfall intensity is r max Expressed as:

[0106] a r max =εμ W g (5)

[0107] Where ε is the degree of tire wear relative to a r max The influence coefficient is usually 0.9; g is the acceleration of gravity, g = 9.8m / s 2 .

[0108] (3.2) The road superedge SG R The impact on the vehicle includes two parts, one is the static obstacle and the other is the lane line, such as Figure 4 As shown;

[0109] For static obstacles, the risk E sta It is expressed as:

[0110]

[0111] R is the road condition influencing factor. In this embodiment, R=exp(|μ W -μ|); Dis ij is the Euclidean distance between vehicle j and static obstacle i; N sta Represents the number of static obstacles; N represents the number of vehicles in the scene;

[0112] The characteristics of the lane line will constrain the vehicle movement. Input N R , using the road potential field method to calculate the risk E of each position on the road R Describe, E R The specific expression is:

[0113] E R =ε l exp(-r l 2 / 2σ 2 ) (7)

[0114] In the formula, ε l According to the linear determination of different lane lines, the ε corresponding to the solid lane line l The value is greater than the value corresponding to the dotted line; r l For different vehicles in N R The lateral distance between the center axis of the vehicle and the lane line, in meters; σ is a constant that determines the speed at which the safety risk increases when the vehicle approaches the lane line;

[0115] SG R The value of is obtained by adding the above two parts to get the final value, which is expressed as:

[0116] SG R =E R +E sta (8)

[0117] (3.3) The traffic super-edge SG T Describes the interaction risk generated by different traffic participants. This risk is directional and needs to be determined based on the positions (x, y) of different traffic participants. TWhen calculating the value of , it is necessary to consider the motion field strength generated by all other traffic participants in front of the corresponding position of the traffic participant, and at the same time consider the risk value of the corresponding traffic participant output by the motion parameter of the vehicle. The risk values ​​of all traffic participants in the scene are calculated by the above method, and the summation result is recorded as the traffic hyperedge SG T The specific calculation method is as follows:

[0118] like Figure 5 As shown in Figure 1, the risk field generated by vehicle 1 will affect vehicles 2 and 3, while the risk field generated by vehicle 3 will not affect vehicle 1. i The expression is:

[0119]

[0120] Wherein, R is the road condition influencing factor. In this embodiment, R=exp(|μ W -μ|); D ij is the Euclidean distance between vehicle j and vehicle i, D / |D| represents the direction of the Euclidean distance; v i is the speed of vehicle i, θ ij v represents vehicle i i The angle between the direction and the distance j between vehicles i and j. According to formula (9), the risks generated by vehicle 1, vehicle 2 and vehicle 3 are constructed respectively. When the longitudinal position of the vehicle is Figure 5 As shown, the risk calculation for vehicle 3 is:

[0121] First determine E1, E2, and E3;

[0122] Then, use the N of vehicle 3 respectively T The risk value of other vehicles to vehicle 3 is calculated by E1 and E2;

[0123] Finally, using the N of vehicles 1 and 2 T And E3 calculates the risk value of vehicle 3 to other vehicles, and the above two values ​​are added together to obtain the final risk value of vehicle 3.

[0124] In actual calculations, it is found that when there are N vehicles in the scene, the above calculation of SG T The process can be expressed as:

[0125]

[0126] (4) Figure 6 As shown in the figure, in the hypergraph construction module, the meteorological dimension node features, road dimension node features, traffic dimension node features and meteorological hyperedge SG W , Road over edge SG Rand traffic super edge SG T , constructing a hypergraph structure.

[0127] The hypergraph constructed by the present invention is defined as in represents the node feature set at time t, ε t represents the set of hyperedges at time t, W t The diagonal matrix representing the hyperedge weights at time t, in this embodiment, W t is a 3*3 vector, and the diagonal elements are {SG W ,SG R ,SG T}, the hyperedge weight at each moment will also change. In the following description, t will be omitted to simplify the expression.

[0128] Defining the Incidence Matrix Represents the number of nodes in the scene, |ε t | represents the number of hyperedges when constructing a hypergraph. In this embodiment, |ε t |=3, H is a matrix describing whether a node belongs to a specified hyperedge, where each element h is expressed as:

[0129]

[0130] In the formula, v represents a node and e represents a hyperedge. At the same time, the node degree matrix is ​​defined as The diagonal elements of for:

[0131]

[0132] At the same time, we define the hyperedge degree matrix D e The diagonal elements d(e) are:

[0133]

[0134] (5) The constructed hypergraph structure is input into the spatial feature information extraction module, which integrates the information of neighboring nodes through hyperedges and updates the features of the nodes to realize spatial feature extraction;

[0135] Based on the spectral domain convolution theory of ordinary graphs, the present invention uses spectral convolution Process the input X signal and transform it into the Fourier domain:

[0136]

[0137] In order to determine the value of the feature matrix Φ, it is necessary to obtain it through the standard normalized hypergraph matrix θ. The expression of θ is:

[0138]

[0139] The Laplacian matrix of a hypergraph is defined as The Δ matrix is ​​a semi-positive definite matrix, and I is a matrix of size The identity matrix of , so the spectral decomposition can be performed:

[0140] Δ=ΦΛΦ T (16)

[0141] represents a diagonal matrix consisting of non-negative eigenvalues; standard orthogonal eigenvector Φ. In practice, in order to reduce the number of parameters to alleviate overfitting and reduce computational complexity, formula (14) is simplified to:

[0142]

[0143] The spatial feature information extraction module transforms the input feature X into a set of new features Where F is the number of output features of each node, so the expression is:

[0144]

[0145] Where Θ refers to the filter parameter matrix learned during training, which is the matrix of parameters to be learned. In fact, Q also has time characteristics, and each t moment corresponds to a value.

[0146] (6) Feature Q is the output of the spatial feature information extraction module and is input to the temporal feature information extraction module. The temporal feature information extraction module uses a gated recurrent unit network (GRU) and takes the current input x as input. t and the hidden state H passed down from the previous node t-1 , this hidden state contains relevant information of the previous node, such as Figure 7 As shown. Combined with x t and H t-1 , the GRU network will get the output Y of the current hidden node t and the hidden state H passed to the next node t .

[0147] First, by transmitting the state H t-1 and input x t To obtain the gate state, where R t is the reset gate, Z t Is the update gate:

[0148] R t =σ(x t W xr +H t-1 W hr +b r ) (19)

[0149] Z t =σ(x t W xz +H t-1 W hz +b z ) (20)

[0150] Among them, σ( ) represents the sigmoid function, Transform the data to a value in the range of 0-1;

[0151] After getting the gate signal, use the reset gate to get the data h t-1' =H t-1 ⊙R t , the data is scaled to between -1 and 1 through tanh; ⊙ represents the multiplication of the corresponding elements in the matrix, where, ; The specific expression is:

[0152]

[0153] Finally use Z t Perform the "Update Memory" phase:

[0154]

[0155] Y t Represents the output vector at each moment, which not only integrates the spatial information of the hypergraph but also extracts the features of the dynamically changing timing information of each node.

[0156] (7) The input of the test scene classification module is Y t , and then output the category of the test scene. The present invention divides the test scene into high, medium and low risks. The true value of the label needs to be obtained through a driving simulator and driver experiments. The trajectory data collected by the drone is arranged in the virtual simulation software according to the true value, and the interactive vehicles are moved according to the real trajectory. Then the image of the forward perspective is played back, and the scene is classified according to the driver's description in the scene, and the true value of the classification label is obtained by this method. In the test scene classification module, the fully connected layer is used to classify Y t After processing, the [3*1] vector is output, and softmax is used to obtain the classification result of the model. The overall model uses cross entropy to define the loss function and uses gradient descent for parameter training. The overall framework of the model is as follows: Figure 8 shown.

Claims

1. A criticality quantification method for intelligent vehicle test scenarios based on spatiotemporal hypergraph learning, characterized by: The following steps are involved: (1) Collect complex meteorological environment data through meteorological data collection equipment; collect traffic participant characteristic data and road structure characteristic data through drone images; (2) In the node feature parameter extraction module, the image Img collected by the drone is t The node feature parameters are extracted from the meteorological environment data collected by the meteorological collection equipment, where t represents the collected scene duration in seconds, and three different dimensions of node feature information are obtained, namely: Meteorological dimension node feature N W ; Road dimension node feature N R ; Traffic dimension node feature N T,i ; (3) The node characteristic parameters are input into the hyperedge characteristic parameter calculation module. The constructed hyperedges are divided into three types: meteorological, road, and traffic hyperedges. The hyperedge characteristic parameter calculation module calculates the three types of hyperedges corresponding to each time t. The values ​​are expressed as meteorological hyperedge SG W , Road over edge SG R and traffic super edge SG T ; (3.1) The meteorological super-edge SG W It mainly describes the impact of complex meteorological environment on roads and traffic. The impact of weather on roads is mainly reflected in the change of road adhesion coefficient, and the impact on traffic is reflected in the functional level of intelligent vehicle perception system; (3.2) The road superedge SG R Including the impact risk caused by static obstacles E sta and the risk E of the lane line affecting each position on the road R ; (3.3) The traffic super-edge SG T Describe the interactive risk generated by different traffic participants, determine it according to the position (x, y) of different traffic participants, consider the motion field strength generated by all other traffic participants in front of the corresponding position of the traffic participant, and consider the motion parameters of the vehicle to output the corresponding risk value of the traffic participant, calculate the risk value of all traffic participants in the scene, and the sum of the results is recorded as the traffic hyperedge SG T The value of (4) Using meteorological dimension node features, road dimension node features, traffic dimension node features, and meteorological hyperedge SG W , Road over edge SG R and traffic super edge SG T , a hypergraph structure is constructed in the hypergraph construction module; the hypergraph is defined as in represents the node feature set at time t, ε t represents the set of hyperedges at time t, W t The diagonal matrix representing the hyperedge weights at time t; Defining the Incidence Matrix Represents the number of nodes in the scene, |ε t | represents the number of hyperedges when constructing a hypergraph, and H is a matrix describing whether a node belongs to a specified hyperedge, where each element h is expressed as: In the formula, v represents a node and e represents a hyperedge. At the same time, the node degree matrix is ​​defined as The diagonal elements of for: At the same time, we define the hyperedge degree matrix D e The diagonal elements d(e) are: (5) The constructed hypergraph structure is input into the spatial feature information extraction module. This module is based on the spectral domain convolution theory of ordinary graphs. It integrates the information of neighboring nodes through hyperedges and updates the features of the nodes to achieve spatial feature extraction. (6) The output of the spatial feature information extraction module is input to the temporal feature information extraction module. The temporal feature information extraction module uses a gated recurrent unit network GRU, and its input is the current input x t And the hidden state H passed down from the previous node t -1 , this hidden state contains the relevant information of the previous node, combined with x t and H t-1 , the GRU network will get the output Y of the current hidden node t and the hidden state H passed to the next node t ; (7) The input of the test scene classification module is Y t , and then output the category of the test scenario. The test scenario category is divided into high, medium and low risk labels. The true value of the label needs to be obtained through a driving simulator and driver experiments.

2. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (1), during the image acquisition process using a drone, it is necessary to confirm the pixel-distance mapping coefficient δ of the drone, in units of m / pixel. This coefficient is determined by the drone's flight altitude H, the camera's longitudinal field of view fov, and the image's longitudinal resolution Pix. The expression for δ is: The video stream is converted into feature data of traffic participants and road structures through δ.

3. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (2), the meteorological dimension node feature N W :[Rain,Vis], the nodes of the meteorological dimension will not change with time. Rain is the rainfall intensity in mm / h; Vis is the fog visibility in m; The road dimension node feature N R :[n r ,R L ,μ,ε l ,(x s ,y s )], the road dimension node features do not have time series, that is, the structural features of the road will not change with time; where n r is the number of lanes, in units of lanes; RL is the lane line curve expression; μ is the road adhesion coefficient; ε l Lane line linearity, including dotted lines and solid lines; (x s ,y s ) is the static obstacle position information, in m; The traffic dimension node feature N T,i :[x,y,v x ,v y ,a x ,a y ],T∈t,i∈(1,2,3…n v ),n v Represents the number of traffic participants in the traffic dimension, N T,i Describes the characteristic information of the i-th vehicle at time T; where (x, y) is the location information of the traffic participant, in units of m; v x ,v y are the lateral and longitudinal speeds of traffic participants, in m / s; a x ,a y are the lateral and longitudinal accelerations of traffic participants, in m / s 2 .

4. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (3.1), the hyperedge feature modeling method for the weather-to-traffic dimension and the hyperedge feature modeling method for the weather-to-road dimension include: (3.1.1) Hyperedge feature modeling for weather and traffic dimensions: The hyperedge feature of the meteorological effect on the traffic dimension mainly describes the changes in the perception and recognition errors of intelligent vehicles caused by complex meteorological environments. The accuracy of the perception results corresponding to the two vehicles under different conditions is collected to fit the three-dimensional surface f W (); f W The input of () is N T The vehicle position (x, y) and N W The complex meteorological parameters in the output are the accuracy of the perception function of the smart car under the condition of the distance between the two cars, f W ( ) Parameter calibration is required for different perception schemes; Meteorological super edge feature SG W The expression is: SG W =exp[-f W (N T ,N W )] (3.1.2) Modeling of hyperedge features in the road dimension for weather: In rainfall conditions, the road adhesion coefficient μ in the road dimension is affected. The value of the road adhesion coefficient after the change due to these two meteorological environments is defined as μ W ; In the case of rainy weather conditions, water on the road creates a water film between the tire and the road surface. The thickness of the water film is h R The relationship with Rain is: h R =(60·Rain) 0.7786 μ W It is expressed as: m W =0.6603-0.0036v y -0.0057h R The maximum braking deceleration a under a specific rainfall intensity is r max Expressed as: a r max =em W g Where ε is the tire wear degree to a r max The influence coefficient of g is gravitational acceleration.

5. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (3.2), the road superedge SG R The impact on the vehicle includes two parts, one is the static obstacle and the other is the lane line; Risk E for static obstacles sta It is expressed as: R is the road condition influencing factor; Dis ij is the Euclidean distance between vehicle j and static obstacle i; N sta Represents the number of static obstacles; N represents the number of vehicles in the scene; The characteristics of the lane line will constrain the vehicle movement. Input N R , using the road potential field method to calculate the risk E of each position on the road R Describe, E R The specific expression is: E R =e l ·exp(-r l 2 / 2σ 2 ) In the formula, ε l According to the linear determination of different lane lines, the ε corresponding to the solid lane line l The value is greater than the value corresponding to the dotted line; r l For different vehicles in N R The lateral distance between the center axis of the vehicle and the lane line, in meters; σ is a constant that determines the speed at which the safety risk increases when the vehicle approaches the lane line; SG R The value of is obtained by adding the above two parts to get the final value, which is expressed as: SG R =And R +E sta 。 6. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (3.3), the traffic superedge SG T The calculation methods include: The risk field E generated by vehicle i i The expression is: Where R is the road condition influencing factor; D ij is the Euclidean distance between vehicle j and vehicle i, and D / |D| represents the Euclidean distance D ij Direction; v i is the speed of vehicle i, θ ij v represents vehicle i i The angle between the direction and the distance D between vehicles i and j; when there are N vehicles in the scene, SG T The calculation process is expressed as:

7. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (5), the spatial feature information extraction module is based on the spectral domain convolution theory of ordinary graphs, using spectral convolution The input X signal is processed and converted into the Fourier domain. In order to reduce the number of parameters to alleviate overfitting and reduce computational complexity, the formula is obtained: The spatial feature information extraction module transforms the input feature X into a set of new features Where F is the number of output features of each node, expressed as: Where Θ refers to the filter parameter matrix learned during training and is the matrix of parameters to be learned.

8. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: The process of step (6) includes: First, by transmitting the state H t-1 and input x t To obtain the gate state, where R t is the gate that controls the reset, Z t is the gate that controls the update: R t =σ(x t W xr +H t-1 W hr +b r ) Z t =σ(x t W xz +H t-1 W hz +b z ) Among them, σ( ) represents the sigmoid function, Transform the data to a value in the range of 0-1; After getting the gate signal, use the reset gate to get the data h t-1' =H t-1 ⊙R t , the data is scaled to between -1 and 1 through tanh; ⊙ represents the multiplication of the corresponding elements in the matrix, where, ; The expression is: Finally use Z t Perform the "update memory" phase: Y t Represents the output vector at each moment, which integrates the spatial information of the hypergraph and extracts the features of the dynamically changing timing information of each node.

9. The method for quantifying criticality of intelligent vehicle test scenarios based on spatiotemporal hypergraph learning according to claim 1 is characterized by: In step (7), the test scene is divided into high, medium and low risk category labels. The true value of the label needs to be obtained through a driving simulator and driver experiments. The trajectory data collected by the drone is arranged in the virtual simulation software according to the true value, and the interactive vehicle is made to move according to the real trajectory. Then the image of the forward perspective is played back, and the scene is classified according to the driver's description in the scene, and the true value of the classification label is obtained by this method; in the test scene classification module, the fully connected layer is used to classify Y t After processing, the [3*1] vector is output and normalized to obtain the classification result of the model.

Images