Scene similarity network training, typical scene obtaining and testing method and platform

Abstract

A scene similarity network training, typical scene acquisition and test method and platform, the training method comprising: acquiring a test scene complete set {Xn} of an automatic driving model and a pre-constructed initial scene similarity network based on an attention mechanism, where {Xn} comprises N test scenes Xn, n = 1, 2,... N; a few-sample test scene set {X1} is obtained from {Xn} to serve as training data, a similarity matrix of {X1} and {Xn} is generated, the initial scene similarity network is trained based on the similarity matrix, a scene similarity network between L test scenes and N test scenes is constructed, {X1} comprises L test scenes X1, l = 1, 2,... L, and 2 < = Llt; n, a scene similarity network oriented to a few-sample test problem is provided, the problem of significant uncertainty under the few-sample test in a simulation test is relieved, the reliability of the few-sample test is improved, the test efficiency is high, and the situation that the test budget is strictly limited can be met.

Description

Technical Field

[0001] This article relates to the field of autonomous driving testing, especially to a scene similarity network training, typical scene acquisition, testing method and platform. Background Art

[0002] In recent years, autonomous driving technology has made significant progress. Autonomous driving tests are required before it is officially put into use. There are currently two testing methods that can be used. One is open road testing, which is based on Monte Carlo testing theory. However, due to the large number of low-risk scenarios in the real world, it is difficult to accurately evaluate the performance of autonomous driving in high-risk situations, resulting in extremely low testing efficiency. Billions of miles of testing are required to accurately evaluate the performance of autonomous driving algorithms in fatal accidents, which is difficult to achieve practical application. At the same time, testing on real roads may also pose a safety hazard to society. The other is computer simulation testing, which uses an autonomous driving test scenario library to test the autonomous driving model. The scenario library used in this method is built based on expert experience or a scenario generation method for a specific model. During testing, specially designed standards are usually used for evaluation, making it difficult to obtain general test results and truly quantify the safety of autonomous driving. In addition, the performance is poor when the number of tests is very small, and large-scale scenario testing leads to long testing cycles, low testing efficiency, and high costs. In addition, the current autonomous driving R&D iteration cycle is short, and the algorithm is updated rapidly, making it difficult to ensure large-scale testing of each autonomous driving algorithm. The test results are completely unreliable under strict testing budget constraints. Summary of the invention

[0003] The embodiments of the present application provide a scene similarity network training, typical scene acquisition, testing method and platform, propose a scene similarity network for the few-sample testing problem, alleviate the significant uncertainty problem under the few-sample testing in the simulation test, improve the reliability of the few-sample testing, have high testing efficiency, and can meet the situation of strict restrictions on the testing budget.

[0004] In a first aspect, an embodiment of the present application provides a scene similarity network training method, comprising:

[0005] Get the full set of test scenarios for the autonomous driving model {X n}, and the pre-built initial scene similarity network based on the attention mechanism, where {X n} Includes N test scenarios X n , n=1,2,……N;

[0006] In {X n} to obtain a few-sample test scene set {X l} as training data, generate {X l} and {X n}, and based on the similarity matrix, the initial scene similarity network is trained to construct a scene similarity network between L test scenes and N test scenes, where {X l} Includes L test scenarios X l , l=1,2,……L,2≤L <N。

[0007] In a second aspect, an embodiment of the present application provides a typical scene acquisition method, which is performed based on the scene similarity network trained in the above embodiment, and includes:

[0008] In the complete set of test scenarios for autonomous driving models {X n} to obtain L test scenarios and generate a small number of test scenario sets to be optimized. Among them, {X n} Includes N test scenarios X n , n=1,2,……N, Includes L test scenarios l=1,2,……L,2≤L <N;

[0009] Based on the scene similarity network, the few-sample test scene set to be optimized Using a predetermined optimization strategy in the full set of test scenarios for the autonomous driving model {X n} to perform multiple rounds of iterative optimization to obtain a set of typical test scenarios Based on Get the typical weight matrix in, Includes L typical test scenarios and In one-to-one correspondence, the predetermined optimization strategy includes a gradient descent strategy, a genetic algorithm strategy, a particle swarm algorithm strategy or a deep learning strategy.

[0010] In a third aspect, the present application embodiment provides a typical scenario testing method, the testing method is based on the typical test scenario set as described in the above embodiment. and a typical weight matrix Execution, including:

[0011] The autonomous driving model to be tested is compared with the typical test scenario set L typical test scenarios in Interact separately to obtain interactive test results in, Represents the typical test scenario of the autonomous driving model to be tested The probability of an event of interest occurring in

[0012] According to the formula Obtain a total probability μ of the event of interest occurring in the autonomous driving model to be tested.

[0013] In a fourth aspect, an embodiment of the present application further provides a typical scenario test platform, including a processor and a memory;

[0014] The memory is used to store typical scenario test programs;

[0015] The processor is used to read the typical scenario testing program and perform the typical scenario testing method as described in the above embodiment.

[0016] Compared with the related art, the embodiment of the present application provides a scene similarity network training, typical scene acquisition, testing method and platform, proposes a scene similarity network for the few-sample testing problem, performs scene similarity learning through a deep learning method based on an attention mechanism, and constructs a similarity network between the few-sample test scenes and the entire set of test scenes for autonomous driving simulation testing, thereby maximizing the similarity between the local information represented by the few-sample test scenes and the global information represented by the entire set of test scenes, alleviating the significant uncertainty problem under the few-sample test in the simulation test, improving the reliability of the few-sample test, and having high testing efficiency and low cost, which can meet the situation with strict restrictions on the test budget.

[0017] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0019] Figure 1 A flow chart of a scene similarity network training method according to an embodiment of the present application;

[0020] Figure 2 This is a flow chart of a typical scene acquisition method according to an embodiment of the present application;

[0021] Figure 3 A flowchart of a typical scenario testing method of an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a typical scenario testing platform for an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0024] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.

[0025] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0026] Autonomous driving technology has made significant progress in recent years, especially in perception, decision-making and execution. However, despite rapid technological progress, safety remains a key issue that needs to be addressed. The road environment faced by autonomous driving systems is extremely complex, including various unpredictable factors such as weather changes, road conditions, and pedestrian behavior. In order to ensure the safe operation of autonomous driving vehicles in various situations, the system must undergo a lot of testing and verification; the existing open road testing methods and computer simulation testing methods cannot meet the current needs of short autonomous driving R&D iteration cycles, rapid algorithm updates, and the need for a large number of autonomous driving tests.

[0027] The present application embodiment provides a scene similarity network training method, including steps S1000-S2000, such as Figure 1 As shown:

[0028] S1000: Get the full set of test scenarios for the autonomous driving model {X n}, and the pre-built initial scene similarity network based on the attention mechanism, where {X n} Includes N test scenarios X n , n=1,2,……N;

[0029] S2000: In {X n} to obtain a few-sample test scene set {X l} as training data, generate {X l} and {X n}, and based on the similarity matrix, the initial scene similarity network is trained to construct a scene similarity network between L test scenes and N test scenes, where {X l} Includes L test scenarios X l , l=1,2,……L,2≤L <N。

[0030] In this embodiment, the test scenario set {X n} can be N scenes under a certain scene type, such as intersection, highway, school surrounding, rural road, rainy and snowy road, etc.; for example, under the intersection scene type, there are N scenes X n , each scene X n They all include several state components, which can be used to indicate the number of background vehicles, the starting position, starting time, driving route, driving speed corresponding to each background vehicle, the number of pedestrians, the location of surrounding buildings, etc. The above scene types and state components are all exemplary descriptions and are not limitations of this application, so they will not be described one by one.

[0031] In this embodiment, deep learning training of the initial scene similarity network may include multiple iterations, and each iteration is performed from the full set of test scenes {X n} to obtain a small sample test scene set {X l} as the training data for this training, and each time the training data {X l}All test scenarios are complete {X n}.

[0032] In this embodiment, N is the complete set of test scenarios {X n}, L is the total number of test scenarios used in each iterative training, and a scene similarity network between L test scenarios and N test scenarios is constructed through step S2000, that is, a scene similarity network between the few-sample test scenarios and the full set of test scenarios. The test results of the autonomous driving model in L test scenarios are called the few-sample test results, and the test results of the autonomous driving model in N test scenarios are called the global test results. The scene similarity network can measure the contribution of the few-sample test results to the global test results, thereby laying the foundation for efficient and accurate few-sample testing.

[0033] The scene similarity network training method of this embodiment proposes a scene similarity network for the few-sample testing problem, performs scene similarity learning through a deep learning method based on an attention mechanism, realizes the extraction and global optimization of scene information, and constructs a similarity network of the few-sample test scenes and the entire set of test scenes for autonomous driving simulation testing, maximizes the similarity between the local information represented by the few-sample test scenes and the global information represented by the entire set of test scenes, alleviates the significant uncertainty problem under the few-sample test in the simulation test, improves the reliability of the few-sample test, has high testing efficiency and low cost, and can meet the situation where the test budget is strictly restricted.

[0034] In an exemplary embodiment, before executing step S2000, steps S1100-S1200 may be included:

[0035] S1100: Get {X n}The corresponding natural probability distribution matrix P N*1 =[p(X1)p(X2)……p(X N )] T , where p(X n ) and X n One-to-one correspondence, representing the test scenario X n The natural probability of the event of interest occurring in ;

[0036] S1200: Get K autonomous driving agent models m in the agent library k The proxy library includes the proxy model m k One-to-one corresponding test indicator true value And each of the proxy models m k In each of the test scenarios X n The probability p of the event of interest occurring in k (X n ), where k = 1, 2, … K.

[0037] In this embodiment, the natural probability distribution matrix P N*1is the pre-obtained statistical data, including N elements p(X n ), respectively p(X1), p(X2), ..., p(X N ), corresponding to the full set of test scenarios {X n} in the test scenarios X1, X2, ..., X N .

[0038] In this embodiment, the occurrence of events of interest may include a collision accident, the distance between two vehicles being less than a preset distance, lane departure, a traffic signal violation, speeding, a sensor failure, a communication failure, etc. The above events of interest are all exemplary descriptions and are not intended to be limitations of this application, and will not be described one by one.

[0039] In this embodiment, the proxy model is an autonomous driving model stored in a proxy library, which includes several proxy models, and the proxy library stores the true value of the test index corresponding to each proxy model. Represents the agent model in the full set of test scenarios {X n}, the proxy library also stores the probability of each proxy model occurring in each of the test scenarios X n The probability p of the event of interest occurring in k (X n ), when executing step S1200, obtain K autonomous driving agent models m k And every m k Corresponding and N p k (X n ).

[0040] In an exemplary embodiment, step S2000 may include: looping through steps S2100-S2600 to perform training until a preset training end condition is reached, wherein the initial scene similarity network is a current scene similarity network trained for the first time:

[0041] S2100: In {X n} to obtain L test scenes as the few-sample test scene set {X l};

[0042] S2200: n} and {X l} Input the current scene similarity network to generate the current scene similarity matrix S L*N ;

[0043] S2300: According to formula W L*1 =S L*N *P N*1Calculate the current weight matrix W L*1 =[w(X1)w(X2)……w(X L )] T , where w(X l ) and {X l Test scenario X in} l One to one correspondence;

[0044] S2400: for each of the proxy models m k Perform the following steps: According to the formula Calculate the current test index estimate corresponding to the proxy model

[0045] S2500: According to the loss function Calculate the current loss value E;

[0046] S2600: Determine whether the preset training end condition is reached according to the current loss value E. If so, use the current scene similarity network as the trained scene similarity network; if not, update the parameters of the current scene similarity network and perform the next training.

[0047] In this embodiment, each training includes steps S2100-S2600, and the few-sample test scene set {X l} is the complete set of test scenarios for the autonomous driving model {X n}, each time training, step S2100 needs to be executed for resampling to obtain L test scenes for this training, so as to improve the generalization ability of the scene similarity network, reduce overfitting, and enhance the robustness and reliability of the scene similarity network.

[0048] In this embodiment, in step S2200, the current scene similarity matrix S L*N is a matrix with L rows and N columns, where l = 1, 2, ... L, n = 1, 2, ... N, located in S L*N The matrix with the lth row and nth column in the graph represents the few-shot test scene set {X l} in the lth test scenario and the complete set of test scenarios {X n} is the similarity of the nth test scene in .

[0049] In this embodiment, in step S2300, formula W L*1 =S L*N *P N*1 P N*1 That is, the P obtained in step S1100 N*1 , P N*1 is a matrix with N rows and 1 column. The current weight matrix W is obtained by calculating the formula L*1is a matrix with L rows and 1 column, the current weight matrix W L*1 Contains L elements w(X l ), corresponding to the few-sample test scene set {X l Test scenario X in} l .

[0050] In this embodiment, in each training, step S2400 needs to be executed K times, that is, the K autonomous driving agent models m obtained in step S1200 are k Execute step S2400 once respectively to obtain k One-to-one correspondence between current test index estimates

[0051] In one implementation of this embodiment, the preset training end condition can be, when executing step S2600, comparing the current loss value E with the first preset threshold value E. g , when E <E g , the current scene similarity network is used as the trained scene similarity network; when E ≥ E g , update the parameters of the current scene similarity network and perform the next training.

[0052] In another implementation of this embodiment, the preset training end condition can be, when executing step S2600, calculating the difference between the current loss value E of this training and the loss value calculated in the previous training. When the difference is within a preset range, the current scene similarity network is used as the trained scene similarity network; when the difference is not within the preset range, the parameters of the current scene similarity network are updated and the next training is performed.

[0053] In an exemplary embodiment, the step S2100 in “in {X n} to obtain L test scenarios” may include step S2110:

[0054] S2110: In {X n} of the N test scenarios X n In the method, a random sampling method, a clustering-based classification sampling method, or a quasi-Monte Carlo sampling method is used to perform sampling to obtain the L test scenes.

[0055] In an exemplary embodiment, step S2200 may include steps S2210-S2230:

[0056] S2210: {X n} for each test scenario X n and {X l} for each test scenario X lInput the current scene similarity network for feature mapping to obtain n One-to-one query vector Q n and X l One-to-one key vector K l ;

[0057] S2220: Using a preset similarity function d to compare the N query vectors Q n and L of the key vectors K l Perform similarity calculation to obtain the similarity matrix D L*N , where D L*N The element in the lth row and nth column represents the similarity d between the lth key vector and the nth query vector. ln ;

[0058] S2230: Calculate the similarity matrix D based on the preset activation function softmax L*N Each column of is normalized to obtain the current scene similarity matrix S L*N .

[0059] In this embodiment, when executing step S2210, N query vectors Q are obtained. n and L key vectors K l In step S2220, the inverse of the Euclidean distance can be used. Or d = cos(Q n , K l ) as the similarity function d.

[0060] In this embodiment, when executing step S2230, the activation function softmax is used to perform the column-wise operation on D L*N Perform normalization to obtain the current scene similarity matrix S L*N , that is, in S L*N In , the sum of the L elements in each column is 1.

[0061] The present application also provides a typical scene acquisition method, which is performed based on the scene similarity network trained in the above embodiment, and includes steps S3000-S4000. Figure 2 As shown:

[0062] S3000: A complete set of test scenarios for autonomous driving models n} to obtain L test scenarios and generate a small number of test scenario sets to be optimized. Among them, {X n} Includes N test scenarios X n , n=1,2,……N, Includes L test scenarios l=1,2,……L,2≤L <N;

[0063] S4000: Based on the scene similarity network, the few-sample test scene set to be optimized Using a predetermined optimization strategy in the full set of test scenarios for the autonomous driving model {X n} to perform multiple rounds of iterative optimization to obtain a set of typical test scenarios Based on Get the typical weight matrix in, Includes L typical test scenarios and In one-to-one correspondence, the predetermined optimization strategy includes a gradient descent strategy, a genetic algorithm strategy, a particle swarm algorithm strategy or a deep learning strategy.

[0064] In this embodiment, after obtaining the trained scene similarity network, based on the scene similarity network, the whole set of test scenes {X n}, that is, obtain L test scenarios from N test scenarios as typical test scenarios, and the typical test scenario set For the complete set of test scenarios {X n}, and use the predetermined optimization strategy to perform multiple rounds of iterative optimization to complete the target optimization search problem of L test scenarios.

[0065] In this embodiment, step S3000 is performed in the test scenario set {X n} to obtain the few-sample test scene set to be optimized in, Yes {X n}; Step S4000 is Each test scenario in In the test scenario set {X n} to optimize and obtain a set of typical test scenarios Also a complete set of test scenarios {X n}, through a predetermined optimization strategy in {X n} to perform optimal search and finally obtain the L typical test scenarios As a typical test scenario set Compared with the massive N test scenarios, a small number of L typical scenarios were screened out, laying the foundation for further efficient typical scenario testing.

[0066] In an exemplary embodiment, step S4000 may include: looping through steps S4100-S4700 to perform the multiple rounds of iterative optimization until the optimized scene set {X l″} and the scene set {X l The current round Manhattan distance Y between ′} is less than the second preset threshold Y g , where the scene set {X l '} is the few-sample test scene set to be optimized

[0067] S4100: The entire set of test scenarios {X n} and the scene set {X l ′} Input the trained scene similarity network to generate the scene similarity matrix S L ' *N ;

[0068] S4200: According to formula W L ' *1 =S L ' *N *P N*1 Calculate the weight matrix W for this round L ' *1 =[w(X1′)w(X2′)……w(X L ′)] T , where w(X l ′) and the scene set {X l Test scenario X in ′} l ′ one-to-one correspondence; among them, P N*1 Represents {X n}The corresponding natural probability distribution matrix, P N*1 =[p(X1)p(X2)……p(X N )] T , p(X n ) and X n One-to-one correspondence, representing the test scenario X n The natural probability of the event of interest occurring in ;

[0069] S4300: Obtain K autonomous driving agent models m in the agent library k , for each proxy model m k Perform the following steps: Get the l 'The probability p(X l ′), according to the formula Calculate the estimated value of the current round of test indicators corresponding to the proxy model Where k = 1, 2, ... K, p k (X l ′) represents the proxy model m k In X lThe probability of the event of interest occurring in ′;

[0070] S4400: Based on the optimization objective function Calculate the maximum error E' of this round; where, With the proxy model m k One-to-one correspondence, representing the agent model m k The corresponding test indicator true value;

[0071] S4500: According to E′ and {X l '}, adopt the predetermined optimization strategy in {X n}For each X l 'Optimize the search and obtain l ′ One-to-one correspondence between the current round of optimization scene X l ″, based on L X l ″Generate the scene set {X after this round of optimization l ″};

[0072] S4600: According to the scene set {X l ″} and the scene set {X l '} Calculate the current round Manhattan distance Y;

[0073] S4700: Compare the current round Manhattan distance Y with a second preset threshold Y g , when Y <Y g , the scene set {X l ″} as the typical test scenario set The current round weight matrix W L ' *1 =[w(X1′)w(X2′)……w(X L ′)] T As a typical weight matrix When Y ≥ Y g , the scene set {X l ″} as the scene set {X before the current round of optimization in the next round of iteration l ′}.

[0074] In this embodiment, when executing step S4100, a method similar to the above steps S2210-S2230 may be used to generate the scene similarity matrix S of this round. L ' *N , the complete set of test scenarios {X n} for each test scenario X n and the scene set {X l Each test scenario X in l′ Input the trained scene similarity network for feature mapping to obtain n One-to-one query vector Q n and X l ′ one-to-one corresponding key vector K l '; Use the preset similarity function d to compare the N query vectors Q n and L key vectors K l 'Calculate the similarity and obtain the similarity matrix D L*N ′, where D L*N The element in the lth row and nth column in ′ represents the lth key vector K l ′ and the nth query vector Q n The similarity d ln '; Based on the preset activation function softmax, the similarity matrix D L*N Each column of ′ is normalized to obtain the scene similarity matrix S of this round L ' *N .

[0075] In this embodiment, the natural probability distribution matrix P in step S4200 N*1 and the natural probability distribution matrix P in step S1100 N*1 =[p(X1)p(X2)……p(X N )] T Same as; K autonomous driving agent models m in step S4300 k The same as step S1200 in which K autonomous driving agent models m are obtained from the agent library k same.

[0076] In this embodiment, after executing steps S4400 and S4500, the scene set {X l ″} is also the complete set of test scenarios {X n}, and the scene X obtained after optimization search in this round according to the predetermined optimization strategy l ″The corresponding scene before this round of optimization X l ′ is more representative than the traditional one, and the distribution of test scenarios is more even in different areas with different probabilities of occurrence of events of interest, so that the typical test scenarios finally obtained are distributed in different probability ranges of occurrence of events of interest. Each typical test scenario can represent other similar scenarios, and these typical test scenarios can be given greater weights in autonomous driving tests. Applying typical test scenarios to autonomous driving tests can improve test efficiency while minimizing test errors.

[0077] In this embodiment, the typical weight matrix Although it only includes L elements With a typical test scenario set Typical test scenarios in One-to-one correspondence, but Contains a complete set of test scenarios {X n} information, can maximize the use of {X n}, reducing the huge uncertainty of testing based only on typical test scenarios.

[0078] In an exemplary embodiment, step S4600 may include steps S4610-S4630:

[0079] S4610: Obtain the scene set {X l ″}L X l ″The T state components a corresponding to each l ″ t ;

[0080] S4620: Obtain the scene set {X l ′} in L X l ′ respectively corresponds to T state components a l ' t ;

[0081] S4630: According to the formula Calculate the Manhattan distance Y of this round.

[0082] The present application also provides a typical scenario testing method, which is based on the typical test scenario set described in the above embodiment. and a typical weight matrix Execution, including steps S5000-S6000, such as Figure 3 As shown:

[0083] S5000: Compare the autonomous driving model to be tested with the typical test scenario set L typical test scenarios in Interact separately to obtain interactive test results in, Represents the typical test scenario of the autonomous driving model to be tested The probability of an event of interest occurring in

[0084] S6000: According to the formula Obtain a total probability μ of the event of interest occurring in the autonomous driving model to be tested.

[0085] In this embodiment, for an autonomous driving model to be tested, only L typical test scenarios are required. Interactive testing is performed, that is, only L tests of step S5000 and one matrix multiplication calculation of step S6000 are required to obtain the total probability μ of the event of interest occurring in the autonomous driving model to be tested, without the need to perform the entire set of test scenarios {X n}Execute N interactive tests, which greatly improves the test efficiency and reduces the test cost. At the same time, due to the typical weight matrix Contains the full set of test scenarios {X n}, the total probability μ of the autonomous driving model to be tested occurring the event of interest can represent the total probability μ of the autonomous driving model to be tested in the entire set of test scenarios {X n}, the probability of an event of interest occurring in the test, and the test reliability is high.

[0086] The typical scenario testing method of this embodiment solves the uncertainty of traditional testing based on a small number of test scenarios, improves the reliability of testing the autonomous driving model in a small number of test scenarios, and the typical weight matrix contains global information of the entire set of test scenarios. The test has strong stability, greatly improved test efficiency, and low cost.

[0087] The embodiment of the present application also provides a typical scenario test platform, including a processor and a memory, such as Figure 4 As shown;

[0088] The memory is used to store typical scenario test programs;

[0089] The processor is used to read the typical scenario testing program and perform the typical scenario testing method as described in the above embodiment.

[0090] In order to illustrate the scene similarity network training method, typical scene acquisition method and typical scene testing method of the embodiment of the present application, a specific example 1 is used to describe in detail below. The specific example includes steps S1-S6:

[0091] S1: Get the full set of test scenarios for the autonomous driving model {X n}, and the pre-built initial scene similarity network based on the attention mechanism, where {X n} Includes N test scenarios X n , n=1,2,……N;

[0092] S2: In {X n} to obtain a few-sample test scene set {X l} as training data, generate {X l} and {X n}, and based on the similarity matrix, the initial scene similarity network is trained to construct a scene similarity network between L test scenes and N test scenes, where {X l} Includes L test scenarios X l , l=1,2,……L,2≤L <N;

[0093] S3: A complete set of test scenarios for autonomous driving models {X n} to obtain L test scenarios and generate a small number of test scenario sets to be optimized. in, Includes L test scenarios l=1,2,……L,2≤L <N;

[0094] S4: Based on the scene similarity network trained in step S2, the few-sample test scene set to be optimized Using a predetermined optimization strategy in the full set of test scenarios for the autonomous driving model {X n} to perform multiple rounds of iterative optimization to obtain a set of typical test scenarios Based on Get the typical weight matrix in, Includes L typical test scenarios and One to one correspondence;

[0095] S5: Compare the autonomous driving model to be tested with the typical test scenario set L typical test scenarios in Interact separately to obtain interactive test results in, Represents the typical test scenario of the autonomous driving model to be tested The probability of an event of interest occurring in

[0096] S6: According to the formula Obtain a total probability μ of the event of interest occurring in the autonomous driving model to be tested.

[0097] In the specific example 1, S1 and S2 are the steps of training the scene similarity network, and S3 and S4 are the steps of using the trained scene similarity network to obtain a typical test scene set. Steps S5 and S6 are based on a typical test scenario set The steps for testing the autonomous driving model to be tested are as follows: After that, only steps S5 and S6 are executed for each autonomous driving model to be tested to complete the test. During the test, the autonomous driving model to be tested is tested in L typical test scenarios. In the test scenario set {X n Compared with N interactions in}, the test efficiency is greatly improved, the test time is shortened, and the test cost is reduced.

[0098] In order to illustrate the effect of the typical scenario testing method of the embodiment of the present application, specific example 2 is described in detail below.

[0099] In this specific example, three methods are used to test the three autonomous driving test models AV1, AV2, and AV3 to be tested. Method 1 is the Monte Carlo method, method 2 is the random quasi-Monte Carlo method based on uniform sampling, and method 3 is the typical scenario test method of the embodiment of the present application. When testing the three methods, samples are taken from the entire set of test scenarios, and a subset of test scenarios is selected as a typical test scenario for testing, including selecting 10 typical test scenarios (i.e., L=10) and selecting 20 typical test scenarios (i.e., L=20) from N=1000 to 10000 test scenarios. The "test error" and "99% confidence error" are used to evaluate the test effect, as shown in Table 1:

[0100] Table 1 Comparison of the effects of three test methods

[0101]

[0102] It can be seen from Table 1 that the typical scenario testing method of the embodiment of the present application, that is, method three, is used to test the autonomous driving test models AV1, AV2, and AV3. Regardless of whether 10 typical test scenarios or 20 typical test scenarios are used for testing, the "test error" and "99% confidence error" are significantly lower than those of method one and method two, and the test reliability is high and the effect is good.

[0103] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term "computer storage medium" includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A scene similarity network training method, characterized in that: include: Get the full set of test scenarios for the autonomous driving model {X n }, and the pre-built initial scene similarity network based on the attention mechanism, where {X n } Includes N test scenarios X n , n=1,2,……N; In {X n } to obtain a few-sample test scene set {X l } as training data, generate {X l } and {X n }, and based on the similarity matrix, the initial scene similarity network is trained to construct a scene similarity network between L test scenes and N test scenes, where {X l } Includes L test scenarios X l , l=1,2,……L,2≤L <N。 2. The scene similarity network training method according to claim 1, characterized in that: Before training the initial scene similarity network, the method includes: Get {X n }The corresponding natural probability distribution matrix P N*1 =[p(X1)p(X2)……p(X N )] T , where p(X n ) and X n One-to-one correspondence, representing the test scenario X n The natural probability of the event of interest occurring in ; Get K autonomous driving agent models m in the agent library k The proxy library includes the proxy model m k One-to-one corresponding test indicator true value And each of the proxy models m k In each of the test scenarios X n The probability p of the event of interest occurring in k (X n ), where k = 1, 2, … K.

3. The scene similarity network training method according to claim 2, characterized in that: In {X n } to obtain a few-sample test scene set {X l } as training data, generate {X l } and {X n }, and training the initial scene similarity network based on the similarity matrix to construct a scene similarity network between the L test scenes and the N test scenes, including: The following steps are executed in a loop for training until a preset training end condition is reached, wherein the initial scene similarity network is the current scene similarity network trained for the first time: In {X n } to obtain L test scenes as the few-sample test scene set {X l }; {X n } and {X l } Input the current scene similarity network to generate the current scene similarity matrix S L*N ; According to the formula W L*1 =S L*N *P N*1 Calculate the current weight matrix W L*1 =[w(X1)w(X2)……w(X L )] T , where w(X l ) and {X l Test scenario X in} l One to one correspondence; For each of the proxy models m k Perform the following steps: According to the formula Calculate the current test index estimate corresponding to the proxy model According to the loss function Calculate the current loss value E; According to the current loss value E, it is determined whether the preset training end condition is reached. If so, the current scene similarity network is used as the trained scene similarity network; if not, the parameters of the current scene similarity network are updated for the next training.

4. The scene similarity network training method according to claim 3, characterized in that: In {X n } to obtain L test scenarios, including: In {X n } of the N test scenarios X n In the method, a random sampling method, a clustering-based classification sampling method, or a quasi-Monte Carlo sampling method is used to perform sampling to obtain the L test scenes.

5. The scene similarity network training method according to claim 3, characterized in that: The n } and {X l } Input the current scene similarity network to generate the current scene similarity matrix S L*N ,include: {X n } for each test scenario X n and {X l } for each test scenario X l Input the current scene similarity network for feature mapping to obtain n One-to-one query vector Q n and X l One-to-one key vector K l ; The similarity function d is used to calculate the N query vectors Q. n and L of the key vectors K l Perform similarity calculation to obtain the similarity matrix D L*N , where D L*N The element in the lth row and nth column represents the similarity d between the lth key vector and the nth query vector. ln ; Based on the preset activation function softmax, the similarity matrix D L*N Each column of is normalized to obtain the current scene similarity matrix S L*N .

6. A typical scene acquisition method, characterized in that: The acquisition method is performed based on the scene similarity network trained according to any one of claims 1 to 5, and includes: In the complete set of test scenarios for autonomous driving models {X n } to obtain L test scenarios and generate a small number of test scenario sets to be optimized. Among them, {X n } Includes N test scenarios X n , n=1,2,……N, Includes L test scenarios Based on the scene similarity network, the few-sample test scene set to be optimized Using a predetermined optimization strategy in the full set of test scenarios for the autonomous driving model {X n } to perform multiple rounds of iterative optimization to obtain a set of typical test scenarios Based on Get the typical weight matrix in, Includes L typical test scenarios and In one-to-one correspondence, the predetermined optimization strategy includes a gradient descent strategy, a genetic algorithm strategy, a particle swarm algorithm strategy or a deep learning strategy.

7. The typical scene acquisition method according to claim 6, characterized in that: The few-sample test scene set to be optimized Using a predetermined optimization strategy in the full set of test scenarios for the autonomous driving model {X n } to perform multiple rounds of iterative optimization to obtain a set of typical test scenarios include: The following steps are executed repeatedly to perform the multiple rounds of iterative optimization until the scene set {X l ″} and the scene set {X l The current round Manhattan distance Y between ′} is less than the second preset threshold Y g , where the scene set {X l '} is the few-sample test scene set to be optimized The entire set of test scenarios {X n } and the scene set {X l ′} Input the trained scene similarity network to generate the scene similarity matrix S L ' *N ; According to the formula W L ' *1 =S L ' *N *P N*1 Calculate the weight matrix W for this round L ' *1 =[w(X1′)w(X2′)……w(X L ′)] T , where w(X l ′) and the scene set {X l Test scenario X in ′} l ′ one-to-one correspondence; among them, P N*1 Represents {X n }The corresponding natural probability distribution matrix, P N*1 =[p(X1)p(X2)……p(X N )] T , p(X n ) and X n One-to-one correspondence, representing the test scenario X n The natural probability of the event of interest occurring in ; Get K autonomous driving agent models m in the agent library k , for each proxy model m k Perform the following steps: Get the l 'The probability p(X l ′), according to the formula Calculate the estimated value of the current round of test indicators corresponding to the proxy model Where k = 1, 2, ... K, p k (X l ′) represents the proxy model m k In X l The probability of the event of interest occurring in ′; According to the optimization objective function Calculate the maximum error E' of this round; where, With the proxy model m k One-to-one correspondence, representing the agent model m k The corresponding test indicator true value; According to E′ and {X l '}, adopt the predetermined optimization strategy in {X n }For each X l 'Optimize the search and obtain l ′ One-to-one correspondence between the current round of optimization scene X l ″, based on L X l ″Generate the scene set {X after this round of optimization l ″}; According to the scene set {X l ″} and the scene set {X l '} Calculate the current round Manhattan distance Y; Compare the current round Manhattan distance Y with the second preset threshold Y g , when Y <Y g , the scene set {X l ″} as the typical test scenario set The current round weight matrix W L ' *1 =[w(X1′)w(X2′)……w(X L ′)] T As a typical weight matrix When Y ≥ Y g , the scene set {X l ″} as the scene set {X before the current round of optimization in the next round of iteration l ′}.

8. The typical scene acquisition method according to claim 7, characterized in that: According to the scene set {X l ″} and the scene set {X l '} Calculate the current round Manhattan distance Y, including: Get the scene set after this round of optimization {X l ″}L X l ″The T state components a corresponding to each l ″ t ; Get the scene set before this round of optimization {X l ′} in L X l ′ respectively corresponds to T state components a l ' t ; According to the formula Calculate the Manhattan distance Y of this round.

9. A typical scenario testing method, characterized in that: The test method is based on a typical test scenario set as described in any one of claims 6-8 and a typical weight matrix Execution, including: The autonomous driving model to be tested is compared with the typical test scenario set L typical test scenarios in Interact separately to obtain interactive test results in, Represents the typical test scenario of the autonomous driving model to be tested The probability of an event of interest occurring in According to the formula Obtain a total probability μ of the event of interest occurring in the autonomous driving model to be tested.

10. A typical scenario testing platform, characterized in that: include: Processor and memory; The memory is used to store typical scenario test programs; The processor is used to read the typical scenario testing program and perform the typical scenario testing method as described in claim 9.

Images