Intelligent vehicle complex test scenario agent model design method based on transfer learning

Abstract

The application relates to a complex test scene agent model design method, in particular to an intelligent automobile complex test scene agent model design method based on transfer learning. First, an agent model architecture is designed, and then a virtual simulation tool is used to obtain test results corresponding to the complex test scene, and the agent model is pre-trained; then, the agent model is corrected based on a small amount of data using transfer learning; finally, a meta-learning fine-tuning method combined with a dynamic learning rate scheduling is introduced, each fine-tuning is divided into two parts of inner task fast adaptation and outer global generalization optimization, the inner layer uses a dynamic cosine annealing learning rate strategy, so that the model can capture specific test scene signals quickly at an early stage with a relatively large learning rate, and the step is gradually reduced in the later period to suppress overfitting; the performance of all tasks on the validation set is gradient-aggregated, an annealing learning strategy is used to dynamically adjust the meta-learning rate, and sufficient exploration in the early stage of meta-update and stable convergence in the later stage are ensured.

Description

TECHNICAL FIELD

[0001] The application relates to a complex test scene agent model design method, in particular to an intelligent vehicle complex test scene agent model design method based on transfer learning. BACKGROUND

[0002] Intelligent vehicles have become an important development direction of the automobile industry. Before they are actually put into use, a large number of tests need to be carried out to ensure the completeness and robustness of their functions in various scenes. A complex test scene is a scene that has more environmental variables, more traffic participants and more uncertainties compared with a conventional scene. For example, for a conventional following vehicle scene, only the motion characteristics of the preceding vehicle and the ego vehicle on the same lane line need to be considered, while for a complex following vehicle scene, the rain or fog weather environment and the participation of different types of traffic target objects around the ego vehicle need to be considered. Intelligent vehicles are more likely to expose their functional defects and design deficiencies in complex test scenes than in conventional scenes, so it is highly necessary to test intelligent vehicles using complex test scenes.

[0003] The test method for intelligent vehicles can be divided into virtual simulation testing, hardware-in-the-loop testing and real vehicle road testing. Each of the three test methods has its own advantages and disadvantages when applied to the process of testing intelligent vehicles in complex test scenes. For virtual simulation testing, the construction of complex scenes can be autonomously controlled, including different rain intensities, different fog visibilities, different traffic participants and various road forms. The cost of obtaining a complex test scene is the lowest among the three test methods. However, the simulation fidelity limits the confidence of the test results, making it impossible to accurately obtain the functional boundaries of the tested intelligent vehicle. For hardware-in-the-loop testing, a part of the tested hardware of the intelligent vehicle is embedded in a virtual simulation test environment. The response of the tested hardware in this process is real and real-time. This method improves the fidelity of the test process, making the test results more reliable than virtual simulation testing. However, the cost of constructing a complex test scene through hardware-in-the-loop testing is higher, and there is a certain time cost in switching test scenes. For real vehicle road testing, the fidelity of this method is the highest. The test process fully and realistically reflects the response and behavior of the intelligent vehicle in a complex test scene. However, due to the high cost of constructing a complex test scene, the long test period and the small number of test scenes, it is difficult to fully describe the performance of the intelligent vehicle in a complex test scene. An agent model is a model used to predict the test results of scenes that have not been tested. Using an agent model to replace simulation, hardware-in-the-loop or real vehicle testing processes can greatly improve test efficiency and thus reduce test costs.

[0004] However, the agent model proposed by the existing research has two deficiencies, on the one hand, the existing method mostly relies on pure virtual simulation environment to construct the agent model, and the deficiency of the simulation degree makes it difficult for the agent model to accurately predict the test results of the test scene; on the other hand, the existing research focuses on constructing the agent model in the logical scene space, and only constructs the test scene in a certain parameter value space range, and this process ignores the interaction characteristics between the ego vehicle and other traffic participants. SUMMARY

[0005] In order to solve the above technical problems, the present application provides a kind of intelligent vehicle complex test scene agent model design method based on transfer learning, steps as follows:

[0006] First, the agent model architecture design is carried out, and the test scene is divided into trajectory, road and weather information, the trajectory information in the complex test scene is extracted using the gated recurrent unit; the road information in the complex test scene is extracted using the convolutional neural network; the weather information is described using the rainfall intensity (unit: mm / h) and the fog visibility (unit: m), and the specific value of the weather parameter is integrated in the form of feature integration; the merged variable is input into the fully connected layer, the mapping from the complex test scene to the test result is realized, and then the test result of the new complex test scene is predicted, and the test result is used as the target of the network to construct the agent model;

[0007] Further, the specific steps of the agent model architecture design are as follows:

[0008] For the time motion trajectory information of the vehicle, the feature extraction is realized using the gated recurrent unit GRU, and the gated recurrent unit GRU includes: the hidden state at time t , input data, hidden state and output data at time t , intermediate hidden state , reset gate , update gate ; the reset gate controls the number of hidden states from the previous time point written into the current GRU node, and the smaller the value of the reset gate, the less the previous information retained; the update gate controls the number of hidden states from the previous time point mixed into the hidden state at the current time, and the greater the value of the update gate, the more the previous information introduced.

[0009] Further, the forward calculation process of GRU is as follows:

[0010]

[0011] wherein, and are the weight matrix and bias matrix of GRU model that need to be trained, respectively; * is Hadamard Product; is a neural network activation function.

[0012] For static road information, convolutional neural network is used to realize feature extraction, and the road information is described in the perspective of the overhead angle. The solid lane line is set to pixel value 1, the dashed lane line is set to pixel value 0.5, and the road surface area is set to pixel value 0. Then the road information is converted into image information, and convolutional neural network is used to realize feature extraction.

[0013] For weather information, rainfall intensity (unit: mm / h) and fog visibility (unit: m) are used to describe the weather, respectively. The values of weather parameters are directly concatenated with the trajectory and road latent space features, and the concatenated variables are input into the fully connected layer. The test results are used as the target of the network to build the proxy model.

[0014] For test results, different test purposes can choose different test result expression methods.

[0015] Secondly, the test results corresponding to the complex test scenarios obtained by using virtual simulation tools provide the basis for establishing an accurate proxy model. The correspondence between the massive virtual simulation test scenarios and test results is used to pre-train the proxy model. Then, the correspondence between the test scenarios and test results of a small number of samples obtained by using hardware-in-the-loop and real vehicle road test methods is used to correct the proxy model based on these small amount of data using transfer learning, realizing the construction of a high-accuracy proxy model under the condition of limited data volume.

[0016] The test results of the proxy model are as follows:

[0017]

[0018] wherein, represents the test result predicted corresponding to the i-th complex test scenario; represents the proxy model; are the trajectory information, road information and weather information in the i-th complex test scenario, respectively.

[0019] ​​Finally, after the original pre-training based on transfer learning and small sample correction agent model process, a meta-learning fine-tuning method combined with dynamic learning rate scheduling is introduced. This method samples multiple tasks in virtual simulation, hardware-in-the-loop and real car road three data sources, simulates the small sample adaptation process in multiple test scenarios, and divides each fine-tuning into two parts: "inner task fast adaptation" and "outer global generalization optimization". The inner layer uses a dynamic cosine annealing learning rate strategy to make the model capture specific test scene signals quickly at an early stage with a larger learning rate, and gradually reduce the step size to suppress overfitting in the later stage. The outer layer aggregates the gradient of all tasks on the validation set and also uses an annealing learning strategy to dynamically adjust the meta-learning rate to ensure sufficient exploration in the early stage of meta-update and more stable convergence in the later stage.

[0020] Further, the meta-learning fine-tuning method combined with dynamic learning rate scheduling is as follows:

[0021] Randomly extract small sample tasks from virtual simulation, hardware-in-the-loop and real car road three data sources Each task contains a part of the training set and test set . The inner layer gradient update is based on a small amount of sample data The outer layer gradient evaluation and parameter summary are based on validation data, which is used to measure the generalization performance of the parameters after the inner layer update.

[0022] After the inner layer update settlement, perform step gradient descent for each task and dynamically adjust the learning rate at each step to accelerate early convergence and stabilize later fine-tuning.

[0023] Specifically, the learning rate of the first step is scheduled using cosine annealing:

[0024]

[0025] where is the upper limit of the inner learning rate, used for fast learning in the early training stage; is the lower limit of the inner learning rate, used for stable fine-tuning in the late training stage; the inner layer refers to the independent execution of step parameter adaptation for each task to quickly capture single scene features; the corresponding model parameter update is:

[0026]

[0027] where is the th task in the inner layer Step update the agent model Parameter state For the model On the dataset Loss calculated Indicates the loss Gradient of the loss function .

[0028] Subsequently, in the outer layer update, the initial parameters On the validation set of all tasks are jointly optimized , and the outer learning rate is also adjusted using cosine annealing The learning rate of the

[0029]

[0030] And the update direction is set to the gradient sum of all tasks on the validation set

[0031]

[0032] Wherein, The lower limit and upper limit of the outer learning rate are respectively The step size of the outer update Global model parameters of meta-learning, which are shared initial parameters at initialization and are constantly optimized and adjusted during the outer update process. The final output aggregated parameters can quickly adapt to new scenarios. Repeat the "inner task fast adaptation" and "outer global generalization optimization" processes After times, the fine-tuned final parameters .

[0033] The beneficial effects of the present application are:

[0034] The method proposed by the present application is specific to complex scenarios, and can more specifically consider the individual motion characteristics and group interaction characteristics of vehicles. The present application combines the advantages of different test methods through transfer learning, and constructs a high-accuracy agent model using a meta-learning fine-tuning method combined with a dynamic learning rate schedule; the present application further realizes integrated feature extraction of vehicle dynamic trajectory, road static retention and meteorological interaction dependence by constructing a multi-modal neural network, thereby improving the application range of the model. The present application has a small number of sample tasks, high efficiency of inner cosine annealing learning rate, strong stability of outer annealing decay, and improved adaptability of the model to complex scenarios through multi-source task simulation. All scheduling parameters can be adjusted online according to the actual computing power and sample size, facilitating engineering deployment and expansion. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The present application is a whole process schematic diagram

[0036] Figure 2 This is a schematic diagram of the proxy model network framework design of the present invention;

[0037] Figure 3 This is a schematic diagram of the gated loop unit structure of the present invention;

[0038] Figure 4 This is a schematic diagram illustrating the transfer learning process of the present invention. Detailed Implementation

[0039] This invention provides a method for designing a proxy model for complex testing scenarios of intelligent vehicles based on transfer learning. First, the proxy model architecture is designed. The proxy model architecture designed in this invention is as follows: Figure 2 As shown, the test scenario is segmented into trajectory, road, and weather information. A gated recurrent unit is used to extract trajectory information from the complex test scenario; a convolutional neural network is used to extract road information from the complex test scenario; specific values ​​of weather parameters are concatenated for feature integration; and a fully connected layer is used to map the complex test scenario to the test result, thereby enabling the prediction of test results for entirely new complex test scenarios, reducing the number of tests and improving testing efficiency. The following sections describe each part in detail:

[0040] For the vehicle's temporal trajectory information, this invention uses a gated recurrent unit (GRU) to extract features. A GRU is a variant model based on a recurrent neural network (RNN) structure. By using gating units, the RNN gains the ability to control the long-term memory and forgetting of its internal information. GRU uses two gating units, thus reducing the computational cost during training. Furthermore, GRU achieves training results similar to LSTM. This invention uses GRU to construct a discriminator. The structure of GRU is as follows: Figure 3 As shown.

[0041] exist Figure 3 middle, for The implicit state of a moment. Let be the input data, hidden state, and output data at time t, respectively. This is an intermediate, implicit state. Represents the reset door. The reset gate controls how much implicit state from the previous time step is written to the current GRU node; the smaller the reset gate value, the less previous information is retained. The update gate controls how much implicit state from the previous time step is mixed into the current implicit state; the larger the update gate value, the more previous information is introduced. The forward computation process of GRU is as follows:

[0042]

[0043] wherein, and are the weight matrix and bias matrix that need to be trained for the GRU model; * is the Hadamard Product; is the neural network activation function.

[0044] For static road information, the application uses a convolutional neural network to realize feature extraction, describes the road information in a bird's-eye view, sets the solid lane line as pixel value 1, the dashed lane line as pixel value 0.5, and the road surface area as pixel value 0, and then converts the road information into image information and realizes feature extraction using a convolutional neural network.

[0045] For weather information, rainfall intensity (unit: mm / h) and fog visibility (unit: m) are used to describe the weather, and the weather parameter values are directly concatenated with the trajectory and road latent space features, and the concatenated variables are input into the fully connected layer, and the test results are used as the target of the network to build a proxy model.

[0046] For test results, different test purposes can select different test result expression methods, for example, for intelligent car perception components, the test result can be the evaluation index result of multi-target tracking; for the whole intelligent car, the test result can be whether a collision occurs in this scenario.

[0047] Secondly, the application uses a virtual simulation tool to obtain the test results corresponding to complex test scenarios at the lowest cost, providing a basis for building an accurate proxy model, as shown in Figure 4 The corresponding relationship between a large number of virtual simulation test scenarios and test results is used to pre-train the proxy model; then a small amount of test scenarios and test results are obtained using hardware-in-the-loop and real car road test methods, and based on these small amounts of data, the proxy model is corrected using transfer learning, realizing a high-accuracy proxy model built under limited data conditions.

[0048] The proxy model is used for key test scenario extraction or derivation process, the key test scenario is a specific scenario that has a high probability of revealing the performance defects of the automatic system, composed of road, traffic and weather three scene parameters, which can cause the tested vehicle to collide or approach to collide with other road entities, and the test results of the proxy model are as follows:

[0049]

[0050] wherein, represents the corresponding first Each complex test scenario corresponds to the predicted test results; Representative proxy model; The first Trajectory information, road information, and weather information in a complex test scenario.

[0051] Finally, following the existing pre-training and few-sample correction process based on transfer learning, this invention introduces an innovative method: a meta-learning fine-tuning method combined with dynamic learning rate scheduling. This method closely integrates with the design requirements of the agent model in complex test scenarios of intelligent vehicles. By sampling multiple tasks from three data sources—virtual simulation, hardware-in-the-loop, and real-vehicle road—it simulates the few-sample adaptation process under various test scenarios, thereby significantly improving the model's generalization ability and convergence speed in real environments.

[0052] To address the slow adaptation and overfitting issues caused by scarce samples and large scene differences in existing fine-tuning stages, this invention decomposes each fine-tuning step into two parts: "rapid adaptation of the inner task" and "global generalization optimization of the outer layer." The inner layer employs a dynamic cosine annealing learning rate strategy, enabling the model to quickly capture specific test scene signals with a large learning rate in the early stages of each small-sample task, while gradually reducing the step size in the later stages to suppress overfitting. The outer layer aggregates gradients of the performance of all tasks on the validation set and uses a similar annealing strategy to dynamically adjust the meta-learning rate, ensuring sufficient exploration in the early stages of meta-updates and more robust convergence in the later stages. This fine-tuning method not only retains the advantage of traditional meta-learning in rapidly adapting to small samples but also incorporates a dynamic scheduling strategy, allowing the surrogate model to quickly adjust to its optimal state with only a few dozen real test data points, while avoiding performance degradation and instability caused by improper scheduling. The following is a detailed mathematical description of the meta-learning fine-tuning method combined with dynamic learning rate scheduling:

[0053] Randomly selected from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road data. Small sample task Each task Includes a small training set and test set ; For a small amount of sample data for inner gradient updates, The validation data for outer layer gradient evaluation and parameter aggregation is used to measure the generalization performance of parameters after inner layer updates. In the inner layer update settlement, this invention performs [the following steps] for each task. The gradient descent is performed step by step, with the learning rate dynamically adjusted at each step to accelerate early convergence and fine-tune later when stabilization is achieved. Specifically, the first step... learning rate of step Cosine annealing scheduling:

[0054]

[0055] wherein, is the upper limit of inner learning rate, used for fast learning in the early stage of training; is the lower limit of inner learning rate, used for stable fine-tuning in the later stage of training; the inner layer refers to the independent execution of each task step parameter adaptation, used for fast capture of single scene characteristics; the corresponding model parameter update is:

[0056]

[0057] wherein, is the updated agent model parameter state of the th task in the inner layer at the th step; is the loss calculated on the dataset for the model ; the embodiment adopts cross-entropy loss; represents the gradient of the loss function with respect to the parameter . The model parameter update mechanism uses a larger step size to quickly approach the optimal region of the new scene in the early stage of fine-tuning, and smoothly reduces the step size in the later stage to prevent overfitting. Subsequently, in the outer layer update, the present application jointly optimizes the initial parameters on the validation sets of all tasks, and also uses cosine annealing to adjust the outer learning rate ; the learning rate of the th dimension update is:

[0058]

[0059] and the update direction is set as the gradient sum of all tasks on the validation set:

[0060]

[0061] wherein, here is the global model parameter of meta-learning, which is used as the shared initial parameter at initialization and is continuously optimized and adjusted in the outer layer update process. The final output aggregated parameter can quickly adapt to the new scene. This design not only allows the model to continuously obtain global optimal information on the multi-task validation, but also ensures that the parameter update is more robust in the later stage of meta-learning through dynamic decay. After repeating the inner and outer layer processes times, the final parameter after fine-tuning can be obtained. All scheduling parameters and can be adjusted online according to actual computing power and sample size.

[0062] Subsequently, a conditional generation model can be used to generate scene elements and the corresponding predicted outcome training the generative model , The method of the present disclosure is not limited to conditional adversarial generative networks, conditional diffusion generative models, etc. For example, for conditional adversarial generative networks, when training to achieve parity between the generator and the discriminator, the generator can be used to derive key test scenarios, which fully apply the proxy model's ability to accurately predict test outcomes.

Claims

1. A method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning, characterized in that: The steps are as follows: First, the proxy model architecture is designed, and the test scenario is divided into trajectory, road and weather information. The gated recurrent unit is used to extract the trajectory information in the complex test scenario; the convolutional neural network is used to extract the road information in the complex test scenario; the specific values ​​of the weather parameters are merged to integrate the features; and then the fully connected layer is used to realize the mapping from the complex test scenario to the test result and predict the test result of the complex test scenario. Secondly, virtual simulation tools are used to obtain test results corresponding to complex test scenarios, and the correspondence between virtual simulation test scenarios and test results is used to pre-train the proxy model; then, hardware-in-the-loop and real vehicle road testing methods are used to obtain the correspondence between test scenarios and test results, and transfer learning is used to correct the proxy model. Finally, after the pre-training and correction of the surrogate model based on transfer learning, a meta-learning fine-tuning method combining dynamic learning rate scheduling is used to sample multiple tasks from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road. This simulates the few-shot adaptation process under various test scenarios. Each fine-tuning is decomposed into two parts: "inner layer task rapid adaptation" and "outer layer global generalization optimization". The inner layer uses a dynamic cosine annealing learning rate strategy to enable the model to quickly capture specific test scenario signals with a large learning rate in the early stages of each few-shot task, and then gradually reduce the step size in the later stages to suppress overfitting. The outer layer performs gradient aggregation on the performance of all tasks on the validation set and uses an annealing learning strategy to dynamically adjust the meta-learning rate. The final output aggregates parameters that can quickly adapt to new scenarios; after repeating the process of "rapid adaptation of inner tasks" and "global generalization optimization of outer layers" several times, the final parameters after fine-tuning are obtained.

2. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 1, characterized in that: The steps for designing a proxy model architecture are as follows: For the vehicle's temporal trajectory information, a gated recurrent unit (GRU) is used to extract features. The GRU includes: Implicit state of time , Input data, hidden state, and output data at each time step Implicit state in the middle Reset door , update door The reset gate controls the amount of implicit state from the previous time point written to the current GRU node. The smaller the value of the reset gate, the less previous information is retained. The update gate controls the amount of implicit state from the previous time point mixed into the implicit state at the current moment. The larger the value of the update gate, the more previous information is introduced. For static road information, a convolutional neural network is used to extract features, describing the road information from an overhead perspective. Solid lane lines are set to a pixel value of 1, dashed lane lines to a pixel value of 0.5, and the road surface area to a pixel value of 0. The road information is then converted into image information, and a convolutional neural network is used to extract features. For meteorological information, the weather is described using rainfall intensity and fog visibility, and the meteorological parameter values ​​are directly merged with the trajectory and road latent space features. The merged variables are then input into the fully connected layer, and the test results are used as the network target to construct the surrogate model.

3. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 2, characterized in that: The forward computation process of GRU is as follows: ； ； ； ； ； In the formula, and These are the weight matrix and bias matrix that the GRU model needs to train, respectively. * is for Hadamard; This is the activation function for the neural network.

4. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 2, characterized in that: The test results for the proxy model are as follows: ,in, Represents the corresponding number Each complex test scenario corresponds to the predicted test results; Representative proxy model; The first Trajectory information, road information, and weather information in a complex test scenario.

5. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 1, characterized in that: The meta-learning fine-tuning method combining dynamic learning rate scheduling is as follows: N small sample tasks are randomly selected from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road testing. Each task Includes a portion of the training set and test set ; For a small amount of sample data for inner gradient updates, Validation data for outer layer gradient evaluation and parameter summarization; Update the settlement in the inner layer and execute it for each task. The gradient descent is performed step by step, with the learning rate dynamically adjusted at each step to accelerate early convergence and fine-tuning in later stages of stabilization. learning rate of step Cosine annealing scheduling: ； In the formula, This is the upper limit of the inner learning rate; This is the lower bound of the inner layer learning rate; the inner layer refers to the layer that executes each task independently. Step parameter adaptation; the corresponding model parameters are updated as follows: ； In the formula, For the first The first task is in the inner layer. Step-updated proxy model Parameter status; In the dataset The above model Calculated loss; Indicates the parameter Find the loss function The gradient; In the outer update, the initial parameters are jointly optimized on the validation set of all tasks. And cosine annealing is also used to adjust the outer learning rate. ;No. The learning rate for dimensional updates is: ； And set the update direction to the sum of gradients of all tasks on the validation set: ； In the formula, These are the lower and upper bounds of the outer learning rate, respectively. The step size for updating the outer layer; These are the global model parameters for meta-learning, shared initial parameters during initialization, and continuously optimized and adjusted during outer layer updates.

Images