Intelligent automobile complex test scene proxy model design method based on transfer learning

Abstract

The invention relates to a complex test scene agent model design method, in particular to an intelligent automobile complex test scene agent model design method based on transfer learning, which comprises the following steps: firstly, carrying out agent model architecture design, secondly, obtaining a test result corresponding to a complex test scene by using a virtual simulation tool, and pre-training an agent model; correcting the proxy model by using transfer learning based on a small amount of data; finally, a meta-learning fine tuning method combined with dynamic learning rate scheduling is introduced, each fine tuning is decomposed into two parts of inner-layer task rapid adaptation and outer-layer global generalization optimization, an inner layer enables the model to rapidly capture specific test scene signals at a large learning rate in the early stage through a dynamic cosine annealing learning rate strategy, and then the test scene signals are rapidly obtained. The step length is gradually reduced in the later period to restrain overfitting; the outer layer carries out gradient aggregation on the expressions of all tasks on the verification set, adopts an annealing learning strategy to dynamically adjust the meta-learning rate, and ensures that full exploration can be carried out at the initial stage of meta-updating and steady convergence is carried out at the later stage.

Description

Technical Field

[0001] This invention relates to a method for designing agent models for complex test scenarios, and more particularly to a method for designing agent models for complex test scenarios of intelligent vehicles based on transfer learning. Background Technology

[0002] Intelligent vehicles have become an important development direction for the automotive industry. Before their actual implementation, extensive testing is required to ensure the completeness and robustness of their functions in various scenarios. Complex test scenarios are those with more environmental variables, traffic participants, and uncertainties compared to conventional scenarios. For example, in a conventional following scenario, only the motion characteristics of the vehicle in front and the vehicle itself on the same lane need to be considered. However, in a complex following scenario, it is necessary to consider weather conditions such as rain or fog, as well as the participation of different types of traffic objects around the vehicle. Intelligent vehicles are often more likely to expose their functional defects and design deficiencies in complex test scenarios than in conventional scenarios. Therefore, testing intelligent vehicles using complex test scenarios is highly necessary.

[0003] Testing methods for intelligent vehicles can be categorized into virtual simulation testing, hardware-in-the-loop testing, and real-vehicle road testing. Each of these methods has its advantages and disadvantages when applied to complex testing scenarios for intelligent vehicles. Virtual simulation testing allows for autonomous control of complex scenario construction, including varying rainfall intensities, fog visibility levels, different traffic participants, and diverse road conditions. It offers the lowest cost among the three methods for obtaining complex testing scenarios. However, its drawback lies in the fact that the simulation's fidelity limits the confidence level of the test results, making it difficult to accurately determine the functional boundaries of the tested intelligent vehicle. Hardware-in-the-loop testing, on the other hand, involves testing a portion of the intelligent vehicle's hardware... Embedding physical entities into a virtual simulation testing environment allows the hardware under test to respond realistically and in real-time. This method improves the realism of the testing process, making the test results more reliable than virtual simulation testing. However, building complex test scenarios using hardware-in-the-loop testing is more expensive, and switching test scenarios incurs time costs. For real-vehicle road testing, this method offers the highest level of realism, comprehensively and realistically reflecting the response and behavior of intelligent vehicles in complex test scenarios. However, the high cost of building complex test scenarios, long testing cycles, and limited number of scenarios make it difficult to fully describe the performance of intelligent vehicles in complex test scenarios. A surrogate model is a model used to predict test results for scenarios that have not been tested. Using a surrogate model to replace simulation, hardware-in-the-loop, or real-vehicle testing processes can significantly improve testing efficiency and reduce testing costs.

[0004] However, the proxy models proposed in existing research have two shortcomings. On the one hand, most existing methods rely solely on purely virtual simulation environments to construct proxy models, and their lack of realism makes it difficult for proxy models to accurately predict test results in test scenarios. On the other hand, existing research focuses on constructing proxy models within logical scenario spaces, and only constructs test scenarios within a certain range of parameter values. This process ignores the interaction characteristics between the vehicle and other traffic participants. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for designing a proxy model for complex testing scenarios of intelligent vehicles based on transfer learning, comprising the following steps:

[0006] First, the proxy model architecture is designed by segmenting the test scenario into trajectory, road, and meteorological information. Gated recurrent units are used to extract trajectory information from complex test scenarios; convolutional neural networks are used to extract road information from complex test scenarios; meteorological information is described using rainfall intensity (unit: mm / h) and fog visibility (unit: m), and the specific values ​​of meteorological parameters are integrated into features using a merging method; the merged variables are input into a fully connected layer to realize the mapping from complex test scenarios to test results, thereby predicting the test results of new complex test scenarios, and using the test results as the target of the network to build a proxy model.

[0007] Furthermore, the specific steps for designing the agent model architecture are as follows:

[0008] For the vehicle's temporal trajectory information, a gated recurrent unit (GRU) is used to extract features. The GRU includes: Implicit state of time , Input data, hidden state, and output data at each time step Implicit state in the middle Reset door , update door The reset gate controls the amount of implicit state from the previous time point written to the current GRU node. The smaller the value of the reset gate, the less previous information is retained. The update gate controls the amount of implicit state from the previous time point mixed into the implicit state at the current moment. The larger the value of the update gate, the more previous information is introduced.

[0009] Furthermore, the forward computation process of GRU is as follows:

[0010]

[0011] In the formula, and These are the weight matrix and bias matrix that the GRU model needs to train, respectively; * represents the Hadamard Product. This is the activation function for the neural network.

[0012] For static road information, a convolutional neural network is used to extract features. The road information is described from an overhead perspective. Solid lane lines are set to a pixel value of 1, dashed lane lines to a pixel value of 0.5, and the road surface area to a pixel value of 0. The road information is then converted into image information, and the convolutional neural network is used to extract features.

[0013] For meteorological information, rainfall intensity (unit: mm / h) and fog visibility (unit: m) are used to describe the weather, and the meteorological parameter values ​​are directly concatenated with the latent space features of the trajectory and road. The concatenated variables are then input into the fully connected layer, and the test results are used as the network target to construct the surrogate model.

[0014] Different test objectives may lead to different ways of expressing test results.

[0015] Secondly, virtual simulation tools are used to obtain test results corresponding to complex test scenarios, providing a foundation for building an accurate proxy model. The proxy model is pre-trained using the correspondence between a large number of virtual simulation test scenarios and test results. Then, hardware-in-the-loop and real vehicle road testing methods are used to obtain the correspondence between a small number of test scenarios and test results. Based on this small amount of data, transfer learning is used to correct the proxy model, thereby achieving the construction of a highly accurate proxy model under limited data conditions.

[0016] The test results for the proxy model are as follows:

[0017]

[0018] In the formula, Represents the corresponding number Each complex test scenario corresponds to the predicted test results; Representative proxy model; The first Trajectory information, road information, and weather information in a complex test scenario.

[0019] Finally, following the existing pre-training and small-sample correction of the surrogate model based on transfer learning, a meta-learning fine-tuning method combining dynamic learning rate scheduling is introduced. This method samples multiple tasks from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road, simulating the small-sample adaptation process under various test scenarios. Each fine-tuning is decomposed into two parts: "inner-layer task rapid adaptation" and "outer-layer global generalization optimization". The inner layer uses a dynamic cosine annealing learning rate strategy to enable the model to quickly capture specific test scenario signals with a large learning rate in the early stages of each small-sample task, and then gradually reduce the step size in the later stages to suppress overfitting. The outer layer performs gradient aggregation on the performance of all tasks on the validation set, and also uses an annealing learning strategy to dynamically adjust the meta-learning rate to ensure sufficient exploration in the early stages of meta-updates and more robust convergence in the later stages.

[0020] Furthermore, the meta-learning fine-tuning method combined with dynamic learning rate scheduling is as follows:

[0021] Randomly selected from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road data. Small sample task Each task Includes a portion of the training set and test set ; For a small amount of sample data for inner gradient updates, This serves as validation data for outer layer gradient evaluation and parameter aggregation, used to measure the generalization performance of inner layer updated parameters.

[0022] Update the settlement in the inner layer and execute it for each task. The gradient descent is used step by step, and the learning rate is dynamically adjusted at each step to accelerate early convergence and fine-tune later when the system is stable.

[0023] Specifically, no. learning rate of step Cosine annealing scheduling:

[0024]

[0025] In the formula, This is the upper limit of the inner learning rate, used for rapid learning in the early stages of training; This is the lower bound of the inner layer's learning rate, used for fine-tuning and stabilization at the end of training; the inner layer refers to the layer that executes independently for each task. Step parameter adaptation is used to quickly capture single scene features; the corresponding model parameters are updated as follows:

[0026]

[0027] In the formula, For the first The first task is in the inner layer. Step-updated proxy model Parameter status; In the dataset The above model Calculated loss; Indicates the parameter Find the loss function gradient.

[0028] Subsequently, in the outer update, the initial parameters are jointly optimized on the validation set of all tasks. And cosine annealing is also used to adjust the outer learning rate. ;No. The learning rate for dimensional updates is:

[0029]

[0030] And set the update direction to the sum of gradients of all tasks on the validation set:

[0031]

[0032] In the formula, These are the lower and upper bounds of the outer learning rate, respectively. The step size for updating the outer layer; These are the global model parameters for meta-learning, shared initial parameters during initialization, and continuously optimized and adjusted during outer layer updates. The final output aggregates these parameters to quickly adapt to new scenarios. This process repeats the "rapid adaptation of inner tasks" and "outer layer global generalization optimization" processes. After this, the final parameters after fine-tuning are obtained. .

[0033] The beneficial effects of this invention are:

[0034] The method proposed in this invention is designed for specific and complex scenarios, and can more specifically consider the individual motion characteristics and group interaction features of vehicles. This invention combines the advantages of different testing methods through transfer learning and utilizes a meta-learning fine-tuning method combined with dynamic learning rate scheduling to construct a high-accuracy surrogate model. Furthermore, this invention constructs a multimodal neural network to achieve integrated feature extraction of vehicle dynamic trajectory, road static preservation, and weather interaction dependencies, thereby improving the model's application scope. This invention features a small number of small sample tasks, high learning rate efficiency with inner-layer cosine annealing, strong stability with outer-layer annealing decay, and multi-source task simulation to improve the model's adaptability to complex scenarios. All scheduling parameters can be adjusted online according to actual computing power and sample size, facilitating engineering deployment and expansion. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0036] Figure 2 This is a schematic diagram of the proxy model network framework design of the present invention;

[0037] Figure 3 This is a schematic diagram of the gated loop unit structure of the present invention;

[0038] Figure 4 This is a schematic diagram illustrating the transfer learning process of the present invention. Detailed Implementation

[0039] This invention provides a method for designing a proxy model for complex testing scenarios of intelligent vehicles based on transfer learning. First, the proxy model architecture is designed. The proxy model architecture designed in this invention is as follows: Figure 2 As shown, the test scenario is segmented into trajectory, road, and weather information. A gated recurrent unit is used to extract trajectory information from the complex test scenario; a convolutional neural network is used to extract road information from the complex test scenario; specific values ​​of weather parameters are concatenated for feature integration; and a fully connected layer is used to map the complex test scenario to the test result, thereby enabling the prediction of test results for entirely new complex test scenarios, reducing the number of tests and improving testing efficiency. The following sections describe each part in detail:

[0040] For the vehicle's temporal trajectory information, this invention uses a gated recurrent unit (GRU) to extract features. A GRU is a variant model based on a recurrent neural network (RNN) structure. By using gating units, the RNN gains the ability to control the long-term memory and forgetting of its internal information. GRU uses two gating units, thus reducing the computational cost during training. Furthermore, GRU achieves training results similar to LSTM. This invention uses GRU to construct a discriminator. The structure of GRU is as follows: Figure 3 As shown.

[0041] exist Figure 3 middle, for The implicit state of a moment. Let be the input data, hidden state, and output data at time t, respectively. This is an intermediate, implicit state. Represents the reset door. The reset gate controls how much implicit state from the previous time step is written to the current GRU node; the smaller the reset gate value, the less previous information is retained. The update gate controls how much implicit state from the previous time step is mixed into the current implicit state; the larger the update gate value, the more previous information is introduced. The forward computation process of GRU is as follows:

[0042]

[0043] In the formula, and These are the weight matrix and bias matrix that the GRU model needs to train, respectively; * represents the Hadamard Product. This is the activation function for the neural network.

[0044] For static road information, this invention uses a convolutional neural network to extract features, describing the road information from an overhead perspective. Solid lane lines are set to a pixel value of 1, dashed lane lines to a pixel value of 0.5, and road surface areas to a pixel value of 0. The road information is then converted into image information, and the convolutional neural network is used to extract features.

[0045] For meteorological information, rainfall intensity (unit: mm / h) and fog visibility (unit: m) are used to describe the weather, and the meteorological parameter values ​​are directly concatenated with the latent space features of the trajectory and road. The concatenated variables are then input into the fully connected layer, and the test results are used as the network target to construct the surrogate model.

[0046] Regarding test results, different test objectives may require different ways of expressing the test results. For example, for intelligent vehicle perception components, the test results may be the evaluation indicators of multi-target tracking; for intelligent vehicles as a whole, the test results may be whether a collision occurs in the scenario.

[0047] Secondly, this invention uses virtual simulation tools to obtain test results corresponding to complex test scenarios in a cost-effective manner, providing a foundation for establishing accurate proxy models, such as... Figure 4 As shown, the proxy model is pre-trained using the correspondence between a large number of virtual simulation test scenarios and test results; then, the correspondence between a small number of test scenarios and test results is obtained using hardware-in-the-loop and real vehicle road testing methods. Based on this small amount of data, transfer learning is used to correct the proxy model, thereby achieving the construction of a highly accurate proxy model under limited data conditions.

[0048] The surrogate model is used in the extraction or derivation of key test scenarios. Key test scenarios are specific scenarios with a high probability of revealing performance defects in automated systems. They consist of three scenario parameters: road, traffic, and weather, and can cause the tested vehicle to collide or nearly collide with other road entities. The test results of the surrogate model are as follows:

[0049]

[0050] In the formula, Represents the corresponding number Each complex test scenario corresponds to the predicted test results; Representative proxy model; The first Trajectory information, road information, and weather information in a complex test scenario.

[0051] Finally, following the existing pre-training and few-sample correction process based on transfer learning, this invention introduces an innovative method: a meta-learning fine-tuning method combined with dynamic learning rate scheduling. This method closely integrates with the design requirements of the agent model in complex test scenarios of intelligent vehicles. By sampling multiple tasks from three data sources—virtual simulation, hardware-in-the-loop, and real-vehicle road—it simulates the few-sample adaptation process under various test scenarios, thereby significantly improving the model's generalization ability and convergence speed in real environments.

[0052] To address the slow adaptation and overfitting issues caused by scarce samples and large scene differences in existing fine-tuning stages, this invention decomposes each fine-tuning step into two parts: "rapid adaptation of the inner task" and "global generalization optimization of the outer layer." The inner layer employs a dynamic cosine annealing learning rate strategy, enabling the model to quickly capture specific test scene signals with a large learning rate in the early stages of each small-sample task, while gradually reducing the step size in the later stages to suppress overfitting. The outer layer aggregates gradients of the performance of all tasks on the validation set and uses a similar annealing strategy to dynamically adjust the meta-learning rate, ensuring sufficient exploration in the early stages of meta-updates and more robust convergence in the later stages. This fine-tuning method not only retains the advantage of traditional meta-learning in rapidly adapting to small samples but also incorporates a dynamic scheduling strategy, allowing the surrogate model to quickly adjust to its optimal state with only a few dozen real test data points, while avoiding performance degradation and instability caused by improper scheduling. The following is a detailed mathematical description of the meta-learning fine-tuning method combined with dynamic learning rate scheduling:

[0053] Randomly selected from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road data. Small sample task Each task Includes a small training set and test set ; For a small amount of sample data for inner gradient updates, The validation data for outer layer gradient evaluation and parameter aggregation is used to measure the generalization performance of the parameters after inner layer updates. In the inner layer update settlement, this invention performs [the following steps] for each task. The gradient descent is performed step by step, with the learning rate dynamically adjusted at each step to accelerate early convergence and fine-tune later when stabilization is achieved. Specifically, the first step... learning rate of step Cosine annealing scheduling:

[0054]

[0055] In the formula, This is the upper limit of the inner learning rate, used for rapid learning in the early stages of training; This is the lower bound of the inner layer's learning rate, used for fine-tuning and stabilization at the end of training; the inner layer refers to the layer that executes independently for each task. Step parameter adaptation is used to quickly capture single scene features; the corresponding model parameters are updated as follows:

[0056]

[0057] In the formula, For the first The first task is in the inner layer. Step-updated proxy model Parameter status; In the dataset The above model The calculated loss, in this embodiment, is the cross-entropy loss; Indicates the parameter Find the loss function The gradient is calculated. This model parameter update mechanism uses a large step size to quickly approach the optimal region of the new scene in the early stages of fine-tuning, while smoothly reducing the step size in the later stages to prevent overfitting. Subsequently, in the outer layer update, this invention jointly optimizes the initial parameters on the validation set of all tasks. And cosine annealing is also used to adjust the outer learning rate. ;No. The learning rate for dimensional updates is:

[0058]

[0059] And set the update direction to the sum of gradients of all tasks on the validation set:

[0060]

[0061] In the formula, Here, the global model parameters for meta-learning are used as shared initial parameters during initialization and are continuously optimized and adjusted during outer layer updates. The final aggregated output parameters can quickly adapt to new scenarios. This design allows the model to continuously acquire globally optimal information in multi-task validation while ensuring more robust parameter updates in the later stages of meta-learning through dynamic decay. The inner and outer layer processes are repeated. After this, the final parameters after fine-tuning can be obtained. All scheduling parameters and It can be adjusted online according to actual computing power and sample size.

[0062] Subsequently, conditional generation models can be used to generate scene elements. and the corresponding prediction results As a condition, train the generative model , The methods are not limited to conditional adversarial generative networks (GANs) and conditional diffusion generative models. For example, in the case of conditional adversarial GANs, when the generator and discriminator are trained to achieve consistency, the generator can be used to derive key test scenarios. This process fully utilizes the surrogate model's ability to accurately predict test results.

Claims

1. A method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning, characterized in that: The steps are as follows: First, the proxy model architecture is designed, and the test scenario is divided into trajectory, road and weather information. The gated recurrent unit is used to extract the trajectory information in the complex test scenario; the convolutional neural network is used to extract the road information in the complex test scenario; the specific values ​​of the weather parameters are merged to integrate the features; and then the fully connected layer is used to realize the mapping from the complex test scenario to the test result and predict the test result of the complex test scenario. Secondly, virtual simulation tools are used to obtain test results corresponding to complex test scenarios, and the correspondence between virtual simulation test scenarios and test results is used to pre-train the proxy model; then, hardware-in-the-loop and real vehicle road testing methods are used to obtain the correspondence between test scenarios and test results, and transfer learning is used to correct the proxy model. Finally, after the pre-training and correction of the surrogate model based on transfer learning, a meta-learning fine-tuning method combining dynamic learning rate scheduling is used to sample multiple tasks from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road. This simulates the few-shot adaptation process under various test scenarios. Each fine-tuning is decomposed into two parts: "inner layer task rapid adaptation" and "outer layer global generalization optimization". The inner layer uses a dynamic cosine annealing learning rate strategy to enable the model to quickly capture specific test scenario signals with a large learning rate in the early stages of each few-shot task, and then gradually reduce the step size in the later stages to suppress overfitting. The outer layer performs gradient aggregation on the performance of all tasks on the validation set and uses an annealing learning strategy to dynamically adjust the meta-learning rate. The final output aggregates parameters that can quickly adapt to new scenarios; after repeating the process of "rapid adaptation of inner tasks" and "global generalization optimization of outer layers" several times, the final parameters after fine-tuning are obtained.

2. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 1, characterized in that: The steps for designing a proxy model architecture are as follows: For the vehicle's temporal trajectory information, a gated recurrent unit (GRU) is used to extract features. The GRU includes: Implicit state of time , Input data, hidden state, and output data at each time step Implicit state in the middle Reset door , update door The reset gate controls the amount of implicit state from the previous time point written to the current GRU node. The smaller the value of the reset gate, the less previous information is retained. The update gate controls the amount of implicit state from the previous time point mixed into the implicit state at the current moment. The larger the value of the update gate, the more previous information is introduced. For static road information, a convolutional neural network is used to extract features, describing the road information from an overhead perspective. Solid lane lines are set to a pixel value of 1, dashed lane lines to a pixel value of 0.5, and the road surface area to a pixel value of 0. The road information is then converted into image information, and a convolutional neural network is used to extract features. For meteorological information, the weather is described using rainfall intensity and fog visibility, and the meteorological parameter values ​​are directly merged with the trajectory and road latent space features. The merged variables are then input into the fully connected layer, and the test results are used as the network target to construct the surrogate model.

3. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 2, characterized in that: The forward computation process of GRU is as follows: ； ； ； ； ； In the formula, and These are the weight matrix and bias matrix that the GRU model needs to train, respectively. * is for Hadamard; This is the activation function for the neural network.

4. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 2, characterized in that: The test results for the proxy model are as follows: ,in, Represents the corresponding number Each complex test scenario corresponds to the predicted test results; Representative proxy model; The first Trajectory information, road information, and weather information in a complex test scenario.

5. The method for designing a proxy model for complex test scenarios of intelligent vehicles based on transfer learning according to claim 1, characterized in that: The meta-learning fine-tuning method combining dynamic learning rate scheduling is as follows: N small sample tasks are randomly selected from three data sources: virtual simulation, hardware-in-the-loop, and real-vehicle road testing. Each task Includes a portion of the training set and test set ; For a small amount of sample data for inner gradient updates, Validation data for outer layer gradient evaluation and parameter summarization; Update the settlement in the inner layer and execute it for each task. The gradient descent is performed step by step, with the learning rate dynamically adjusted at each step to accelerate early convergence and fine-tuning in later stages of stabilization. learning rate of step Cosine annealing scheduling: ； In the formula, This is the upper limit of the inner learning rate; This is the lower bound of the inner layer learning rate; the inner layer refers to the layer that executes each task independently. Step parameter adaptation; the corresponding model parameters are updated as follows: ； In the formula, For the first The first task is in the inner layer. Step-updated proxy model Parameter status; In the dataset The above model Calculated loss; Indicates the parameter Find the loss function The gradient; In the outer update, the initial parameters are jointly optimized on the validation set of all tasks. And cosine annealing is also used to adjust the outer learning rate. ;No. The learning rate for dimensional updates is: ； And set the update direction to the sum of gradients of all tasks on the validation set: ； In the formula, These are the lower and upper bounds of the outer learning rate, respectively. The step size for updating the outer layer; These are the global model parameters for meta-learning, shared initial parameters during initialization, and continuously optimized and adjusted during outer-layer updates.

Images