Optimal reactive manifold active learning method based on simulation experiment

Abstract

This application relates to the field of decision-making technology for intelligent connected vehicles, and in particular to an active learning method for optimal reaction manifolds based on simulation experiments. The method includes: constructing an initial training dataset; mapping heterogeneous decision parameters of the vehicle to a low-dimensional continuous latent space and constructing a Gaussian process surrogate model; calculating the contribution of candidate samples to learning the optimal reaction manifold using a manifold-oriented acquisition function; generating a candidate sample set and selecting high-value subsets using a latent space geometry-aware space-filling design; performing cluster analysis in the latent space to obtain batch sequential sampling points; performing evaluation in parallel and updating the training dataset; retraining the model; and using a multi-dimensional adaptive stopping criterion to determine whether to terminate the iteration. This method can improve the optimization accuracy of game-theoretic decision-making in intelligent connected vehicles.

Description

Technical Field

[0001] This application relates to the field of decision-making technology for intelligent connected vehicles, and in particular to an active learning method for optimal reaction manifolds based on simulation experiments. Background Technology

[0002] In the field of autonomous driving decision-making and control for intelligent connected vehicles, key scenarios such as autonomous lane changing on highways, navigating urban intersections, following other vehicles in congested areas, and emergency obstacle avoidance involve interactive game-theoretic decision-making between the vehicle (player two) and surrounding traffic participants (player one / environment), which is a core element affecting driving safety, traffic efficiency, and passenger comfort. The payoff function for this type of problem... Factors such as traffic efficiency, collision risk, fuel economy, and ride comfort typically lack explicit analytical forms, and gradient information cannot be directly obtained. Internally, these factors involve strong coupling effects across multiple physics fields, including vehicle dynamics, traffic flow theory, environmental perception uncertainties, and multi-vehicle interactive games, making it difficult to establish accurate game models. Engineers can only obtain corresponding data between the vehicle's control strategy and the surrounding traffic environment response through a limited number of high-fidelity simulations (Prescan / CarSim co-simulation, SUMO traffic flow simulation, VISSIM microscopic traffic simulation, etc.) or real-vehicle road tests. However, each simulation or real-vehicle test comes with high computational costs and time commitments. These dual constraints of "black box characteristics" and "expensive evaluation" constitute the core challenge of game-theoretic decision optimization for intelligent connected vehicles: how to efficiently approximate the optimal control strategy from a completely unknown game structure under the condition of only a limited number of evaluations.

[0003] To address these challenges, proactive optimization methods, such as Bayesian optimization, have been widely adopted in autonomous driving strategy tuning. These methods construct Gaussian process surrogate models with equal probability, using a data acquisition function to balance exploration and development, sequentially selecting the most promising strategy parameters for evaluation, thereby approximating the optimal strategy with as few trials as possible. However, in practical engineering applications, the design requirements of autonomous driving systems often go beyond simple single optimal strategy search. Taking autonomous lane-changing scenarios on highways as an example: under different external conditions such as relative speeds and distances of surrounding vehicles, traffic density in adjacent lanes, and the vehicle's current speed, the optimal throttle opening, steering angle, braking pressure, and lane-changing decisions (whether to change lanes and when to change lanes) exhibit regular changes. System designers not only need to know the optimal control parameters under a specific condition, but also urgently need to grasp the complete mapping law of "how the optimal control strategy continuously changes with the surrounding traffic environment," that is, to construct the optimal reaction manifold from the surrounding traffic environment space to the vehicle's optimal control strategy space. This manifold can support advanced engineering tasks such as adaptive decision-making under all operating conditions, robust control under risk situation awareness, and safety redundancy decision-making in emergency scenarios, which is of fundamental significance for improving the operating condition adaptability and intelligence level of intelligent connected vehicles.

[0004] Existing mainstream optimization methods have significant technical limitations when addressing the game-theoretic decision-making optimization needs of intelligent connected vehicles.

[0005] First, in terms of objective paradigm, classic active optimization methods, represented by Bayesian optimization, aim to find a single global optimum. Their core mechanisms, such as sampling function design and stopping criterion formulation, revolve around the optimal value, lacking the ability to model and optimize policy mapping relationships (i.e., the optimal response manifold). Even if Bayesian optimization is run multiple times independently to traverse different surrounding traffic environment parameters, the sampling efficiency will be low due to the independence of each optimization process and the inability to share information, and the continuity and smoothness of the manifold cannot be guaranteed.

[0006] Second, regarding the adaptability of variable types, autonomous driving strategy optimization commonly involves mixed-type decision variables: for example, continuous variables (throttle opening, steering angle, braking pressure, expected following distance) and discrete categorical variables (lane-changing decisions (left lane change / right lane change / lane keeping), driving mode selection (economy mode / comfort mode / sport mode), traffic light response strategies (constant speed / deceleration to yield / acceleration to rush the light)). The kernel function of traditional Gaussian processes is difficult to define effectively in this mixed space. Even with simple one-hot encoding or integer encoding, problems such as distance metric failure and covariance structure distortion lead to low modeling accuracy of the surrogate model, inaccurate uncertainty estimation, and a significant decrease in sampling efficiency.

[0007] Third, regarding boundary sampling capabilities, the accurate construction of the optimal response manifold requires maintaining global modeling accuracy across the entire surrounding traffic environment. This is especially true in the boundary regions of the environment (such as dangerous conditions involving extreme relative speeds or minimal safety distances), where the payoff function typically changes drastically, placing extremely high demands on the directionality and density of the sampling strategy. Existing sampling functions (such as those aimed at improving EI) naturally tend to explore local areas near the current optimal solution, lacking a directional guidance mechanism for actively exploring boundary regions. This results in insufficient accuracy in manifold boundary characterization, while boundary information is precisely the key to accurately constructing a complete mapping relationship.

[0008] Fourth, in terms of evaluation strategies, existing methods mostly employ single-point sequential sampling, making it difficult to fully utilize parallel simulation computing resources. Even with batch Bayesian optimization methods, their batch selection strategies are still designed around "point optimality," lacking a batch diversity guarantee mechanism for manifold learning. Furthermore, existing methods lack mechanisms for quantitatively evaluating the quality of mapped manifold estimations and for adaptive stopping. In practical autonomous driving strategy development, this often results in either incomplete or distorted policy mapping relationships due to premature stopping, or a waste of valuable simulation computing resources due to blindly continuing sampling.

[0009] In summary, in the typical high-cost, black-box optimization scenario of game-theoretic decision-making in intelligent connected vehicles, existing technologies have significant shortcomings in terms of using the "optimal reaction manifold" as the learning objective, supporting mixed-type variables, and achieving efficient sequential sampling and adaptive stopping. Summary of the Invention

[0010] Therefore, it is necessary to provide an optimal reaction manifold active learning method based on simulation experiments that can improve the accuracy of game-theoretic decision optimization for intelligent connected vehicles, addressing the aforementioned technical problems.

[0011] An active learning method for optimal reactive manifolds based on simulation experiments, the method comprising: Step 1: Obtain the heterogeneous decision parameters of the intelligent connected vehicle as input variables. The heterogeneous decision parameters include continuous variables and discrete categorical variables. Obtain the comprehensive benefit index corresponding to the heterogeneous decision parameters as output variables and construct the initial training dataset. The continuous variables include throttle opening, steering angle, braking pressure, and expected following distance. The discrete categorical variables include lane change decision, driving mode selection, and traffic light response strategy. Step 2: Map the heterogeneous decision parameters to a low-dimensional continuous latent space, and construct a Gaussian process surrogate model on the low-dimensional continuous latent space. The Gaussian process surrogate model is used to predict the mean and variance of the comprehensive return index. Step 3: Based on the current Gaussian process surrogate model, a manifold-oriented acquisition function is used to calculate the contribution of each candidate sample in the candidate sample set to learning the optimal response manifold. The optimal response manifold is used to characterize the mapping relationship between the vehicle's optimal control strategy and environmental changes under different surrounding traffic conditions. The surrounding traffic environment includes the relative speed and relative distance of adjacent vehicles, traffic density of adjacent lanes, the current speed of the vehicle, traffic light phase and remaining time, and pedestrian crossing probability. Step 4: Use latent space geometry-aware spatial filling design to generate a candidate sample set covering the decision space, and select a high-value candidate sample subset from the candidate sample set based on contribution. Step 5: Map the high-value candidate sample subset to the latent space through a deep embedding network, perform cluster analysis in the latent space, and use the representative samples of each cluster as batch sequential sampling points. Step 6: Perform high-fidelity simulation evaluation in parallel on the batch sequential sampling points, obtain the comprehensive benefit observation value corresponding to the batch sequential sampling points, and add the newly obtained observation value to the training dataset; Step 7: Retrain the deep embedding network and Gaussian process proxy model based on the updated training dataset; Step 8: Use a multi-dimensional adaptive stopping criterion to determine whether the currently estimated optimal reaction manifold tends to be stable or whether the uncertainty is below the threshold. If the stopping condition is met, terminate the iteration; otherwise, return to step 3 to continue execution.

[0012] The above-mentioned optimal reaction manifold active learning method based on simulation experiments first utilizes a block encoder to uniformly map continuous and discrete categorical variables in intelligent connected vehicle decision-making to a low-dimensional continuous latent space, solving the modeling problem that traditional Gaussian process kernel functions cannot be effectively defined on mixed-type variables. Simultaneously, contrastive learning and knowledge-enhancing regularization terms are introduced for self-supervised pre-training to obtain semantically preserved embedding representations. Combined with the construction of a Gaussian process surrogate model in the latent space and joint fine-tuning by maximizing the marginal log-likelihood with physical consistency constraints, high-precision prediction and uncertainty quantification of comprehensive benefit indicators are achieved under limited sample conditions, solving the problems of low prediction accuracy caused by small sample overfitting and the failure of heterogeneous input metrics. Building upon this foundation, an adaptive sampling method oriented towards the optimal response manifold is designed. A manifold-guided sampling function is constructed to directly measure the contribution of candidate samples to the learning of the mapping relationship, breaking through the paradigm limitation of traditional Bayesian optimization, which focuses only on the search for a single optimal policy. Furthermore, by combining latent space geometry-aware spatial filling design and cluster analysis, a sequential sample solving algorithm, unrestricted by batch size, is constructed. This maintains sample diversity while preserving efficient point usage, solving the core problem that existing methods cannot efficiently learn the complete mapping relationship from the surrounding traffic environment to the vehicle's optimal control policy. Simultaneously, a multi-dimensional adaptive stopping criterion integrating manifold structure stability and uncertainty convergence is established. By calculating the maximum Hausdorff distance between manifolds within a sliding window and the maximum shrinkage rate of uncertainty, sampling is automatically terminated when the manifold estimation tends to stabilize or the uncertainty falls below a threshold. This avoids premature stopping leading to mapping relationship distortion or the waste of high simulation resources caused by blind sampling. Therefore, this application introduces a deep embedded probabilistic surrogate model into the active learning framework to uniformly represent heterogeneous inputs, and designs a manifold-oriented acquisition function and a batch sampling strategy to efficiently approximate the global mapping relationship. Under a limited simulation evaluation budget, this significantly improves the characterization accuracy and learning efficiency of the optimal reaction manifold of intelligent connected vehicles, thereby improving the game decision optimization accuracy of intelligent connected vehicles. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating an active learning method for optimal reactive manifolds based on simulation experiments in one embodiment. Figure 2 This is a flowchart illustrating the active learning algorithm in one embodiment. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0015] In one embodiment, such as Figure 1 As shown, an active learning method for optimal reactive manifolds based on simulation experiments is provided, including the following steps: Step 1: Obtain the heterogeneous decision parameters of the intelligent connected vehicle as input variables. The heterogeneous decision parameters include continuous variables and discrete categorical variables. Obtain the comprehensive benefit index corresponding to the heterogeneous decision parameters as output variables and construct the initial training dataset. The continuous variables include throttle opening, steering angle, braking pressure, and expected following distance. The discrete categorical variables include lane change decision, driving mode selection, and traffic light response strategy.

[0016] The heterogeneous decision-making parameters of intelligent connected vehicles include quantitative factors, qualitative factors, and functional inputs. Quantitative factors are continuous variables, such as throttle opening, steering angle, braking pressure, and expected following distance. Qualitative factors are discrete categorical variables, such as lane-changing decisions (left lane change / right lane change / lane keeping), driving mode selection (economy mode / comfort mode / sport mode), and traffic light response strategies (constant speed / slowing down to yield / accelerating to beat the light). Functional inputs are functions defined in a continuous domain, such as time-varying expected trajectory curves, spatially distributed obstacle distance functions, or historical speed sequences. Comprehensive benefit indicators include heterogeneous decision-making parameters such as traffic efficiency, collision risk, fuel economy, and ride comfort.

[0017] Step 2: Map the heterogeneous decision parameters to a low-dimensional continuous latent space, and construct a Gaussian process surrogate model on the low-dimensional continuous latent space. The Gaussian process surrogate model is used to predict the mean and variance of the comprehensive return index.

[0018] First, a block encoder is designed to encode each type of input into a numerical vector.

[0019] The quantitative factor is normalized to eliminate the influence of dimensions. ; in, and They represent the first The maximum and minimum values ​​of each continuous design parameter are used. Discrete categorical parameters are assigned a unique vector representation for each discrete combination using one-hot encoding or random fixed embedding. The functional performance curves are dimensionality-reduced using functional principal component analysis, and their dimensionality is reduced to that of the basis functions. Expand ： ;

[0020] Take the expansion coefficient vector As a feature representation. If the functional data is given in a discrete form with a high sampling rate, time series feature extraction methods can also be used.

[0021] The block encoding results are concatenated to construct a deep embedding network. heterogeneous input space Mapping to a low-dimensional continuous latent space : ; To characterize the uncertainty of embeddings, a variational autoencoder architecture is introduced, extending deterministic embeddings to probability distributions, and defining the encoder output as a variational posterior: ; in and By embedded network After the wiring is mapped, the parameters are uniformly categorized. .

[0022] In the subspace We construct a Gaussian process regression model as the main thread for surrogate modeling: ; The mean function It can be a constant or a linear function. Kernel function Take RBF core ; Other kernel functions, such as the Matern kernel, can also be used.

[0023] The goal of the pre-training phase is to learn a structurally sound and semantically preserving latent space under conditions without black-box evaluation. That is, optimizing parameters A multi-objective contrastive learning framework is constructed to jointly optimize the three losses: ; in, To compare the losses, positive and negative sample pairs are constructed through heterogeneous data augmentation; The variational autoencoder loss consists of the reconstruction loss and the KL divergence. The knowledge-enhancing regularization term is constructed based on the sample similarity provided by domain experts.

[0024] Data augmentation strategies include: adding noise to quantitative factors, randomly permuting qualitative factors within equivalence classes, truncating and scaling functional inputs, and discarding nodes and perturbing edges in graph structures. The specific form of the contrastive loss is: ; The variational autoencoder loss consists of the reconstruction loss and the KL divergence. ; a priori Refactoring items Decompose based on input type: continuous variables are distributed using a Gaussian distribution, discrete variables are distributed using a categorical distribution, and functional variables are reconstructed using function values. To enhance knowledge through regularization, samples with similarity provided by domain experts are used. Construct a similarity matrix get, ; This regularization term encourages similar samples to remain close together in the latent space, aligning the embedding space with human cognition.

[0025] The goal of the fine-tuning phase is to utilize combustion chamber evaluation data that includes input-output pairs. Adapt to downstream tasks. First, fix the embedded network parameters. Only optimize the hyperparameters of the Gaussian process and : ; in, , ; For the kernel matrix; Physical consistency loss is used to quantify model predictions. The deviation between the (i.e., the comprehensive benefit index) and the vehicle dynamics constraints and game-theoretic decision priors: ; in, The differential equations or conservation law operators representing the dominating system are concretized into four categories of constraints in the game-theoretic decision-making scenario of intelligent connected vehicles: vehicle dynamics constraints, kinematic consistency constraints, game safety constraints, and prior traffic rule constraints. The introduction of these constraints not only enhances the extrapolation capability and interpretability of the Gaussian process surrogate model, but also ensures that, under a limited simulation evaluation budget, the policy parameters recommended by the active learning method are physically feasible, safe in the game, and engineering-practical, thus achieving a paradigm shift from "data-driven" to "physical and data-driven."

[0026] When the sample size is sufficient, the same objective function can be used to further fine-tune the embedded network parameters. To avoid fine-tuning To disrupt the semantics obtained from pre-training, two methods are employed: firstly, an early stopping mechanism is introduced; secondly, a penalty term for changes in pre-training parameters is added to the loss function. ; in It is a hyperparameter. These are the pre-trained parameters. When the sample size is sufficient, the same objective function can be used to further fine-tune the embedded network parameters. The predicted mean and predicted variance are respectively: ; ; in, It is a mean function. The covariance vector between test points and training points ， The kernel matrix of the training set. To observe the noise variance, It is the identity matrix. This is the output vector for the game-theoretic decision-making of intelligent connected vehicles. For the mean vector of the training set, To predict the mean, To predict variance, the superscript T indicates the transpose operation. The kernel function value represents the test point itself. The predicted mean, as the optimal estimate for game-theoretic decision evaluation of intelligent connected vehicles, can be directly used for design scheme evaluation and performance prediction. The predicted variance provides the confidence interval of the prediction, helping engineers judge the reliability of the prediction results. The predicted variance can be decomposed into cognitive uncertainty and random uncertainty. Cognitive uncertainty is caused by data sparsity and can be reduced by increasing training data. Random uncertainty is caused by inherent noise in the data itself and cannot be eliminated. The predicted mean and predicted variance of the game-theoretic decision simulation evaluation of intelligent connected vehicles provide a complete probabilistic prediction for the optimization of game-theoretic decision-making in intelligent connected vehicles.

[0027] Step 3: Based on the current Gaussian process surrogate model, the contribution of each candidate sample in the candidate sample set to learning the optimal response manifold is calculated using a manifold-oriented acquisition function. The optimal response manifold is used to characterize the mapping relationship between the vehicle's optimal control strategy and environmental changes under different surrounding traffic conditions. The surrounding traffic environment includes the relative speed and relative distance of adjacent vehicles, traffic density of adjacent lanes, the current speed of the vehicle, traffic light phase and remaining time, and pedestrian crossing probability.

[0028] Consider a game theory decision-making scenario, where the input consists of the driver's strategy and our strategy. And surrounding traffic environment parameters - opponent strategy Composition, output This represents the total revenue of the vehicle minus our revenue: ; The above equation is an accurate estimate of the optimal reaction manifold under a finite sampling budget.

[0029] This manifold depicts the complete mapping relationship from the surrounding traffic environment space to the vehicle's optimal control strategy.

[0030] To accurately estimate the manifold under a finite sampling budget, the following manifold-oriented acquisition function is designed to measure sample points. For learning Contributions: 1. To improve manifold learning, define variance to reduce gain: ; in Estimated through posterior sampling. The expected gain acquisition function is: ; The above gain calculation is complex; in practice, an approximation based on sensitivity can be used. ; in This indicates the sensitivity of the function value change at that point to the optimal solution, which can be estimated using the implicit function theorem combined with Gaussian process covariance propagation.

[0031] 2. Manifold Boundary Exploration (MBE). This involves finding the optimal manifold that satisfies... Define auxiliary functions ,but The region corresponds to the potential optimal manifold. Construct the acquisition function: ; in It is the standard normal probability density function. for The estimated standard deviation can be obtained through gradient propagation of a Gaussian process: ;

[0032] 3. Manifold Uncertainty Sampling (MUS). This directly measures the potential contribution of points to the learned manifold. ; Weight Reflecting the point's impact on the estimation The importance of this, a feasible design is: ; Guide the sampling to be close to the currently guessed optimal manifold , This is the scale parameter.

[0033] If each of the three methods has its own advantages and disadvantages, an adaptive switching strategy should be adopted to dynamically select the most suitable acquisition function based on the current model state.

[0034] Step 4: Use latent space geometry perception to fill the space to generate a candidate sample set that covers the decision space, and select a high-value candidate sample subset from the candidate sample set according to the contribution.

[0035] In the original heterogeneous space Generate initial samples with uniform coverage and define the mixing distance function: ; To avoid the subjectivity of manually set coefficients, an embedded perception-based coefficient estimation method is proposed: ; Align the distance metric with the latent space geometry.

[0036] Based on the aforementioned distance, a structure is constructed in the input space. and potential space The space filling design has good uniformity. :Require like yes The Latin hypercube design on top, and right Coverage radius ; Minimize as much as possible. This application employs the following iterative optimization algorithm to construct the above design: ① Initialization: Generation Latin hypercube design Solve the inverse mapping ; Obtain initial design ,calculate right Coverage radius

[0037] ② Disturbance: Random permutation design Given two rows of elements in a certain column, generate a new latent space design. Inverse mapping yields New design in and calculate .

[0038] ③Update: If Then update .

[0039] ④ Iteration: Repeat steps ② and ③ until convergence or the maximum number of iterations is reached.

[0040] The above methods can ensure the uniformity of both the input and latent spaces, but the inverse mapping may suffer from ill-conditioned problems such as no solution, multiple solutions, or unstable solutions. This application introduces the original spatial physical constraints as regularization terms: ; in Penalties for violating physical constraints, including boundary constraints. Equality constraints or inequality constraints Furthermore, a multi-starting-point optimization strategy is employed to address pathological problems.

[0041] After generating the candidate sample set, the acquisition function value is calculated for each candidate sample. The candidate samples are then sorted in descending order of their acquisition function values, and the top few are selected. A set of efficient candidate points ,in This represents the screening ratio.

[0042] Step 5: Map the high-value candidate sample subset to the latent space through a deep embedding network, perform cluster analysis in the latent space, and use the representative samples of each cluster as batch sequential sampling points.

[0043] A subset of high-value candidate samples is mapped to the latent space through a deep embedding network to obtain a latent space point set. The latent space point set is then clustered by K-means and divided into several clusters with the same number as the batch size. Calculate the center of each cluster in the latent space, and map the cluster centers in the latent space back to the original decision space through nearest neighbor search. Use the mapped sample points as representative samples of the cluster.

[0044] Step 6: Perform high-fidelity simulation evaluation in parallel on the batch sequential sampling points, obtain the comprehensive benefit observation value corresponding to the batch sequential sampling points, and add the newly obtained observation value to the training dataset.

[0045] For the batch sequential sampling points selected in step 5, perform high-fidelity simulations (including Prescan / CarSim joint simulation, SUMO traffic flow simulation, and VISSIM micro traffic simulation) or real-vehicle road tests in parallel to obtain the corresponding comprehensive benefit observations, and add the newly obtained observations to the training dataset.

[0046] Step 7: Retrain the deep embedding network and Gaussian process proxy model based on the updated training dataset.

[0047] Based on the updated training dataset, repeat the training process in step 2 to retrain the deep embedding network and the Gaussian process surrogate model, and update the model parameters.

[0048] Step 8: Use a multi-dimensional adaptive stopping criterion to determine whether the currently estimated optimal reaction manifold tends to be stable or whether the uncertainty is below the threshold. If the stopping condition is met, terminate the iteration; otherwise, return to step 3 to continue execution.

[0049] An adaptive stopping criterion is constructed to integrate the manifold estimation error and its uncertainty evolution, and sampling is terminated when the manifold structure tends to stabilize, the uncertainty is below a threshold, or the evaluation budget is exhausted.

[0050] Define the sliding window size The maximum shrinkage rate of the uncertainty of the optimal reaction manifold within the window is: ; in ; The optimal reaction manifold currently estimated The maximum prediction variance.

[0051] The maximum Hausdorff distance between the optimal reactive manifolds within the window is: ; The Hausdorff distance between point sets is defined as:

[0052] Based on the above definitions, a multi-dimensional stopping criterion is constructed: Criterion 1 is that the manifold structure is stable, that is: ; Criterion 2 is uncertainty convergence, that is: ; Guideline 3 is budget exhaustion, i.e., reaching the maximum number of assessments. The iteration terminates when the above stopping conditions are met; otherwise, it returns to step 3 to continue execution.

[0053] The above-mentioned optimal reaction manifold active learning method based on simulation experiments first utilizes a block encoder to uniformly map continuous variables (throttle opening, steering angle, braking pressure, expected following distance, etc.) and discrete categorical variables (lane change decision, driving mode selection, traffic light response strategy, etc.) in intelligent connected vehicle decision-making to a low-dimensional continuous latent space. This solves the modeling problem that traditional Gaussian process kernel functions cannot be effectively defined on mixed-type variables. Simultaneously, contrastive learning and knowledge enhancement regularization terms are introduced for self-supervised pre-training to obtain semantically preserved embedding representations. Combined with the Gaussian process surrogate model constructed on the latent space and joint fine-tuned by maximizing the marginal log-likelihood with physical consistency constraints, this achieves high-precision prediction and uncertainty quantification of comprehensive benefit indicators (traffic efficiency, collision risk, fuel economy, ride comfort, etc.) under limited sample conditions. This solves the problems of low prediction accuracy caused by small sample overfitting and the failure of heterogeneous input metrics. Building upon this foundation, an adaptive sampling method oriented towards the optimal response manifold is designed. A manifold-guided sampling function (including expectation-improving manifold learning, manifold boundary exploration, and manifold uncertainty sampling) is constructed to directly measure the contribution of candidate samples to the learning of the mapping relationship. This breaks through the paradigm limitation of traditional Bayesian optimization, which focuses solely on searching for a single optimal policy. Furthermore, by combining latent space geometry-aware spatial filling design and cluster analysis, a sequential sample solving algorithm, unrestricted by batch size, is constructed. This maintains sample diversity while preserving efficient point usage, addressing the core problem that existing methods cannot efficiently learn the complete mapping relationship from the surrounding traffic environment to the vehicle's optimal control policy. Simultaneously, a multi-dimensional adaptive stopping criterion integrating manifold structural stability and uncertainty convergence is established. By calculating the maximum Hausdorff distance between manifolds within a sliding window and the maximum shrinkage rate of uncertainty, sampling automatically terminates when the manifold estimation tends to stabilize or the uncertainty falls below a threshold. This avoids premature stopping leading to mapping relationship distortion or the waste of high simulation resources caused by blind sampling. Therefore, this application introduces a deep embedded probabilistic surrogate model into the active learning framework to uniformly represent heterogeneous inputs, and designs a manifold-oriented acquisition function and a batch sampling strategy to efficiently approximate the global mapping relationship. Under a limited simulation evaluation budget, this significantly improves the characterization accuracy and learning efficiency of the optimal reaction manifold of intelligent connected vehicles, thereby improving the game decision optimization accuracy of intelligent connected vehicles.

[0054] In one embodiment, the manifold-guided acquisition function is the desired improved manifold learning function as follows: ; in, Indicates the decision parameters of the vehicle and surrounding traffic environment parameters The input points are composed of and These represent the prediction mean and prediction variance of the Gaussian process surrogate model at the input point, respectively. exist Performance values ​​observed at the point The reduction in variance of the post-optimal reaction manifold. This is the symbol for a Gaussian distribution.

[0055] Specifically, this acquisition function directly measures the expected contribution of candidate samples to reducing the uncertainty of optimal response manifold estimation. Compared to traditional acquisition functions that only focus on a single optimal point, this acquisition function is directly geared towards the manifold learning task, enabling more efficient allocation of the finite sampling budget and accelerating the convergence of manifold estimation.

[0056] In one embodiment, the manifold-guided acquisition function is the manifold boundary exploration function: ; in, To predict the mean values ​​for the vehicle's decision parameters The gradient norm, for The estimated standard deviation, It is the standard normal probability density function.

[0057] Specifically, this acquisition function, based on the mathematical property that the gradient is zero on the optimal manifold, locates the potential manifold boundary region by predicting the gradient norm of the mean. It can efficiently identify the boundary location of the optimal reactive manifold, significantly improving the accuracy of manifold boundary characterization.

[0058] In one embodiment, the manifold-guided acquisition function is the manifold uncertainty sampling function. ; in, The prediction variance of the Gaussian process surrogate model. This is the weighting function.

[0059] Specifically, this sampling function multiplies the prediction variance by a weighting function, guiding the sampling to approach the currently guessed optimal manifold region. It is computationally efficient, easy to implement, and suitable for engineering applications with limited computational resources.

[0060] In one embodiment, the latent space geometry-aware space-filling design is constructed using the following hybrid distance function: ; in, For a set of continuous variables, For a set of discrete categorical variables, For the first The class difference penalty coefficient for a discrete categorical variable. For indicator functions, For the first The sample point at the th th The values ​​that can be taken on a continuous variable, For the first The value of each sample point on the $k$-th continuous variable. For the first Sample points of decision parameters for intelligent connected vehicles. For the first Sample points of decision parameters for intelligent connected vehicles. For the first The sample point at the th th The values ​​that can be taken on a discrete categorical variable. For the first The sample point at the th th The values ​​that can be taken on a discrete categorical variable.

[0061] Specifically, this hybrid distance function considers both the numerical differences of continuous variables and the categorical differences of discrete categorical variables. It effectively measures the similarity between sample points of heterogeneous decision parameters in intelligent connected vehicles, ensuring uniform coverage of the initial samples in the decision space.

[0062] In one embodiment, the category difference penalty coefficient is: ; in, For deep embedding in the network, These are learnable parameters for deeply embedded networks.

[0063] This coefficient estimates the difference penalty for discrete categorical variables in the original space by measuring the distance between the deep embedded network and the latent space. This avoids the subjectivity of manually setting the penalty coefficient and automatically aligns the distance metric with the geometry of the latent space.

[0064] In one embodiment, cluster analysis is performed in the latent space, including: A subset of high-value candidate samples is mapped to the latent space through a deep embedding network to obtain a latent space point set. The latent space point set is then clustered by K-means and divided into several clusters with the same number as the batch size. Calculate the center of each cluster in the latent space, and map the cluster centers in the latent space back to the original decision space through nearest neighbor search. Use the mapped sample points as representative samples of the cluster.

[0065] Specifically, let the current iteration round be... An evaluation dataset already exists. The probabilistic surrogate model provides the predicted mean. and prediction variance Using latent space geometric perception for spatial filling design, in the input space Generate a large number of candidate sample sets ,in This ensures that the candidate set covers the parameter space uniformly.

[0066] Calculate the acquisition function value for each candidate sample. The aforementioned options can be selected. , or Alternatively, an adaptive switching strategy can be used for dynamic selection. Candidate samples are sorted in descending order of their acquisition function values, and the top-ranked samples are selected. A set of efficient candidate points ,in This represents the screening ratio.

[0067] High-value candidate set Mapping to latent space through deep embedding networks The latent space point set is obtained. K-means clustering is performed in the latent space to divide it into... Cluster Simultaneously record the corresponding data in the original space. Cluster Calculate the center of each cluster: ; The latent space cluster centers are mapped back to the original space as sequential sample points through nearest neighbor search: ; This principle aims to retain efficient sampling points while avoiding overly concentrated sampling within the same cluster.

[0068] This clustering analysis method employs a space-filling design during the candidate sample generation phase to ensure global coverage; the sampling function focuses on high-value regions; the clustering algorithm divides high-value regions into several sub-regions to avoid over-concentration of batch sampling points; and the representative sample selection strikes a balance between utility and diversity. The entire algorithm is unaffected by batch size. It is not limited and can be flexibly adapted to different parallel computing resources.

[0069] In one embodiment, the multi-dimensional adaptive stopping criterion includes: Calculate the maximum Hausdorff distance between optimal reactive manifolds within the sliding window, and the maximum shrinkage rate of manifold uncertainty within the window. The stopping condition is met when the maximum Hausdorff distance is less than the first threshold, the maximum shrinkage rate is less than the second threshold, and the maximum prediction variance on the current manifold is less than the third threshold.

[0070] Specifically, the stopping criterion comprehensively judges the manifold structure stability and uncertainty convergence from two dimensions, avoiding inaccurate manifold estimation due to premature stopping, and also avoiding waste of resources caused by blindly continuing sampling.

[0071] In one embodiment, the maximum shrinkage rate of the manifold uncertainty within the window is... ; in, To adjust the sliding window size, For the current iteration round, For the first Optimal Reaction Manifold during Round Iteration Maximum prediction variance To prevent the division by zero small constant, No. The maximum prediction variance during rounds of iteration.

[0072] Specifically, the formula quantifies the maximum relative rate of decrease of manifold uncertainty within the sliding window, and can sensitively capture the stagnation state of uncertainty convergence, providing a reliable quantitative basis for automatic stopping.

[0073] In one embodiment, the maximum Hausdorff distance between the optimal reactive manifolds within the sliding window is: ; ; in, For two manifold point sets and The Hausdorff distance between them is calculated using a mixed distance function. As a measure of distance between points Indicates the decision parameters of the vehicle and surrounding traffic environment parameters The input points are composed of For the first The optimal reaction manifold estimated during round iteration, For the first The optimal reaction manifold estimated during round iteration, manifold A sample point on the surface. This represents the mixing distance between two sample points.

[0074] Specifically, the Hausdorff distance measures the maximum mismatch between two manifold point sets, directly quantifying the structural differences in manifold estimations between different iterations, and providing an intuitive and reliable criterion for stopping decisions regarding structural stability.

[0075] In a specific embodiment, the four modules of initial space filling design, model training, batch sequential sampling, and stopping criteria are combined to obtain Algorithm 1 and Algorithm 2 in Tables 1 and 2. Figure 2 The active learning algorithm corresponding to this application is shown in Table 1, which contains the first 29 rows of Algorithm 1, and Table 2, which contains the last 12 rows of Algorithm 1.

[0076] When solving for batch sequential sampling, the candidate sample set The size remains consistent, but different methods are used in each iteration. This ensures that the complexity of batch sequential design solutions does not increase with iterations, and also guarantees that sequential sampling covers the entire space. The optimal reactive manifold solution strategy is to first... Discretize into Then to Find the optimal policy for each point in the equation. The size increases with the sample size to ensure the accuracy of the optimal manifold; Table 1

[0077] Table 2

[0078] The parameters related to the stopping criteria in the algorithm include , , as well as Improper parameter settings can lead to premature shutdown or wasted budget. Determined based on computing resources and available time; Coverage radius threshold The discretization size can be set according to spatial geometric relationships. The function; the threshold of relative change in uncertainty It can be set to an absolute value, such as 0.01; absolute threshold for uncertainty. Need to be based on the response The fluctuation range can be adaptively set, for example, to the standard deviation of the observed response sample. times.

[0079] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An active learning method for optimal reactive manifolds based on simulation experiments, characterized in that, The method includes: Step 1: Obtain the heterogeneous decision parameters of the intelligent connected vehicle as input variables. The heterogeneous decision parameters include continuous variables and discrete categorical variables. Obtain the comprehensive benefit index corresponding to the heterogeneous decision parameters as output variables and construct the initial training dataset. The continuous variables include throttle opening, steering angle, braking pressure, and expected following distance. The discrete categorical variables include lane change decision, driving mode selection, and traffic light response strategy. Step 2: Map the heterogeneous decision parameters to a low-dimensional continuous latent space, and construct a Gaussian process surrogate model on the low-dimensional continuous latent space. The Gaussian process surrogate model is used to predict the mean and variance of the comprehensive return index. Step 3: Based on the current Gaussian process surrogate model, a manifold-guided acquisition function is used to calculate the contribution of each candidate sample in the candidate sample set to learning the optimal response manifold. The optimal response manifold is used to characterize the mapping relationship between the vehicle's optimal control strategy and environmental changes under different surrounding traffic conditions. The surrounding traffic environment includes the relative speed and relative distance of adjacent vehicles, traffic density of adjacent lanes, the current speed of the vehicle, traffic light phase and remaining time, and pedestrian crossing probability. Step 4: Generate a candidate sample set covering the decision space using latent space geometry-aware spatial filling design, and select a high-value candidate sample subset from the candidate sample set based on the contribution. Step 5: Map the high-value candidate sample subset to the latent space through a deep embedding network, perform cluster analysis in the latent space, and use the representative samples of each cluster as batch sequential sampling points. Step 6: Perform high-fidelity simulation evaluation in parallel on the batch sequential sampling points to obtain the comprehensive benefit observation value corresponding to the batch sequential sampling points, and add the newly obtained observation value to the training dataset; Step 7: Retrain the deep embedding network and the Gaussian process proxy model based on the updated training dataset; Step 8: Use a multi-dimensional adaptive stopping criterion to determine whether the currently estimated optimal reaction manifold is stable or whether the uncertainty is below the threshold. If the stopping condition is met, terminate the iteration; otherwise, return to step 3 to continue execution.

2. The method according to claim 1, characterized in that, The manifold-guided acquisition function is the desired improved manifold learning function. in, Indicates the decision parameters of the vehicle and surrounding traffic environment parameters The input points are composed of and These represent the prediction mean and prediction variance of the Gaussian process surrogate model at the input point, respectively. exist Performance values ​​observed at the point The reduction in variance of the post-optimal reaction manifold. This is the symbol for a Gaussian distribution.

3. The method according to claim 1, characterized in that, The manifold-guided acquisition function is the manifold boundary exploration function. in, To predict the mean values ​​of the vehicle's decision parameters The gradient norm, for The estimated standard deviation, It is the standard normal probability density function.

4. The method according to claim 1, characterized in that, The manifold-guided acquisition function is the manifold uncertainty sampling function. in, The prediction variance of the Gaussian process surrogate model. This is the weighting function.

5. The method according to claim 1, characterized in that, The latent space geometry-aware spatial filling design is constructed using the following hybrid distance function: in, For a set of continuous variables, For a set of discrete categorical variables, For the first The class difference penalty coefficient for a discrete categorical variable. For indicator functions, For the first The sample point at the th th The values ​​that can be taken on a continuous variable, For the first The sample point at the th th The values ​​that can be taken on a continuous variable, For the first Sample points of decision parameters for intelligent connected vehicles For the first Sample points of decision parameters for intelligent connected vehicles For the first The sample point at the th th The values ​​of each discrete categorical variable For the first The sample point at the th th The values ​​that can be taken on a discrete categorical variable.

6. The method according to claim 5, characterized in that, The category difference penalty coefficient is in, For deep embedding in the network, These are learnable parameters for deeply embedded networks.

7. The method according to claim 1, characterized in that, Cluster analysis is performed in the latent space, including: The high-value candidate sample subset is mapped to the latent space through a deep embedding network to obtain a latent space point set. The latent space point set is then subjected to K-means clustering and divided into several clusters with the same number as the batch size. Calculate the center of each cluster in the latent space, and map the cluster centers in the latent space back to the original decision space through nearest neighbor search. Use the mapped sample points as representative samples of the cluster.

8. The method according to claim 1, characterized in that, The multi-dimensional adaptive stopping criteria include: Calculate the maximum Hausdorff distance between optimal reactive manifolds within the sliding window, and the maximum shrinkage rate of manifold uncertainty within the window. The stopping condition is met when the maximum Hausdorff distance is less than a first threshold, the maximum shrinkage rate is less than a second threshold, and the maximum prediction variance on the current manifold is less than a third threshold.

9. The method according to claim 8, characterized in that, The maximum shrinkage rate of the manifold uncertainty within the window is... in, To adjust the sliding window size, For the current iteration round, For the first Optimal Reaction Manifold during Round Iteration Maximum prediction variance To prevent the division by zero small constant, No. The maximum prediction variance during rounds of iteration.

10. The method according to claim 8, characterized in that, The maximum Hausdorff distance between the optimal reactive manifolds within the sliding window is: in, For two manifold point sets and The Hausdorff distance between them is calculated using a mixed distance function. As a measure of distance between points Represents the vehicle's decision parameters and surrounding traffic environment parameters The input points are composed of For the first The optimal reaction manifold estimated during round iteration, For the first The optimal reaction manifold estimated during round iteration, manifold A sample point on the surface. This represents the mixing distance between two sample points.

Images