Behavior prediction model construction method and behavior prediction method for manual control stage of automatic driving vehicle

Abstract

The invention provides a behavior prediction model construction method and a behavior prediction method in a manual control stage of an automatic driving vehicle, and relates to the technical field of intelligent automobile and man-machine cooperative driving, and the method comprises the steps: converting multi-dimensional structured data into a sequence text which can be analyzed by a large language model based on a semantic template, and inputting the sequence text; taking the converted semantic sequence as input, constructing a multi-modal large model guided generative manual control behavior prediction model, and outputting a manual control behavior sequence; and taking a multi-modal large model guided generative manual control behavior prediction model as a teacher model, and performing knowledge distillation on the teacher model to obtain a manual control stage behavior prediction model based on cross-scene knowledge distillation migration. According to the method, the manual control behavior prediction model guided by the multi-modal large model is constructed, and the construction of the manual control behavior prediction model adaptive to different driving scenes is completed by means of a cross-scene knowledge distillation migration technology.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicles and human-machine collaborative driving technology, specifically to a method for constructing a behavior prediction model and a behavior prediction method for the manual control phase of autonomous vehicles. Background Technology

[0002] With the continuous development of autonomous driving technology, vehicles still struggle to operate fully autonomously in some complex traffic scenarios, and manual intervention will remain a long-term concern for the foreseeable future. When an autonomous driving system exceeds its designed operating range or its perception and decision-making capabilities are limited, how to reasonably trigger a takeover request and ensure the safety and effectiveness of the driver's control actions after takeover is a key issue affecting the reliability of human-machine cooperative driving systems.

[0003] Existing research largely focuses on the takeover request triggering mechanism or the driver's situational awareness recovery process, paying insufficient attention to the behavioral characteristics and evolutionary patterns of the manual control phase itself. In actual takeover processes, driver control behavior is influenced by multiple factors, including the complexity of traffic scenarios, the evolution of risks, and individual driving differences. Its operational range, response timing, and persistence all exhibit significant dynamic changes. Treating manual control behavior merely as an instantaneous event or a simple control input makes it difficult to accurately characterize its dynamic relationship with scenario changes, and also fails to effectively assess the driver's controllability and risk response capabilities after takeover.

[0004] In recent years, with the application and development of data-driven methods and large-scale model technology in the field of autonomous driving, human control behavior prediction models based on multi-source information fusion have demonstrated high accuracy and expressive power in complex scenarios, providing a new technical path for understanding the human-machine interaction process. However, such models typically have characteristics such as large parameter scale, high computational overhead, and strong dependence on scene distribution, making them difficult to deploy directly on in-vehicle systems with limited computing power and strict real-time requirements, thus restricting their application and promotion in actual human-machine cooperative driving systems. At the same time, existing human control behavior prediction models are mostly built for specific scenarios or single tasks, lacking a systematic abstraction and transfer mechanism for common knowledge between different traffic scenarios. This leads to performance degradation when the models are applied across scenarios, making it difficult to meet the requirements of the diversity and variability of scenarios in real traffic environments.

[0005] Therefore, this invention aims to construct a complete technical system from vehicle-scene dynamic interaction analysis to cross-scene manual control behavior prediction, in order to solve the aforementioned bottleneck problems. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method for constructing a behavior prediction model for the manual control phase of autonomous vehicles, as well as a behavior prediction method. By constructing a manual control behavior prediction model guided by a multimodal large model, and leveraging cross-scenario knowledge distillation and transfer technology, a manual control behavior prediction model adapted to different driving scenarios is constructed.

[0007] On one hand, the present invention provides a method for constructing a behavior prediction model for an autonomous vehicle during the manual control phase, comprising: Based on semantic templates, multi-dimensional structured data is transformed into sequence text input that can be parsed by a large language model. The multi-dimensional structured data includes takeover request type, driver attributes, expected takeover request timing, expected timing of manual intervention, and driving risk location. Using the transformed semantic sequence as input, a multimodal large model-guided generative human control behavior prediction model is constructed to output a human control behavior sequence. The model includes: a semantic encoding layer, used to extract deep semantic features of the input sequence; a scene adaptation layer, used to enhance the feature representation of core scene elements based on an attention mechanism; and a behavior generation layer, used to output a human control behavior sequence that conforms to driving logic and associate it with scene disturbance influence mechanism through a gating mechanism. The generative artificial control behavior prediction model is trained using a dual-objective supervised training strategy of maximum likelihood estimation and diversity constraints, and the weights of the model loss function are dynamically adjusted. The system analyzes scene features in the input semantic sequence in real time, generates a dynamic weight matrix based on a differentiated scene adaptation network, and adjusts the prediction strategy for human control behavior in real time according to the dynamic weight matrix.

[0008] Furthermore, it also includes: constructing a behavior prediction model for the artificial control stage based on cross-scenario knowledge distillation and transfer; the behavior prediction model for the artificial control stage based on cross-scenario knowledge distillation and transfer is a student model, and the student model is obtained by performing knowledge distillation on the generative behavior prediction model for artificial control guided by the multimodal large model.

[0009] Furthermore, the training data for the generative artificial control behavior prediction model includes: Differentiated relationships are established based on the dynamic changes in the scene, and the manual control behavior is dynamically adjusted based on the relationships; the dynamically adjusted manual control behavior data is used as training data.

[0010] Furthermore, differentiated relationships are established based on the dynamic characteristics of the scene, including: Construct a comprehensive appropriateness score system for human control behaviors that includes multiple evaluation dimensions; The fitness of human control behavior to dynamic scenarios is quantitatively analyzed based on the scoring system. Based on the dynamic changes in the scenario and the results of quantitative analysis, differentiated relationships are established.

[0011] Furthermore, differentiated relationships are established based on the dynamic characteristics of the scenario, and the manual control behavior is dynamically adjusted based on these relationships, including: Based on the quantitative analysis results, a dual-objective incremental model fine-tuning strategy is adopted to optimize the human control behavior adaptability prediction model, thereby obtaining the human control behavior adaptability prediction result. The human control behavior adaptability prediction model adopts a deep learning network architecture. The input layer receives feature vectors formed by scene features, risk level, and driver state data, and the output layer outputs a comprehensive adaptability metric of human control behavior and scene. Based on the prediction results of the adaptability of the human control behavior, a meta-task distribution containing multiple scenario conditions is constructed. Each meta-task takes the scenario feature vector and the driver state vector as input and the optimal solution of the adaptability of the human control behavior as output. Meta-training is performed using a model-independent meta-learning framework to form a meta-knowledge base that includes scene and behavior association rules, operation parameter initialization benchmarks, and driver state adaptation strategies. When encountering a new scenario, the core features of the scenario extracted by the traffic environment adaptation module are matched with the meta-knowledge base for similarity. After locating the most similar meta-task type, the initialization parameters of the meta-model are fine-tuned using real-time samples. The behavioral parameters are adjusted in combination with temporal correlation characteristics, and the driver state adaptation index is incorporated to achieve personalized adjustment.

[0012] Furthermore, the dual-objective incremental model fine-tuning strategy includes a dual objective of an adaptive loss term and an elastic weight consolidation constraint term; The elastic weight consolidation constraint term is designed based on the initial baseline parameters and Fisher information matrix. Parameters with high criticality are allowed to have a small range of change when fine-tuning, while parameters with low criticality are allowed to have a larger adjustment space to adapt to new scenarios. The fitness loss term is the mean square error between the predicted fitness result of the human control behavior output by the human control behavior fitness prediction model and the actual fuzzy quantization fitness.

[0013] Furthermore, a dynamic weight matrix is ​​generated based on a differentiated scenario adaptation network, and the prediction strategy for human control behavior is adjusted in real time according to the dynamic weight matrix, including: The relationship between the characteristic sequence of scene features and the time series of appropriateness of human control behavior was analyzed using an autoregressive moving average model. Significantly associated variables were screened and their influence weights were quantified. A differentiated scenario adaptation network is constructed, comprising a traffic environment adaptation module, a manual control behavior adaptation module, and a risk adaptation module. The traffic environment adaptation module filters core influencing factors of the scenario based on correlation analysis results and outputs predictions of environmental changes. The manual control behavior adaptation module dynamically adjusts operating parameters according to the suitability evaluation system and driver status indicators. The risk adaptation module formulates adaptation strategies based on real-time risk value classification combined with correlation patterns.

[0014] Furthermore, a behavior prediction model for the manual control stage based on cross-scenario knowledge distillation and transfer is constructed, including: We explore the dynamic correlation between human control behavior and the scene, model the dynamic evolution process of posterior knowledge through a variational Bayesian state-space model, and output the temporal distribution results of posterior knowledge. A lightweight student model is constructed based on the teacher model. A multi-level distillation architecture is used to achieve hierarchical knowledge transfer. The student model is fine-tuned by freezing some layers and combining cross-scenario data. We design a multi-task joint optimization objective function that integrates scene adaptation characteristics, and balance each training objective through dynamic weight allocation to obtain an end-to-end differentiable prediction model for human control behavior.

[0015] Furthermore, the multi-level distillation architecture includes: selecting key intermediate layers as distillation nodes according to the three-order structure of the teacher model, extracting activation tensors through intermediate layer attention alignment, and parsing features such as scene semantic representation, driver temporal dependence and dynamic risk level layer by layer to achieve knowledge layered transfer and alignment.

[0016] In another aspect, the present invention also provides a method for predicting the behavior of autonomous vehicles during the manual control phase, which uses the aforementioned behavior prediction model for the manual control phase to predict the behavior of autonomous vehicles during the manual control phase.

[0017] The beneficial effects of this invention are as follows: (1) This invention achieves rapid cross-scenario adaptation by mining the differentiated correlation characteristics between behavior and scenario. By combining multimodal large model and knowledge distillation technology, it generates artificial control behavior sequence that fits the driver's operation preferences, scenario dynamic changes and risk evolution laws, solves the problems of poor scenario adaptability and ignoring individual driving heterogeneity of traditional prediction models, and ensures the rationality and accuracy of behavior prediction.

[0018] (2) This invention constructs a lightweight student model through multi-level intermediate semantic feature distillation and frozen layer fine-tuning strategy; introduces scenario adversarial robustness enhancement strategy to simulate disturbance factors in actual driving, improve the model’s fault tolerance in complex and variable scenarios, and at the same time, multi-task joint optimization of objective function overcomes the problem of representation information decay, ensuring stable output of the model in different human-vehicle-environment coupling scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a method for constructing a behavior prediction model for an autonomous vehicle during the manual control phase, as described in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] The prediction of human takeover behavior suffers from a lack of standards and inaccuracies, making it difficult to adapt to complex and ever-changing traffic scenarios and failing to consider individual driver differences. This invention addresses this by establishing an evaluation system to clarify the standards for takeover behavior; dynamically adjusting adaptation rules for different scenarios and individual driver differences to achieve adaptation of takeover behavior to scenarios and drivers; and generating executable and practical behavior sequences based on a large language model to provide behavioral guidance for the human takeover process in autonomous driving.

[0024] like Figure 1 As shown in the figure, a method for constructing a behavior prediction model for an autonomous vehicle during the manual control phase, as described in this embodiment of the invention, specifically includes the following steps: (i) Constructing a large-scale multimodal model-guided prediction model for artificial control behavior; S1. Construct a multi-dimensional evaluation system for the appropriateness of artificial control behavior; S11. Construct a comprehensive appropriateness score system for artificial control behavior, which includes multiple evaluation dimensions; ; in, The overall appropriateness score for human-controlled behavior; Weights were assigned to each dimension, including scenario element suitability, risk level responsiveness, driver state suitability, intervention operation rationality, and result feedback effectiveness; the analytic hierarchy process (AHP) was used to determine these weights. ; , For dimension The number of secondary indicators, Secondary indicator The standardized value.

[0025] Construct the secondary indicators as shown in Table 1. Standardization of secondary indicators: [Using...] Standardization maps indicator values ​​to [0,1], and positive indicators are... Processing, negative indicators deal with.

[0026] Table 1

[0027] S12. Based on the constructed scoring system, a quantitative analysis of the adaptability of human control behavior to dynamic scenarios is conducted. The fit is categorized into five fuzzy evaluation levels: extremely unsuitable, somewhat unsuitable, moderately suitable, fairly suitable, and extremely suitable. For each secondary indicator, a corresponding membership function is designed, distinguishing between positive, negative, and interval indicators. Each secondary indicator is standardized, and the membership degree V of each indicator is fitted. A weighted average method is used to calculate the comprehensive membership degree, thus completing the mapping of the fit level between the current manual control behavior and the scenario. The triangular membership function is used, with values ​​of extremely ill-fitting (0-0.2), poorly ill-fitting (0.1-0.3), moderately fitting (0.2-0.4), well-fitting (0.3-0.5), and extremely fitting (0.4-1.0). The weighted average method is used to calculate the overall membership degree.

[0028] S13. Based on the quantitative analysis results, the adaptive prediction model of human control behavior is optimized by adopting a dual-objective incremental model fine-tuning strategy. With the dual objectives of improving the adaptability of human control behavior and maintaining the stability of model parameters, an incremental model fine-tuning strategy is designed: (1) Construction of the basic model for predicting the adaptability of human control behavior: A deep learning network architecture is adopted. The input layer receives feature vectors formed by data such as scene features, risk level, and driver status. The output layer outputs the comprehensive adaptability metric of human control behavior and scene. Preferably, a CNN-LSTM hybrid architecture (CNN 3 layers + LSTM 2 layers + fully connected 2 layers) is adopted, with an input dimension of 48 and an output dimension of 1, namely the adaptability score.

[0029] (2) Construction of Elastic Weight Constraints: For the initial baseline parameters of the model constructed in (1), calculate the influence of each parameter on the model performance and output the Fisher information matrix. The value of each position in the matrix corresponds to the criticality of a parameter. Based on the initial baseline parameters and the Fisher information matrix, design constraint rules. Parameters with high criticality are allowed a smaller range of change during fine-tuning, while parameters with low criticality are allowed a larger adjustment space to adapt to new scenarios. Specifically, the Fisher information matrix is ​​calculated based on the importance of 2000 sets of initial training data. Key parameters include convolutional kernel weights and LSTM hidden layer weights. Design constraint rules. Parameters with high criticality are allowed a smaller range of change during fine-tuning, while parameters with low criticality are allowed a larger range of change.

[0030] (3) Design of a dual-objective optimization function: A dual-objective optimization function is designed with the dual objectives of improving the adaptability of manual control behavior and maintaining the stability of model parameters. The function includes an adaptability loss term and an elastic weight consolidation constraint term. The adaptability loss term is the mean square error between the model prediction adaptability and the actual fuzzy quantization adaptability in (1). The fine-tuning objective of the dual-objective optimization function design is to continuously adjust the parameters to minimize the total loss function, thereby simultaneously reducing the prediction error of new scenarios and maintaining parameter stability.

[0031] S2. Establish differentiated relationships based on the dynamic changes in the scene, and dynamically adjust the manual control behavior based on the relationships; S21. Analysis of the correlation between manual control behavior and scene characteristics; We used the Autoregressive Integrated Moving Average (ARIMA) model to explore the changing patterns and correlation strengths of human control behaviors under different scenario characteristics.

[0032] (1) Construction and processing of core time series variables: Construct a time series feature sequence of the suitability of artificial control behavior, which includes the total suitability score, the standardized values ​​of each secondary index, the quantized values ​​corresponding to the fit level after fuzzy quantization, and the residual sequence of the predicted fit and actual fit of the bi-objective fine-tuning model. , Fine-tuning the prediction residuals of the bi-objective model; constructing a model that includes the number of dynamic interference sources. Lane deviation coefficient Following distance of the vehicle in front Target safe speed Scene characteristic feature sequence ;right and All time series are subjected to the ADF stationarity test. If the series is non-stationary, it is converted into a stationary series through d-order differencing. Preferably, d can be of order 1.

[0033] (2) Construction of autoregressive moving average model: based on the stationary scenario time series As input, the overall appropriateness of the control behavior is a stationary sequence. For the output, construct an autoregressive moving average. Model: ; in, It is a constant. Let the order be the autoregressive order. The autoregressive coefficient for historical appropriateness. The moving average order is... The moving average coefficient of the error term. The number of scene characteristic variables. Let be the influence coefficient of the k-th scenario variable. This is the intermediate error term. The model residuals are used. In this embodiment, the AIC criterion is used to determine the autoregression order p=3, the moving average order q=2, and the difference order d=1.

[0034] (3) Differential correlation characteristic analysis: Parameters of the autoregressive moving average model under different fit levels Perform a t-test (α=0.05) and calculate the statistic. Significantly non-zero coefficients were selected to form a set of significant association variables for each group; standardized coefficients were then used. Quantify the weights of the impact of variables on appropriateness in different scenarios. Among them, These are the estimated values ​​of the coefficients. For standard error, The standard deviation is denoted as .

[0035] S22, Scenario-Differentiated Response Architecture; Construct a three-element scenario-differentiated response architecture that includes a traffic environment adaptation module, a manual control behavior adaptation module, and a risk adaptation module.

[0036] The traffic environment adaptation module integrates dynamic and static scene indicators, uses ARIMA correlation analysis to screen core influencing factors for each scene, completes scene grouping and mapping, and outputs the priority of key elements and predictions of environmental changes. The traffic environment adaptation module is built using Python's scikit-learn library to implement K-means clustering (k=5, corresponding to 5 adaptation levels) to complete scene grouping and mapping; it uses an LSTM model (128 hidden units, 80 training epochs) to predict scene characteristic sequences and outputs predictions of environmental changes in the next 2 seconds.

[0037] The manual control behavior adaptation module uses the appropriateness evaluation system as a benchmark and dynamically adjusts the operation parameters by combining ARIMA temporal correlation features. It also incorporates the driver state adaptation index, increases the operation amplitude threshold for aggressive drivers, and reduces the operation response delay for conservative drivers, so as to achieve personalized behavior optimization based on scenario characteristics and individual differences.

[0038] The risk adaptation module is based on real-time risk value classification (three levels: low (0-0.3), medium (0.3-0.7), and high (0.7-1)) and formulates adaptation strategies in combination with the reverse correlation pattern of ARIMA.

[0039] S23. Adjustment of behavior through manual control; (1) Constructing a meta-task distribution: Different operating conditions within each scenario group (such as different interference source densities, road curvature, and risk levels) are defined as independent meta-tasks. The input of each meta-task is a scenario feature vector (number of dynamic interference sources, lane deviation coefficient, etc.) and a driver state vector (driving experience, situational awareness duration, etc.). The output is the optimal solution for the appropriateness of manual control behavior. Meta-training is performed using the Model Independent Meta-Learning (MAML) framework to form a meta-knowledge base of scenario-behavior association rules, operation parameter initialization benchmarks, and driver state adaptation strategies. In this embodiment, the scenarios are divided into two categories: highways and urban roads. Each category contains 10 operating conditions, totaling 20 meta-tasks. 500 samples are collected for each meta-task to construct a meta-dataset.

[0040] (2) MAML Meta-training: When encountering a new, unseen scenario, the traffic environment adaptation module based on the ternary architecture extracts the core features of the scenario in real time and performs similarity matching with the scenario features in the meta-knowledge base to locate the most similar meta-task type. Then, a rapid adaptation mechanism is activated: using a small number of real-time samples in the new scenario, gradient fine-tuning is performed based on the initialization parameters of the meta-model. Combined with the temporal correlation characteristics revealed by the ARIMA model, behavioral parameters are adjusted in a targeted manner—for scenarios with strong transient correlation, the operation response speed is enhanced; for scenarios with strong persistent correlation, the operation stability is optimized. At the same time, driver state adaptation indicators are integrated to achieve personalized adjustment: through the driver state-behavior adaptation rules stored in the meta-knowledge base, only key parameters are fine-tuned for experienced drivers to retain operational autonomy; for drivers with insufficient situational awareness, the range of operational parameter constraints is expanded based on meta-knowledge to ensure that the rapid adaptation process is both in line with the characteristics of the scenario and adapted to individual differences. In this embodiment, a two-layer fully connected layer is used as the meta-model. The input is a 64-dimensional feature vector, and the output is the optimal solution for suitability. The number of iterations is 1000 rounds, the batch size is 16, and the loss function is cross-entropy. When a new scene appears, similarity matching is performed: the cosine similarity algorithm is used to calculate the similarity between the feature vector of the new scene and the feature center in the meta-knowledge base to locate the most similar meta-task.

[0041] S3. Prediction of artificial control behavior based on large language models; Based on a multi-dimensional optimization strategy and a differentiated scenario-adaptive network collaboration method, this paper adopts the influence mechanism of scenario perturbation on the manual control process and iteratively optimizes to obtain a generative language model of manual control behavior that meets high scenario adaptability.

[0042] S31. Multimodal data transformation based on semantic templates; Select the takeover request type, driver attributes, expected takeover request timing, expected human intervention timing, and driving risk location as inputs. Develop a dedicated semantic template library. The template library designs structured semantic mapping rules for different scenario types, transforming multi-dimensional structured data into sequence text input that can be parsed by a large language model. At the same time, it embeds scenario perturbation feature labels to achieve standardized transformation of multimodal data into semantic sequences.

[0043] S32. Construction of a prediction model for artificially controlled behavior; Using the transformed semantic sequence as input, a generative predictive model for human control behavior is constructed. The model architecture adopts a three-tier structure of semantic encoding, scene adaptation, and behavior generation: The semantic encoding layer extracts deep semantic features of the input sequence through a Transformer encoder and integrates scene feature weights output by a differentiated scene adaptation network; in this embodiment, a 6-layer Transformer encoder with 8 heads, 256 hidden layer dimensions, and dropout=0.3 is used to extract semantic feature vectors. The scene adaptation layer introduces an attention mechanism, based on the scene feature weights output by the differentiated scene adaptation network; it embeds the previously mentioned scene grouping logic and strengthens the feature representation of core scene elements through the attention mechanism. The behavior generation layer outputs a human control behavior sequence that conforms to driving logic based on the decoding network, and associates it with the scene disturbance influence mechanism through a gating mechanism to ensure that the behavior generation is adapted to the dynamic changes of the scene; in this embodiment, a 4-layer Transformer decoder with 8 heads, 256 hidden layer dimensions, and outputs a human control behavior sequence of length 10; it embeds a gating loop unit to associate it with the scene disturbance influence mechanism.

[0044] S33. Supervised training of the model and optimization of prediction results; A dual-objective supervised training strategy combining maximum likelihood estimation and diversity constraints is adopted: using real sequences of manually controlled behaviors as labels, the deviation between the model's predicted distribution and the real distribution is minimized through maximum likelihood estimation; a diversity constraint mechanism is introduced to prevent predicted behaviors from falling into a single mode, improving adaptability to complex scenarios. During training, quantitative rules for the impact mechanism of scene perturbations are continuously incorporated, and multi-dimensional optimization strategies are combined to dynamically adjust the model's loss function weights, iteratively optimizing model parameters to ensure that the predicted behavior not only conforms to driving operation specifications but also accurately responds to dynamic changes in scene perturbations. In this embodiment, dual-objective supervised training is used, training with 100,000 semantic sequence-behavior sequence paired samples, of which 80,000 are used for training and 20,000 for validation; then, using real behavior sequences as labels, maximum likelihood estimation is performed to minimize the cross-entropy loss between the model's predicted distribution and the real distribution; a perplexity index is introduced, with a constraint value ≤50, to avoid predicting a single behavior; and the loss function weights are dynamically adjusted based on the intensity of scene perturbations.

[0045] S34. Generation of scene-differentiated weights and adjustment of manual control behavior; The model uses a built-in scene feature analysis module to analyze the scene type, disturbance intensity, and risk level in the input semantic sequence in real time, and generates a dynamic weight matrix based on a differentiated scene adaptation network. The weight matrix assigns differentiated influence weights to different scene elements and in turn affects the model generation process. The prediction strategy for human control behavior is adjusted in real time according to the weight matrix: when a high-weight scene element points to sudden disturbances, priority is given to optimizing parameters related to behavior response speed, specifically optimizing the first 3 steps of the behavior sequence; when it points to static constraints, the focus is on adjusting the operation accuracy and stability parameters, optimizing the middle 5 steps of the behavior sequence, ultimately achieving accurate prediction and real-time adaptation of human control behavior in dynamic scenes.

[0046] (ii) Constructing a behavior prediction model for the artificial control stage based on cross-scenario knowledge distillation and transfer; By introducing knowledge distillation technology, the high-precision large model constructed by S1 is used as the teacher model to supervise and constrain the lightweight student model. This significantly reduces the model complexity while preserving its ability to represent human control behavior patterns and interaction mechanisms to the greatest extent. Furthermore, by combining cross-scenario transfer strategies, the model's adaptability to different road structures and traffic conditions is enhanced, thereby achieving the model's cross-scenario transfer capability.

[0047] S21. Dynamic Association Mining and Soft Label Optimization of Human Control Behavior-Scene: Mining the dynamic association characteristics between typical human control behaviors and specific scenarios, and using soft label optimization technology to enhance the rationality of the distribution expression and prediction of human control behaviors.

[0048] (1) Dynamic correlation mining and causal analysis between behavior and scenario: Analyze the changes in human-controlled behavior under different scenarios, including the threshold range of operation amplitude, the temporal characteristics of response speed, and the dynamic changes in operation persistence; mine the multi-dimensional dynamic correlation features between behavior and scenario, including the time lag correlation and spatial correlation between scenario feature changes and behavior response, the magnitude of scenario feature changes and the magnitude of behavior adjustment, and clarify the behavior patterns corresponding to different combinations of scenario elements. In this embodiment, the sliding window method is used to extract temporal features such as operation amplitude threshold, response speed, and operation persistence, and the mutual information method is used to calculate the correlation strength between scenario features and behavior features.

[0049] (2) Temporal Causal Relationship Analysis and Evolutionary Knowledge Modeling: Using the temporal sequence of scene features as the candidate set of dependent variables and the temporal sequence of human control behavior as the candidate set of effect variables, statistically significant causal pairs are selected through significance testing, and the causal strength value and optimal lag order are calculated. Combining the human-vehicle-environment coupling characteristics, a variational Bayesian state-space model is used to model the dynamic evolution process of posterior knowledge: posterior knowledge is decomposed into three categories: scene-behavior adaptation rules, driver operation preferences, and risk response strategies. The coupling characteristics are used as the state input variables of the model, and behavior-scene adaptation degree and causal consistency are used as observation variables. The model parameters are iteratively optimized through variational inference algorithm to capture the evolution law of posterior knowledge with the driving process and output the temporal distribution results of posterior knowledge. In this embodiment, Granger causality test is used to screen significant causal pairs and calculate the optimal lag order; a variational Bayesian state-space model is constructed: the state variables are scene-behavior adaptation rules, driver operation preferences, and risk coping strategies; the observation variables are behavior-scene adaptation degree and causal consistency; the variational inference is iterated for 500 rounds with a learning rate of 1e-3 and a model residual of ≤0.05, and the temporal distribution results of the posterior knowledge are output.

[0050] (3) Knowledge distillation for teacher-student model construction: Using the artificial control behavior prediction model constructed in (I) as the teacher model, a lightweight student model is constructed using the knowledge distillation method. The number of encoder layers in the student model is reduced from 6 layers to 4 layers, and the number of decoder layers is reduced from 4 layers to 2 layers, with a bottleneck layer dimension of 256. Low-rank decomposition technology is used to compress the parameters of the fully connected layers, reducing the number of model parameters to less than 1 / 3 of that of the teacher model, while ensuring the effective transmission of core features. The teacher model outputs a probability distribution of behavior, which is defined as a soft label. This soft label contains the prediction probability of a single behavior, integrates scene-differentiated weights, temporal correlation of behavior, posterior knowledge confidence, and other multi-dimensional information.

[0051] (4) Temperature-driven soft tag optimization: Introducing temperature parameters The soft labels were optimized and adjusted. The optimal temperature parameters were selected through grid search and cross-validation. .

[0052] Temperature-driven logic is used to reconstruct the distribution of the original soft labels: the original logits output by the teacher model are calculated using temperature-adjusted Softmax, the probability distribution is flexibly adjusted, the weight of extreme probability values ​​is reduced, and the probability gradient between similar behavior categories is amplified.

[0053] Soft tag reconstruction:

[0054] in, For the original logits, To optimize the soft label probability.

[0055] S22, Semantic Feature Distillation Architecture and Cross-Scenario Knowledge Transfer: (1) Multi-level distillation architecture: A multi-level distillation architecture is constructed, consisting of bottom-level features, middle-level associations, and high-level decisions. Key intermediate layers are selected as distillation nodes according to the three-order structure of the teacher model. Activation tensors are extracted through attention alignment in the intermediate layers. Features such as scene semantic representation, driver temporal dependence, and dynamic risk level are analyzed layer by layer to achieve hierarchical knowledge transfer and alignment. Specifically, the third layer of the semantic encoding layer, the attention output of the scene adaptation layer, and the second layer of the behavior generation layer of the teacher model are selected as distillation nodes. The difference in activation tensors between corresponding nodes in the student model and the teacher model is calculated using the MSE loss function. .

[0056] (2) Freezing Partial Layer Fine-tuning: A partial layer fine-tuning strategy is adopted, freezing the core layer carrying cross-scene general knowledge, unfreezing the flexible layers related to scene adaptation, and fine-tuning with a small amount of cross-scene data. By optimizing the soft label constraints and the coordination of intermediate layer feature space alignment, semantic drift and catastrophic forgetting are avoided to achieve cross-scene knowledge transfer, thereby enhancing model adaptability and semantic parsing stability. Specifically, the frozen layer is selected as the first two layers of the Transformer encoder of the student model, and the unfrozen layer is selected as the last two layers of the encoder, the scene adaptation layer, and the entire decoder layer. The cross-scene dataset is selected for training for 30 rounds with a learning rate of 1e-5 and a loss function of distillation loss + feature alignment loss, denoted as loss function. .

[0057] (3) Adversarial Example Injection: A scenario-based adversarial robustness enhancement strategy is introduced, injecting features, parameters, and perceptual noise tailored to the driving scenario to generate adversarial examples, constructing a joint loss function of basic distillation loss + adversarial loss. The noise intensity and sample ratio are dynamically adjusted to enable the model to learn the scenario disturbance tolerance mechanism, improving the robustness and fault tolerance of predicting human control behavior in complex and variable scenarios. Specifically, based on the Fast Gradient Sign Method (FGSM algorithm), driving scenario-specific noise is injected; the loss function is denoted as... .

[0058] S23. Construction of a prediction model for human control behavior: Design a strategy to enhance the robustness of takeover scenarios and construct a prediction model for human control behavior based on end-to-end differentiability.

[0059] Define the scenario complexity vector , Based on driving risk level, The intensity of scene disturbance. Calculate the L2 norm for driver fit. Obtain the posterior knowledge confidence P of the variational Bayesian state-space model output, and set the feature alignment weight balancing factor. ,according to , , Generate dynamic weights.

[0060] Design a multi-task joint optimization objective function that incorporates scene adaptation features, and balance the various training objectives through dynamic weight allocation: ; Based on scenario complexity vector The posterior knowledge confidence P is dynamically generated, specifically as follows:

[0061]

[0062]

[0063] in, It is composed of driving risk level, scene disturbance intensity, and driver suitability. Let P be the scene complexity norm, and P be the posterior knowledge confidence. For adjustment coefficients, This is the feature alignment weight balancing factor.

[0064] in, To calculate the difference in behavioral probability distributions between the student and teacher models for the distillation loss of the soft label after temperature optimization, KL divergence is used to inherit higher-order prediction knowledge. For the multi-level intermediate feature alignment loss, the mean of the L2 norm after attention alignment of each distillation node is taken to ensure the hierarchical transmission of features such as scene semantics and temporal dependencies. To address the scenario-based adversarial loss, the prediction error is calculated based on adversarial examples injected with driving scenario-specific noise, thereby enhancing the model's fault tolerance.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a behavior prediction model for an autonomous vehicle during the manual control phase, characterized in that, include: Based on semantic templates, multi-dimensional structured data is transformed into sequence text input that can be parsed by a large language model. The multi-dimensional structured data includes takeover request type, driver attributes, expected takeover request timing, expected timing of manual intervention, and driving risk location. Using the transformed semantic sequence as input, a multimodal large model-guided generative human control behavior prediction model is constructed to output a human control behavior sequence. The model includes: a semantic encoding layer, used to extract deep semantic features of the input sequence; a scene adaptation layer, used to enhance the feature representation of core scene elements based on an attention mechanism; and a behavior generation layer, used to output a human control behavior sequence that conforms to driving logic and associate it with scene disturbance influence mechanism through a gating mechanism. The generative artificial control behavior prediction model is trained using a dual-objective supervised training strategy of maximum likelihood estimation and diversity constraints, and the weights of the model loss function are dynamically adjusted. The system analyzes scene features in the input semantic sequence in real time, generates a dynamic weight matrix based on a differentiated scene adaptation network, and adjusts the prediction strategy for human control behavior in real time according to the dynamic weight matrix.

2. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 1, characterized in that, Also includes: A behavior prediction model for the artificial control stage based on cross-scenario knowledge distillation and transfer is constructed. The behavior prediction model for the artificial control stage based on cross-scenario knowledge distillation and transfer is a student model. The student model is obtained by performing knowledge distillation on the generative behavior prediction model for artificial control guided by the multimodal large model.

3. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 2, characterized in that, The training data for the generative artificial control behavior prediction model includes: Differentiated relationships are established based on the dynamic changes in the scene, and the manual control behavior is dynamically adjusted based on the relationships; the dynamically adjusted manual control behavior data is used as training data.

4. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 3, characterized in that, Establish differentiated relationships based on the dynamic characteristics of the scene, including: Construct a comprehensive appropriateness score system for human control behaviors that includes multiple evaluation dimensions; The fitness of human control behavior to dynamic scenarios is quantitatively analyzed based on the scoring system. Based on the dynamic changes in the scenario and the results of quantitative analysis, differentiated relationships are established.

5. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 4, characterized in that, Establish differentiated relationships based on the dynamic characteristics of the scene, and dynamically adjust manual control behavior based on these relationships, including: Based on the quantitative analysis results, a dual-objective incremental model fine-tuning strategy is adopted to optimize the human control behavior adaptability prediction model, thereby obtaining the human control behavior adaptability prediction result. The human control behavior adaptability prediction model adopts a deep learning network architecture. The input layer receives feature vectors formed by scene features, risk level, and driver state data, and the output layer outputs a comprehensive adaptability metric of human control behavior and scene. Based on the prediction results of the adaptability of the human control behavior, a meta-task distribution containing multiple scenario conditions is constructed. Each meta-task takes the scenario feature vector and the driver state vector as input and the optimal solution of the adaptability of the human control behavior as output. Meta-training is performed using a model-independent meta-learning framework to form a meta-knowledge base that includes scene and behavior association rules, operation parameter initialization benchmarks, and driver state adaptation strategies. When encountering a new scenario, the core features of the scenario extracted by the traffic environment adaptation module are matched with the meta-knowledge base for similarity. After locating the most similar meta-task type, the initialization parameters of the meta-model are fine-tuned using real-time samples. The behavioral parameters are adjusted in combination with temporal correlation characteristics, and the driver state adaptation index is incorporated to achieve personalized adjustment.

6. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 5, characterized in that, The dual-objective incremental model fine-tuning strategy includes two objectives: an adaptive loss term and an elastic weight consolidation constraint term. The elastic weight consolidation constraint term is designed based on the initial baseline parameters and Fisher information matrix. Parameters with high criticality are allowed to have a small range of change when fine-tuning, while parameters with low criticality are allowed to have a larger adjustment space to adapt to new scenarios. The fitness loss term is the mean square error between the predicted fitness result of the human control behavior output by the human control behavior fitness prediction model and the actual fuzzy quantization fitness.

7. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 2, characterized in that, A dynamic weight matrix is ​​generated based on a differentiated scenario adaptation network, and the prediction strategy for human control behavior is adjusted in real time according to the dynamic weight matrix, including: The relationship between the characteristic sequence of scene features and the time series of appropriateness of human control behavior was analyzed using an autoregressive moving average model. Significantly associated variables were screened and their influence weights were quantified. A differentiated scenario adaptation network is constructed, comprising a traffic environment adaptation module, a manual control behavior adaptation module, and a risk adaptation module. The traffic environment adaptation module filters core influencing factors of the scenario based on correlation analysis results and outputs predictions of environmental changes. The manual control behavior adaptation module dynamically adjusts operating parameters according to the suitability evaluation system and driver status indicators. The risk adaptation module formulates adaptation strategies based on real-time risk value classification combined with correlation patterns.

8. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 2, characterized in that, Construct a behavior prediction model for the manual control stage based on cross-scenario knowledge distillation and transfer, including: We explore the dynamic correlation between human control behavior and the scene, model the dynamic evolution process of posterior knowledge through a variational Bayesian state-space model, and output the temporal distribution results of posterior knowledge. A lightweight student model is constructed based on the teacher model. A multi-level distillation architecture is used to achieve hierarchical knowledge transfer. The student model is fine-tuned by freezing some layers and combining cross-scenario data. We design a multi-task joint optimization objective function that integrates scene adaptation characteristics, and balance each training objective through dynamic weight allocation to obtain an end-to-end differentiable prediction model for human control behavior.

9. The method for constructing a behavior prediction model for the manual control phase of an autonomous vehicle according to claim 8, characterized in that, The multi-level distillation architecture includes: selecting key intermediate layers as distillation nodes according to the three-order structure of the teacher model; extracting activation tensors through attention alignment of intermediate layers; and parsing features such as scene semantic representation, driver temporal dependence, and dynamic risk level layer by layer to achieve knowledge layered transfer and alignment.

10. A method for predicting the behavior of an autonomous vehicle during the manual control phase, characterized in that, The behavior prediction model for the manual control phase as described in any one of claims 2 to 9 is used to predict the behavior of autonomous vehicles during the manual control phase.

Images