Method and device for constructing dynamic adjustment strategy of driving style of autonomous vehicle, and medium

Abstract

The present application relates to a kind of automatic driving vehicle driving style dynamic adjustment strategy construction method, equipment and medium, the method will traditional fixed rule-based automatic driving style adjustment process change into the self-adaptive adjustment process based on driver internal cognitive state and takeover behavior information, by semantic alignment's latent dynamics modeling and model reinforcement learning strategy optimization framework, realize the dynamic matching between automatic driving behavior and driver trust and cognitive workload.Compared with prior art, the present application realizes the dynamic matching between automatic driving behavior and driver trust and cognitive workload, effectively improves the safety and collaborative efficiency of man-machine co-driving.

Description

Technical Field

[0001] This invention relates to the field of human-machine collaborative control for autonomous driving, and in particular to a method, device and medium for constructing a dynamic adjustment strategy for the driving style of autonomous vehicles. Background Technology

[0002] With the development of autonomous driving technology, in Level 2 to Level 3 autonomous driving systems, vehicles can autonomously complete longitudinal and lateral control tasks under certain conditions, while the driver gradually transforms from a traditional vehicle operator into a supervisor and potential takeover operator of the autonomous driving system. In this human-machine co-driving mode, the driver needs to continuously monitor the operating status of the autonomous driving system and promptly take over when the system's capabilities are insufficient or environmental risks increase.

[0003] During the operation of an autonomous driving system, the system's driving behavior continuously influences the driver's perception of the system's capabilities. For example, when an autonomous driving system can stably complete driving tasks in complex traffic scenarios, the driver may gradually increase their trust in the system; conversely, when the system exhibits unstable behavior or fails to effectively handle traffic situations in certain scenarios, the driver may decrease their trust in the system and require more frequent manual intervention. Therefore, the operational behavior of an autonomous driving system not only determines the vehicle's motion state but also shapes the driver's level of trust through continuous human-machine interaction.

[0004] Driver trust levels have a significant impact on the safe operation of autonomous driving systems. Excessive distrust can lead to frequent vehicle takeovers, increasing unnecessary human-machine interaction conflicts and disrupting the system's normal operation. Conversely, excessive trust can reduce driver monitoring of the traffic environment, delaying takeovers when the system is inadequate or when risk events occur, thus increasing the risk of traffic accidents. Therefore, ensuring that driver trust levels align with the actual capabilities of the autonomous driving system in the current traffic situation is considered a crucial condition for achieving safe human-machine co-driving.

[0005] Besides trust factors, the driver's cognitive workload also significantly impacts the safety of human-machine collaborative driving systems. When a driver is under high cognitive workload, their attentional resources may be heavily consumed, reducing their ability to monitor the autonomous driving system's operation. Conversely, when a driver is under low cognitive workload for an extended period, decreased attention or alertness may occur, affecting their readiness to take over in emergency situations. Therefore, maintaining a moderate level of driver cognitive workload during autonomous driving operation is crucial for ensuring driver readiness to take over and the efficiency of human-machine collaboration.

[0006] However, in existing autonomous driving systems, vehicle driving behavior is typically determined by preset control strategies or fixed driving style parameters, with the system rarely considering the impact of driver state changes on the human-machine interaction process. When the driver's trust level or cognitive workload changes, the autonomous driving system often struggles to adaptively adjust its driving behavior based on the driver's current state, which can easily lead to a mismatch between driver trust and system capabilities, or result in the driver being under excessively high or low cognitive workload.

[0007] On the other hand, internal cognitive states such as driver trust level and cognitive workload are difficult to measure directly through sensing devices, and can usually only be indirectly inferred from driver behavior characteristics, physiological signals, or subjective evaluation data. These observational information often contain uncertainty and noise, making it difficult for the system to obtain a stable and decision-making-ready representation of the driver's state. Furthermore, in different traffic scenarios, the reasonable range of driver trust level and workload is closely related to the complexity of the traffic environment and the processing capacity of the autonomous driving system in the current scenario. However, existing technologies lack a human-machine co-driving adjustment method that can simultaneously consider driver state, traffic scenario requirements, and the capabilities of the autonomous driving system. A search revealed Chinese invention patent application CN120534387A, which discloses an autonomous vehicle control method based on driving style and dynamic trust assessment. This method introduces a driver trust assessment mechanism based on partially observable Markov decision mechanism (POMDP) ​​and uses driving style for personalized adjustment. However, it does not consider the dynamics of the driver's cognitive workload and lacks a mechanism to couple trust assessment with underlying motion control to actively maintain an appropriate workload. Therefore, it still struggles to systematically solve the problem of matching trust level with system capabilities while ensuring that the driver's cognitive workload remains within an appropriate range under partially observable conditions.

[0008] Therefore, under the condition that the driver's internal state can only be partially observed, how to dynamically adjust the driving behavior of the autonomous driving system, so as to keep the driver's trust level in reasonable match with the system's capabilities and scenario requirements, and at the same time keep the driver's cognitive workload within an appropriate range, thereby further improving the safety and collaborative efficiency of human-machine co-driving, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art by providing a method, device, and medium for constructing a dynamic adjustment strategy for the driving style of autonomous vehicles. This method is used to dynamically adjust the driving behavior of the autonomous driving system during autonomous driving operation based on the driver's internal state and traffic scenario requirements, so that the driver's trust level is consistent with the processing capacity of the autonomous driving system in the current scenario, while keeping the driver's cognitive workload within a reasonable range, thereby improving the safety and collaborative efficiency of the autonomous driving human-machine co-driving system.

[0010] The objective of this invention can be achieved through the following technical solutions: According to a first aspect of the present invention, a method for constructing a dynamic adjustment strategy for the driving style of an autonomous vehicle is provided, comprising: S1. Collect multi-source observation data during the operation of autonomous vehicles, and construct a partially observable Markov decision model for human-machine co-driving based on the multi-source observation data. The state variables in the state space include the driver's trust level, the driver's cognitive workload, and the scene processing capability of the autonomous driving system. S2. Based on the multi-source observation data, train a potential dynamic model, and by performing time-series modeling on the observation sequence and behavior sequence, obtain the potential state representation of the human-machine co-driving system in the potential space; S3. Construct a semantic decoding network on the latent state representation, and use a weak supervision method based on semantic anchors to semantically align the latent state representation so that the latent state representation corresponds to the driver's trust level, the driver's cognitive workload, and the scene processing capability of the autonomous driving system, respectively. S4. Based on the semantically aligned latent states, construct a reinforcement learning decision model, generate interaction trajectories through the latent dynamics model, and optimize the policy in the semantic state space to obtain a dynamic adjustment policy for the driving behavior of autonomous vehicles.

[0011] Preferably, the multi-source observation data specifically includes: Vehicle traffic environment status data, including collision time between vehicles and potential conflict targets, traffic scene complexity indicators, and vehicle operating status information; Driver behavior data, including driver takeover behavior indicator variables and the distance between the driver's foot and the pedals; and, Driver physiological signal data, including multi-channel EEG signal characteristics.

[0012] Preferably, the construction of a partially observable Markov decision model for human-machine co-driving based on the multi-source observation data specifically includes: The state space contains state variables including the driver's trust level, the driver's cognitive workload, and the autonomous driving system's scene processing capabilities. The motion space, within which motion variables include the longitudinal acceleration adjustment of the vehicle and the lateral acceleration adjustment of the vehicle; The observation space includes collision indicator variables, driver takeover behavior indicator variables, counterfactual event outcome indicators, driver subjective cognitive workload questionnaire results, scenario difficulty indicators, EEG characteristics, collision time indicators between the vehicle and potential conflict targets, and the distance between the driver's foot and the pedal.

[0013] Preferably, the weakly supervised method based on semantic anchors constructs semantic anchors by combining event-level semantic information and time-step-level agent features to perform semantic supervision training on the latent state representation; Specifically, the semantic anchor points include: The semantic anchor of driver trust is determined by the relationship between driver takeover behavior indicator variables and counterfactual operating results of the autonomous driving system. Semantic anchors for driver cognitive workload were constructed using the results of a driver's subjective cognitive workload questionnaire and EEG characteristics; and... The semantic anchor of the autonomous driving system's task processing capability is constructed using scenario difficulty indicators and collision time indicators between the vehicle and potential conflict targets.

[0014] Preferably, a Top-K strategy is used to select high-confidence semantic anchors for supervised training of the latent state representation.

[0015] Preferably, the latent dynamics model specifically includes: obtaining an observation embedding representation by encoding multi-source observation data, updating the latent state through a recursive state update network, and using a latent state prediction network to predict the distribution of the latent state at the next moment based on the current latent state and the current action; wherein, the latent dynamics model introduces an observation reconstruction network, which is used to reconstruct the real observation at the next moment based on the predicted latent state at the next moment. The latent dynamics model is trained using a model ensemble approach, and the latent state estimate is obtained by combining the prediction results of multiple latent dynamics sub-models.

[0016] Preferably, the reward function of the reinforcement learning decision model includes a trust matching reward and a cognitive workload adjustment reward. The trust matching reward is used to penalize the difference between the driver's trust level and the scene processing capability of the autonomous driving system, and the cognitive workload adjustment reward is used to constrain the driver's cognitive workload to remain within a set range.

[0017] Preferably, the reinforcement learning decision model performs multi-step trajectory prediction in the latent state space and uses the generated interaction trajectory to train the policy offline.

[0018] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement any of the methods described above.

[0019] According to a third aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0020] Compared with the prior art, the present invention has the following advantages: (1) This invention constructs a partially observable Markov decision model that includes the driver's trust level, the driver's cognitive workload and the scene processing capability of the autonomous driving system. It integrates the driver's internal state and traffic scene requirements into the same decision framework, enabling the autonomous driving system to dynamically adjust its driving behavior according to the current scene complexity and system capability boundary. This reduces the phenomenon of excessive trust or distrust of the autonomous driving system by the driver, keeps the driver's trust level consistent with the autonomous driving system's capability, and improves the safety and stability of human-machine co-driving process.

[0021] (2) This invention constructs a latent dynamics model and introduces a semantic alignment mechanism to map unobservable variables such as driver trust level, cognitive workload and autonomous driving system scene processing capability into latent state representations with clear semantic meanings. It also uses semantic anchors to perform weakly supervised training on the latent state, thereby reducing the uncertainty of the latent state representation and enabling the system to stably estimate the driver's internal state under partially observable conditions, thus providing a reliable state information basis for the behavior adjustment of the autonomous driving system.

[0022] (3) This invention constructs a model reinforcement learning-based driving behavior regulation strategy in a semantically consistent latent state space. By jointly optimizing the driver's trust matching degree and cognitive workload level, the autonomous driving system can dynamically adjust its driving behavior according to the driver's real-time state and changes in traffic scenarios. In situations with high traffic risks or when the system's capabilities are close to their limits, the strategy can enhance the driver's perception of the system state and increase the driver's participation. In situations with low risks and when the system can stably handle driving tasks, it reduces unnecessary human-machine interaction interference, thereby maintaining a moderate cognitive workload for the driver while ensuring driving safety and improving the overall collaborative efficiency of the human-machine co-driving system.

[0023] (4) The present invention uses the Top-K strategy to select high-confidence semantic anchors to supervise the training of latent state representation, thereby forming a stable semantic correspondence between latent state and driver trust, cognitive workload and system capability.

[0024] (5) The latent dynamics model of the present invention is trained by model ensemble method, and a more stable and reliable latent state estimate is obtained by combining the prediction results of multiple latent dynamics sub-models. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention.

[0026] Figure 2 This is a schematic diagram of the model training process. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] Example This invention does not replace the underlying vehicle safety controller and the basic control algorithm for autonomous driving. Instead, it adjusts the driving behavior generated by the autonomous driving system at a higher level while meeting the underlying safety constraints, so that the driving style of the autonomous vehicle can be adaptively adjusted according to changes in the driver's state.

[0029] like Figure 1 As shown, this embodiment provides a method for constructing a dynamic adjustment strategy for the driving style of autonomous vehicles, including: S1. Collect multi-source observation data during the operation of autonomous vehicles, and construct a partially observable Markov decision model for human-machine co-driving based on the multi-source observation data. The state variables in the state space include the driver's trust level, the driver's cognitive workload, and the autonomous driving system's scene processing capability.

[0030] In this embodiment, the driving style adjustment task of an autonomous vehicle during operation is modeled as a partially observable Markov decision process (POMDP), which is represented as a seven-tuple. ,in, Representing the state space, Represents the action space, Represents the observation space. Represents the state transition function. Represents the observation generating function, Represents the reward function, This represents the discount factor.

[0031] In this embodiment, the state space This is used to characterize the real state of the human-machine co-driving system at the current moment. Its state variables include the driver's trust level, the driver's cognitive workload, and the autonomous driving system's scene processing capability in the current traffic situation. The state is represented as: , in, Indicates time The driver's level of trust is used to characterize the degree of subjective reliance of the driver on the capabilities of the autonomous driving system; Indicates time Driver cognitive workload is used to describe the level of cognitive resource consumption experienced by a driver while monitoring the operation of an automated driving system. These metrics represent the processing capabilities of the autonomous driving system in the current traffic scenario, characterizing its ability to safely complete driving tasks under the current conditions. It should be noted that driver trust level and cognitive workload are both internal cognitive states of the driver and cannot be directly measured by onboard sensors. Similarly, the scenario processing capabilities of the autonomous driving system are difficult to characterize directly through a single observation. Therefore, all of the above variables are modeled as potential states.

[0032] Since the driver's trust level and cognitive workload are internal states that cannot be directly observed, the true state of the system is probabilistically estimated through the belief state, and this belief state is used as the state representation for subsequent policy learning.

[0033] In this embodiment, the action space This describes the adjustment actions taken to regulate the driving behavior of an autonomous driving system. These actions act on the vehicle control commands generated by the autonomous driving system, performing upper-level behavioral adjustments without altering the basic control logic of the underlying vehicle safety controller. Specifically, at any given moment... The action is represented as: , in, Indicates time The vehicle longitudinal acceleration adjustment amount is used to adjust the driving behavior of autonomous vehicles during acceleration, deceleration, and following. Indicates time The vehicle's lateral acceleration adjustment is used to regulate the vehicle's driving behavior during steering, lane changing, and path keeping. By jointly adjusting longitudinal and lateral driving behavior parameters, the driving style of the autonomous driving system can be altered, thereby continuously influencing the driver's cognitive judgment of the system's capabilities and further affecting the dynamic evolution of the driver's trust level and cognitive workload.

[0034] In this embodiment, since the true state of the system cannot be directly observed, it is determined through observation space. Acquire observational information related to driver behavior and the traffic environment, such as Figure 2 As shown, time observation vector Represented as: , in, Indicates time The collision indicator variable is used to indicate whether a collision or safety failure has occurred during the current operation of the autonomous driving system; Indicates time The driver takeover behavior indicator variable is used to indicate whether the driver has taken over the autonomous driving system manually at the current moment; Indicates time The counterfactual event outcome index is used to characterize the possible operating results of the autonomous driving system if the driver does not take over in the current traffic scenario. This result is obtained by predicting the operating results of the autonomous driving system under non-intervention conditions through traffic simulation software. Indicates time Results of the driver's subjective perception of workload questionnaire; Indicates time The scenario difficulty index is used to characterize the overall difficulty level of the current traffic scenario in terms of traffic interaction complexity and potential conflict level; Indicates time The brainwave characteristics are used to reflect changes in the driver's real-time cognitive state. Indicates time The collision time metric between a vehicle and a potential conflict target is used to characterize the safety margin in the current traffic situation; Indicates time The distance between the driver's foot and the pedal is used to characterize the driver's readiness to take over.

[0035] Since the true state of the system cannot be directly obtained, this embodiment further introduces belief states to perform probabilistic estimation of the true state of the system. The belief state is used to represent the posterior distribution of the current state given a historical observation sequence and an action sequence, and it is expressed as: , in, for The state of belief at any moment Represents from the initial moment to the present moment. The observation sequence at time, Indicates from the initial time to The action sequence corresponding to each moment. By introducing a belief state, the current state of the system can be probabilistically estimated under the condition that the driver's trust level and cognitive workload are not directly observable. This belief state is then used as the state representation for subsequent potential dynamics modeling, semantic alignment, and reinforcement learning policy optimization. Since the driving behavior of an autonomous driving system not only changes the vehicle's motion state but also continuously affects the driver's trust level and cognitive workload through the driver's perception and cognitive processes, the driving behavior regulation problem exhibits obvious sequential decision-making characteristics, and there is a dynamic coupling relationship between the driver's internal state and autonomous driving behavior. Through the above modeling method, changes in the driver's internal state, autonomous driving behavior regulation, and changes in the traffic environment can be uniformly incorporated into the same decision-making framework, thus providing a unified modeling foundation for subsequent potential dynamics model training, semantic alignment, and model-based reinforcement learning policy learning.

[0036] S2. Based on multi-source observation data, a potential dynamic model is trained. By performing time-series modeling on the observation sequence and behavior sequence, the potential state representation of the human-machine co-driving system is obtained in the potential space.

[0037] After completing the modeling of the partially observable Markov decision process, it is necessary to estimate the true state of the system under the condition that the driver's internal state is not directly observable. To this end, this embodiment constructs a latent dynamics model. By performing time-series modeling on the observation and action sequences obtained during the operation of the autonomous driving system, the dynamic evolution law of the human-machine co-driving system state is learned in the latent space, thereby obtaining an approximate representation of the belief state.

[0038] In this embodiment, the latent dynamics model consists of an observation encoding network, a recursive state update network, a latent state prediction network, and an observation reconstruction network. The observation encoding network extracts features from the driver's behavior information, physiological signal characteristics, and traffic environment risk indicators in the observation vector and generates an observation embedding representation. The recursive state update network updates the latent state based on the current observation embedding representation, historical latent states, and historical driving behavior adjustment actions. The latent state prediction network predicts the next potential state based on the current potential state and current actions. The observation reconstruction network reconstructs or predicts the next observation based on the predicted next potential state, thereby learning the dynamic evolution relationship of the human-machine co-driving system state in the latent space.

[0039] Specifically, firstly, the defined observation vector Encoding is performed to map multi-source observation information to a latent feature space, thereby obtaining the observation embedding representation. The computation process is as follows: ,in, Represents the observation coding function, Indicates time The observation embedding vector.

[0040] The observation coding function is used to fuse driver behavior information, physiological signal characteristics, and traffic environment risk indicators to obtain an observation representation that reflects the current system state.

[0041] After obtaining the observation embedding representation, the observation embedding is fused with historical latent states and autonomous driving system behavior information. The latent states are then updated using a recursive state update function to obtain the system's state representation in the latent space. The latent state update process is represented as follows: , in, Indicates time Potential state variables, Represents the potential state variables of the previous time step. This indicates the driving behavior adjustment action at the previous moment. This represents a recursive state update function.

[0042] Through the aforementioned recursive update mechanism, the latent state can continuously integrate historical observation information and behavioral information over time, thereby forming a compact representation of the current state of the system.

[0043] In the latent space, the state evolution of the human-machine co-driving system is described by a latent state prediction network. Given the current latent state... and current action The latent state prediction network outputs the latent state prediction result for the next time step, and its calculation process is expressed as follows: , in, This represents the potential state for the next moment, predicted based on the current potential state and the current action. This represents the latent state prediction function. The latent state prediction function is used to characterize the dynamic coupling relationship between the driving behavior of the autonomous driving system, changes in driver state, and changes in the traffic environment within the latent space.

[0044] Furthermore, to ensure that the predicted potential state results reflect the actual system evolution, this embodiment introduces an observation reconstruction network, which, based on the predicted potential state for the next time step,... Actual observation of the next moment The process of reconstruction or prediction is represented as follows: , in, This represents the predicted observation at the next moment. This represents the observation reconstruction function. The observation reconstruction function is used to map the latent state prediction results back to the observation space, so that the training of the latent dynamic model is directly constrained by the real observation data.

[0045] During model training, by learning from historical interaction data, the latent dynamics model can accurately predict the system's evolution over time. Unlike directly constraining the prediction of latent states and encoding the next-time latent state, this embodiment optimizes model parameters by minimizing the difference between predicted observations and actual next-time observations, thus directly addressing the dynamic evolution learning of latent states with real data. Accordingly, the dynamic loss function is expressed as: , in, Indicates that, given a predicted potential state Observation of the real next moment under the condition The conditional probability distribution is obtained. By optimizing the above loss function, the latent dynamics model can accurately characterize the dynamic changes in the state of the human-machine co-driving system in the latent space.

[0046] S3. Construct a semantic decoding network on the latent state representation, and use a weak supervision method based on semantic anchors to semantically align the latent state representation so that the latent state representation corresponds to the driver's trust level, the driver's cognitive workload, and the autonomous driving system's scene processing capability, respectively.

[0047] In S2, the latent state representation is obtained through the latent dynamics model. Next, it is necessary to address the problem of the lack of explicit semantic interpretation of latent states. Since it is difficult to obtain precise ground truth labels for each time step due to driver trust, cognitive workload, and the task processing capabilities of autonomous driving systems, this embodiment employs a weak supervision method based on semantic anchors to semantically align latent states. This method provides the overall direction of semantic variables through event-level information and combines time-step-level proxy features to characterize the degree of consistency between different time steps within a trial and this semantic direction, thereby constructing a stable semantic supervision signal in the absence of precise time labels.

[0048] In potential state Three semantic decoding heads are constructed, corresponding to driver trust, cognitive workload, and autonomous driving system task processing capabilities, respectively, including: First, the decoder outputs an unnormalized representation: , in, , , These are the corresponding semantic decoding headers; Then, probabilistic semantic predictions are obtained using the Sigmoid function: , in, , and These represent the probability estimates of the driver being in a state of high trust, a state of high cognitive workload, and the autonomous driving system having the ability to handle the current scenario, respectively.

[0049] To construct time-step-level semantic tags under weak supervision, this embodiment employs a unified semantic anchor construction framework. Let the semantic variable index be... First, the semantic direction is determined by event-level information. Secondly, a normalized consistency index is constructed from the time-step-level proxy features. This is used to characterize the degree of consistency between the time step and the semantic direction.

[0050] Based on this, the time step-level anchor value is defined as: This definition fuses event-level semantic direction with time-step-level agent features, thereby identifying time steps that are more consistent with the overall semantic trend within each trial.

[0051] For the semantic variable of driver trust, the event-level semantic direction is indicated by the counterfactual outcome indicator. Indicator variables of driver takeover behavior Jointly determined: This definition characterizes the consistency between driver behavior and the system's actual capabilities: if the system objectively cannot complete the task but the driver does not take over, it can be considered closer to a high trust tendency; conversely, if the system can safely complete the task but the driver still takes over, it is closer to a low trust tendency. The time-step agent uses the distance between the driver's foot and the pedals. And normalize within trials: Since a larger foot distance usually indicates a lower level of driver readiness to take over, it is closer to the semantics of high trust.

[0052] For semantic variables related to cognitive workload: the event-level semantic direction is determined by the results of the driver's subjective cognitive workload questionnaire. Given, represented as: The time-step agent is constructed from EEG features; specifically, it is computed within a sliding time window. , and Frequency band power, and define workload parameters: Then, normalization is performed within the trials: Neuroscience research indicates that larger This typically corresponds to a higher cognitive workload, and therefore aligns with the high-load semantic direction.

[0053] For semantic variables related to the task processing capability of autonomous driving systems: event-level semantic directions are considered jointly with counterfactual outcomes. With scene difficulty index Construct, represented as: This definition reflects the relative relationship between capability level and scenario complexity: if the system can safely complete the task in a high-difficulty scenario, its capability semantics are high; conversely, if the system cannot avoid collisions in a low-difficulty scenario, its capability semantics are low. The time-step agent uses a collision time metric and normalizes it within trials. Larger This indicates a greater safety margin, and therefore is closer to the semantic state in which the system has the capability to handle the current scenario.

[0054] In this embodiment, a scenario difficulty index is constructed based on multidimensional risk factors. This is used to quantitatively assess the difficulty of a test scenario, and the calculation expression is: , in, The baseline score is set to 0.2 in this embodiment; environmental factors It is a binary variable used to characterize the degree of perceptual limitation. Represents ideal environments such as sunny days. This indicates an environment where visibility / road conditions are obstructed (such as rainy days or nighttime); speed risk classification. A discrete hierarchical strategy is adopted, and the values ​​are selected. Defined based on the scene's baseline speed: Represents low speed (e.g., 40, 70 km / h), This represents a medium speed (e.g., 90, 100 km / h). Represents high speed or significant speeding (e.g., 120 km / h and above); interaction complexity A counter rule is used, whose value is equal to the number of additional difficulty conditions added to the scene. This represents standard conditions (such as pedestrians walking slowly and sufficient distance). This represents adding one more difficulty factor (such as oncoming traffic or shortened distance). This represents adding two more difficult terms (such as pedestrian running with oncoming traffic); critical correction terms. It is used to handle nonlinear risk growth under physical limits, and the calculation formula is: ,in The number of times a lethal factor is triggered, including: extreme speed. And operational difficulties (due to obstructed vision, excessive relative speed or very little space margin). , , The weighting coefficient for the corresponding item is set to 0.2 in this embodiment.

[0055] In this embodiment, the experimental scenario data settings are shown in Table 1 below.

[0056] Table 1

[0057] Since time-step proxy features may be affected by instantaneous noise or local disturbances, this embodiment further introduces a confidence screening mechanism based on local stability.

[0058] First, for each type of semantic variable, the anchor confidence score is defined as: ,in, Indicates the first Time-step proxy features corresponding to class semantic variables The confidence level represents the size of the local time window. This confidence level characterizes the stability of the proxy features within the local time window: if the feature changes are small near a certain time step, the behavior, physiological or environmental state reflected at that moment is more stable, and therefore more suitable as a semantic anchor.

[0059] Subsequently, in each trial, the previous step was selected based on the confidence level. Each time step constitutes an anchor point set: At the anchor time step, a binary cross-entropy supervision is applied to the semantic prediction to obtain the semantic alignment loss. : .

[0060] In addition, to ensure the continuous evolution of semantic variables over time, a time-smoothing regularization term is introduced. : .

[0061] Ultimately, semantic alignment loss and potential dynamics loss together constitute the overall training objective of the model, such as Figure 2 As shown: .

[0062] By jointly optimizing the above objective functions, the latent dynamics model can learn an interpretable latent state representation that is consistent with driver trust, cognitive workload, and the task processing capabilities of the autonomous driving system while maintaining its temporal prediction capabilities. This provides a semantically consistent state foundation for subsequent reward function design and policy learning.

[0063] S4. Based on the semantically aligned latent states, construct a reinforcement learning decision model, generate a trajectory with the same parameter distribution as the original experimental scenario through the latent dynamics model, and optimize the policy in the semantic state space to obtain a dynamic adjustment policy for the driving behavior of autonomous vehicles.

[0064] After completing the semantic alignment of the latent states, a state representation with clear semantic meaning can be obtained. Specifically, the latent states are then processed through a semantic decoding network. Mapped to semantic state vectors: ,in, This represents a probability estimate of the driver's level of trust. This represents a probability estimate of the driver's cognitive workload. This represents an estimate of the autonomous driving system's task processing capability in the current traffic scenario. All the semantic variables mentioned above are mapped to intervals. And this increases the semantic state space used for policy learning.

[0065] After obtaining the semantic state representation, a reinforcement learning decision model is constructed in this semantic state space to learn the driving behavior adjustment strategy of the autonomous driving system. To achieve a balance between driver trust correction and cognitive workload, this embodiment constructs a reward function based on semantic state. An ideal human-machine co-driving state should satisfy two basic principles: first, the driver's trust level should be consistent with the actual capabilities of the autonomous driving system in the current scenario to avoid over-trust or under-trust; second, the driver's cognitive workload should be maintained within a moderate range to ensure that the driver still has sufficient attentional resources when takeover is required.

[0066] Based on the above principles, a trust matching reward term is first constructed to constrain the consistency between the driver's trust level and the system's capability. Its expression is as follows: This reward penalizes the discrepancy between the driver's level of trust and the system's capabilities, thereby encouraging strategies to reduce trust mismatch.

[0067] Furthermore, to prevent drivers from being in a state of excessively high or low cognitive workload for extended periods, a workload adjustment reward is introduced. Let the desired cognitive workload level be... The cognitive workload adjustment reward item is represented as follows: This reward can help keep the driver's cognitive workload within a reasonable range, thereby preventing the driver from reducing their ability to monitor the autonomous driving system due to excessively high or low workload.

[0068] Combining the two parts of the reward, we obtain the overall reward function for reinforcement learning: , in, and These represent the weighting coefficients of the trust matching reward and the cognitive workload adjustment reward, respectively.

[0069] To avoid the policy from making overly optimistic estimates in regions where the predictions of the underlying dynamic model are unreliable, a penalty term based on model uncertainty is further introduced during the policy training process.

[0070] In this embodiment, the latent dynamics model is trained using a model ensemble approach: Let the first Potential dynamic sub-models The prediction of the potential state at the next moment is as follows: ,in, For the first Parameters of a potential dynamic sub-model This indicates the number of potential dynamic sub-models.

[0071] The mean of the prediction results of each sub-model is defined as: .

[0072] Calculate the current state-action pair The uncertainty of the model is expressed as: .

[0073] The aforementioned uncertainty characterizes the degree of divergence in the model ensemble's predictions of the potential state at the next moment. The more dispersed the predictions of each sub-model, the higher the risk of extrapolation error in the model near the current state.

[0074] Based on this, construct the reward function after penalty: ,in, This is the uncertainty penalty coefficient. By introducing this penalty term, we can suppress the policy from over-exploiting the model error region to generate high estimated returns, thereby improving the stability and reliability of the policy learning process.

[0075] After the reward function is defined, a model-based reinforcement learning policy is trained based on the latent dynamics model. Specifically, the latent dynamics model trained in S2 is used as an approximation model of the environment, and multi-step trajectory prediction is performed in the latent state space to generate synthetic interaction data. In each time step, the current semantic state is first determined... With policy function Sampling driving behavior adjustment actions Then, the potential state at the next moment is predicted using a potential dynamics model. And obtain the corresponding semantic state through the semantic decoder. Then, the immediate reward is calculated based on the above reward function. By repeating the above process, a length of [length missing] can be generated. The synthetic trajectories are used to construct training data for policy learning.

[0076] Furthermore, positional information reflecting the progress of traffic interaction is incorporated into the observation vector to characterize the relative stage of the vehicle in the current interaction process. For example, positional features can be constructed based on information such as the vehicle's relative position in the scene, distance from potential conflict areas, or trajectory normalization progress, and then input as part of the observation vector into the underlying dynamics model, enabling the model to perceive the evolutionary stage of the current interaction process.

[0077] In the policy optimization phase, this embodiment employs the maximum entropy reinforcement learning method (SAC) to optimize the policy. Specifically, a stochastic policy is learned by maximizing the weighted sum of the expected cumulative reward and the policy entropy, and its optimization objective is expressed as: ,in, Represents policy entropy. This represents the entropy weighting coefficient. By introducing the policy entropy term, the stability of the policy during the exploration phase can be improved, and the policy can be prevented from converging to a local optimum too early.

[0078] In this embodiment, the reinforcement learning decision model includes a policy network and a value evaluation network. The policy network outputs driving behavior adjustment actions based on the semantic state vector; the value evaluation network evaluates the long-term reward corresponding to the current state or state-action pair and guides the policy network update to obtain a driving behavior adjustment strategy for regulating the vehicle's longitudinal and lateral acceleration.

[0079] After completing the policy training, the learned driving behavior adjustment strategy is deployed in the upper-level behavior adjustment module of the autonomous driving system. During actual operation, the latent state representation is updated and the semantic state vector is calculated based on real-time observation information. Then, the policy network outputs driving behavior adjustment actions, thereby adjusting the longitudinal and lateral acceleration control commands generated by the autonomous driving system at the behavioral level. This allows the driving style of the autonomous vehicle to be dynamically adjusted according to the driver's trust level, cognitive workload, and traffic scenario requirements.

[0080] Through the above steps, dynamic matching between the driving behavior of autonomous vehicles and the driver's state can be achieved, thereby improving the safety and collaborative efficiency of autonomous human-machine co-driving systems.

[0081] In this embodiment, by introducing a latent dynamics model and semantic alignment mechanism within the framework of a partially observable Markov decision process, a unified model of the autonomous driving human-machine co-driving system is achieved. This allows for the joint description of the driver's internal state, the autonomous driving system's behavior, and the evolution of the traffic environment within the same state space. Unlike traditional autonomous driving control methods that rely solely on vehicle dynamics or traffic environment states for decision-making, this embodiment uses the driver's trust level and cognitive workload as important state variables of the human-machine co-driving system and estimates them through latent state modeling. This enables the autonomous driving system to explicitly consider the impact of changes in the driver's cognitive state on the human-machine collaboration process during decision-making. Within this modeling framework, the autonomous driving system's driving behavior not only affects the vehicle's motion state but also continuously influences the driver's trust level and cognitive workload through the driver's perception and understanding processes, further affecting the driver's takeover behavior indicator variable, thus forming a dynamic closed-loop interaction relationship of "driving behavior—cognitive state—behavioral response." By incorporating the aforementioned interaction processes into a partially observable decision model, changes in traffic environment risk, autonomous driving system behavior, and driver internal state can be described simultaneously in a unified state space. This enables the autonomous driving system to adjust its driving behavior in real time based on the driver's state in a dynamic traffic environment, thereby improving the collaborative efficiency of the human-machine co-driving system.

[0082] Furthermore, in this embodiment, the state evolution process of the human-machine co-driving system is learned through a latent dynamics model, thereby obtaining a compact representation of the driver's internal state and the system's capability state in the latent space. Unlike traditional methods that estimate driver state based on rules or single observation indicators, this embodiment recursively models historical observation sequences and behavior sequences, enabling the latent state to continuously integrate driver behavior information, physiological signal features, and traffic environment risk indicators, thus forming a temporal representation of the system state. To further improve the interpretability of the latent state representation, this embodiment introduces a weakly supervised semantic alignment mechanism based on semantic anchors. By combining event-level semantic information with time-step-level proxy features, it provides stable semantic supervision signals for the latent state even in the absence of precise labels. Through this mechanism, a stable correspondence can be established between the latent state and the driver's trust level, driver's cognitive workload, and the autonomous driving system's scene processing capability, thereby obtaining a state representation with clear semantic meaning while maintaining the temporal prediction capability of the latent dynamics model, providing an interpretable semantic state space for subsequent policy learning.

[0083] Building upon this foundation, this embodiment further utilizes a semantically consistent latent state space to construct a model-based reinforcement learning policy learning framework. Unlike traditional autonomous driving control methods that rely on human experience or fixed rules for driving style adjustment, this embodiment constructs a trust matching reward term and a cognitive workload adjustment reward term, enabling the reinforcement learning policy to optimize driving behavior while simultaneously achieving driver trust correction and cognitive workload balance. By using a latent dynamics model to perform multi-step predictions of the system state, synthetic interaction trajectories can be generated in the latent space, thereby improving policy learning efficiency while reducing real-world interaction. Through these methods, the autonomous driving system can dynamically adjust its driving behavior based on changes in traffic scenarios and driver cognitive states, ensuring that the driver's trust level remains consistent with the system's capabilities, while avoiding excessively high or low driver cognitive load, thus achieving more stable human-machine collaborative driving control.

[0084] Through the aforementioned modeling and policy learning mechanism, this embodiment transforms the autonomous driving style adjustment problem, which originally relied solely on fixed rules or empirical parameters, into a sequential decision-making problem optimized in the semantic state space. This enables the autonomous driving system to continuously perceive the driver's state and dynamically adjust its driving behavior throughout the entire driving process, rather than performing one-off control only when a specific event occurs. This method allows the autonomous driving system to increase driver involvement when traffic risks are high or the system's capabilities are nearing their limits, while reducing unnecessary human-machine intervention when the system can stably handle driving tasks. This, in turn, ensures safety while improving driver comfort and system coordination efficiency.

[0085] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0086] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0087] Multiple components in the device are connected to an I / O interface, including: input units such as a keyboard, mouse, etc.; output units such as various types of displays, speakers, etc.; storage units such as disks, optical disks, etc.; and communication units such as network interface cards, modems, wireless transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. The processing unit performs the various methods and processes described above, such as the method of the present invention. For example, in some embodiments, the method of the present invention may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method of the present invention described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the method of the present invention by any other suitable means (e.g., by means of firmware).

[0088] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0089] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0091] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for constructing a dynamic adjustment strategy of a driving style of an autonomous vehicle, characterized in that, include: S1. Collect multi-source observation data during the operation of autonomous vehicles, and construct a partially observable Markov decision model for human-machine co-driving based on the multi-source observation data. The state variables in the state space include the driver's trust level, the driver's cognitive workload, and the scene processing capability of the autonomous driving system. S2. Based on the multi-source observation data, train a potential dynamic model, and by performing time-series modeling on the observation sequence and behavior sequence, obtain the potential state representation of the human-machine co-driving system in the potential space; S3. Construct a semantic decoding network on the potential state representation, and use a weak supervision method based on semantic anchors to semantically align the potential state representation so that the potential state representation corresponds to the driver's trust level, the driver's cognitive workload, and the scene processing capability of the autonomous driving system, respectively. S4. Based on the semantically aligned latent states, construct a reinforcement learning decision model, generate interaction trajectories through the latent dynamics model, and optimize the policy in the semantic state space to obtain a dynamic adjustment policy for the driving behavior of autonomous vehicles. 2.The method of claim 1, wherein, The multi-source observation data specifically includes: Vehicle traffic environment status data, including collision time between vehicles and potential conflict targets, traffic scene complexity indicators, and vehicle operating status information; Driver behavior data, including driver takeover behavior indicator variables and the distance between the driver's foot and the pedals; and, Driver physiological signal data, including multi-channel EEG signal characteristics.

3. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 2, characterized in that, The construction of the partially observable Markov decision model for human-machine co-driving based on the multi-source observation data specifically includes: The state space contains state variables including the driver's trust level, the driver's cognitive workload, and the autonomous driving system's scene processing capabilities. The motion space, within which motion variables include the longitudinal acceleration adjustment of the vehicle and the lateral acceleration adjustment of the vehicle; The observation space includes collision indicator variables, driver takeover behavior indicator variables, counterfactual event outcome indicators, driver subjective cognitive workload questionnaire results, scenario difficulty indicators, EEG characteristics, collision time indicators between the vehicle and potential conflict targets, and the distance between the driver's foot and the pedal.

4. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 3, characterized in that, The weakly supervised method based on semantic anchors constructs semantic anchors by combining event-level semantic information and time-step-level agent features to perform semantic supervision training on the latent state representation. Specifically, the semantic anchor points include: The semantic anchor of driver trust is determined by the relationship between driver takeover behavior indicator variables and counterfactual operating results of the autonomous driving system. Semantic anchors for driver cognitive workload were constructed using the results of a driver's subjective cognitive workload questionnaire and EEG characteristics; and... The semantic anchor of the autonomous driving system's task processing capability is constructed using scenario difficulty indicators and collision time indicators between the vehicle and potential conflict targets.

5. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 4, characterized in that, A Top-K strategy is used to select high-confidence semantic anchors for supervised training of the latent state representation.

6. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 1, characterized in that, The potential dynamic model specifically includes: The observation embedding representation is obtained by encoding multi-source observation data, and the latent state is updated by a recursive state update network. The latent state prediction network is used to predict the distribution of the latent state at the next moment based on the current latent state and the current action. The latent dynamics model introduces an observation reconstruction network to reconstruct the real observation at the next moment based on the predicted latent state at the next moment. The latent dynamics model is trained using a model ensemble approach, and the latent state estimate is obtained by combining the prediction results of multiple latent dynamics sub-models.

7. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 1, characterized in that, The reward function of the reinforcement learning decision model includes a trust matching reward and a cognitive workload adjustment reward. The trust matching reward is used to penalize the difference between the driver's trust level and the scene processing capability of the autonomous driving system. The cognitive workload adjustment reward is used to constrain the driver's cognitive workload to remain within a set range.

8. The method for constructing a dynamic adjustment strategy for driving style of an autonomous vehicle according to claim 1, characterized in that, The reinforcement learning decision model performs multi-step trajectory prediction in the latent state space and uses the generated interaction trajectory to train the policy offline.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the program, it implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images