Vehicle intelligent interaction method and system based on AI

Abstract

The embodiment of the invention provides an AI-based vehicle intelligent interaction method and system, and belongs to the field of artificial intelligence interaction. The method comprises the following steps: dynamically judging a cognitive load level in a current driving scene through collaborative perception of a driver state and an environment state; determining a target man-machine interaction strategy of the vehicle based on the cognitive load level; wherein the target man-machine interaction strategy comprises a voice output mode, information density of screen display content and a feedback prompt mode; determining a corresponding trigger rule based on a switching relationship between the current man-machine interaction strategy and the target man-machine interaction strategy; and responding to a trigger signal of the determined trigger rule, executing man-machine interaction strategy switching, and executing vehicle interaction based on the switched target man-machine interaction strategy. According to the scheme of the invention, closed-loop adaptive control of a human-vehicle interaction strategy on a cognitive state is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence interaction, and in particular to an AI-based vehicle intelligent interaction method and an AI-based vehicle intelligent interaction system. Background Art

[0002] As vehicles become increasingly intelligent, human-machine interaction within the cockpit becomes increasingly complex. Multiple interaction methods, such as voice control, navigation assistance, visual interfaces, and entertainment systems, operate simultaneously while driving, forcing drivers to receive and process a large amount of information while simultaneously maneuvering the vehicle. In this context, coordinating the method and content of information output to meet driving needs while minimizing the cognitive burden on the driver has become a core challenge facing current intelligent cockpit interaction systems.

[0003] Cognitive load is an important indicator of a driver's psychological stress and information processing ability. Studies have shown that a driver's cognitive load is influenced by multiple factors, including the driving environment (e.g., traffic complexity and road type), the current task (e.g., navigation or making a phone call), and the driver's own state (e.g., fatigue and concentration). When cognitive load is excessively high, drivers may struggle to accurately process vehicle prompts or take timely action, increasing the risk of traffic accidents. Conversely, in scenarios with lower cognitive load, oversimplified information interaction can lead to distraction or ineffective utilization of intelligent functional resources.

[0004] Existing interactive systems usually use preset strategies or simple conditional triggering mechanisms to control information output, such as determining whether to activate voice prompts based on vehicle speed or whether the vehicle is in navigation mode. However, such interactive mechanisms usually ignore the comprehensive assessment of the driver's individual status and environmental complexity, and cannot dynamically adapt to the driver's cognitive tolerance in different driving scenarios. In addition, the current switching of interactive modes often lacks refined control, and it is difficult to automatically or semi-automatically adjust the interaction frequency, content details, prompt form, etc. according to changes in cognitive load, resulting in the system still causing information interference or response delays to the driver in some scenarios. Therefore, how to realize a technical mechanism that can perceive the driver's cognitive load state and intelligently switch the interaction strategy based on this state is still one of the key issues that need to be deeply addressed in the current field of intelligent driving interaction. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide an AI-based vehicle intelligent interaction method and system to at least solve the problem that existing in-vehicle interaction systems lack a dynamic recognition and response mechanism for the driver's cognitive load state, resulting in the inability to effectively switch interaction strategies according to the driving situation, thereby affecting driving safety and interaction experience.

[0006] In order to achieve the above-mentioned objectives, the first aspect of the present invention provides an AI-based vehicle intelligent interaction method, the method comprising: dynamically judging the cognitive load level in the current driving scenario through collaborative perception of the driver status and the environmental status; determining the target human-computer interaction strategy of the vehicle based on the cognitive load level; wherein the target human-computer interaction strategy includes the voice output mode, the information density of the screen display content, and the feedback prompt form; determining the corresponding triggering rules based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy; executing the human-computer interaction strategy switching in response to the trigger signal of the determined triggering rule, and executing the vehicle interaction based on the switched target human-computer interaction strategy.

[0007] Optionally, the perception rules of the driver's status include: collecting the driver's facial image, eye movement trajectory and head posture data; performing key point extraction and posture estimation processing on the facial image to generate a first type of status feature; performing time series pattern analysis on the eye movement trajectory to generate a second type of status feature; performing posture angle resolution operation on the head posture data to generate a third type of status feature; and fusing the first, second and third types of status features into a driver status representation vector for characterizing the driver's status.

[0008] Optionally, the perception rules of the environmental state include: collecting current road type, traffic density, weather conditions and vehicle speed information to construct an environmental data set; converting the environmental data set into a node set for constructing an environmental state graph structure; performing a graph embedding generation operation on the environmental state graph structure to extract an environmental feature vector representing the complexity of the scene; combining the geographic information system with real-time traffic event identification to adjust the contextual semantics of the environmental feature vector, and outputting the environmental feature vector after adjustment to characterize the environmental state.

[0009] Optionally, the judgment rule for the cognitive load level in the current driving scenario is: taking the driver state representation vector and the weighted environment feature vector as input to perform splicing of the driver state representation vector and the weighted environment feature vector to obtain a joint input tensor as a cognitive state sample; inputting the cognitive state sample into the graph neural network model to perform cross-modal graph reasoning operations; wherein, during the reasoning process, the key path nodes are identified based on the attention mechanism, and their feature vectors are aggregated, the aggregated feature vectors are input into the classification module, and the current cognitive load level label is output.

[0010] Optionally, the graph neural network model is constructed using a meta-learning method, and the corresponding construction rules include: constructing a task-level training sample set, which contains historical cognitive state sequences and load labels of multiple drivers; executing a meta-training process to optimize the initial parameters of the model; collecting target driver samples after deployment, and performing fine-tuning update operations based on the target driver samples to obtain a graph neural network model suitable for the target driver.

[0011] Optionally, the rules for executing the meta-training process include: collecting driver state data and environmental state data, and organizing them into a training sample sequence with a time series structure according to chronological order; generating a cognitive load level label corresponding to the sample based on the historical driving scenarios corresponding to the sample sequence, combined with expert scores, EEG signals and / or self-assessment scales; using the training sample sequence and its labels as input to execute a training process based on the graph neural network model; during the training process, constructing a joint loss function, which includes a cross-entropy loss term for classification accuracy and an auxiliary loss term for feature attention enhancement; introducing a temporal smoothing regularization term in the loss function to constrain the consistency of the model output at adjacent moments; completing the model training under the condition of minimizing the loss function, and outputting a deployable graph neural network model.

[0012] Optionally, the target human-computer interaction strategy of the vehicle is determined based on the cognitive load level, including: retrieving a matching template in an interaction strategy template library according to the cognitive load level; parsing structural parameters regarding voice output, information density, and feedback prompts in the template; performing parameter fine-tuning optimization operations in combination with the current driving task type; constructing an interaction strategy configuration file, including voice broadcast frequency, interface display complexity, and feedback type instructions; and outputting the configuration file as the target human-computer interaction strategy.

[0013] Optionally, based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy, the corresponding trigger rules are determined, including: calculating the amplitude of the interaction load change based on the structural differences between the current interaction strategy and the target interaction strategy; judging whether the switching scenario type is from low load to high load or from high load to low load in combination with the direction of cognitive load change; generating the corresponding trigger logic type, including automatic triggering, authorization triggering or confirmation triggering; establishing a trigger rule mapping table to record the binding relationship between each trigger logic and the system signal channel as the determined trigger rule.

[0014] The second aspect of the present invention provides an AI-based vehicle intelligent interaction system, which includes: a monitoring unit for dynamically judging the cognitive load level in the current driving scenario through collaborative perception of the driver's state and the environmental state; a strategy determination unit for determining the vehicle's target human-computer interaction strategy based on the cognitive load level; wherein the target human-computer interaction strategy includes the voice output mode, the information density of the screen display content, and the feedback prompt form; a rule determination unit for determining the corresponding trigger rule based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy; an execution unit for executing the human-computer interaction strategy switching in response to the trigger signal of the determined trigger rule, and executing the vehicle interaction based on the switched target human-computer interaction strategy.

[0015] On the other hand, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, enables the computer to execute the above-mentioned AI-based vehicle intelligent interaction method.

[0016] Through the above technical solution, the solution of the present invention integrates the driver status and environmental status to judge the cognitive load level during driving in real time, thereby achieving a dynamic understanding of the driving situation. Based on the recognition results, the system can adaptively adjust interaction strategies such as voice prompts, interface display and feedback methods, so that the output information is more in line with the driver's cognitive tolerance. At the same time, by setting switching rules between interaction strategies, the system can automatically or semi-automatically complete strategy transitions when the load changes, avoiding information overload or insufficient interaction, and improving the efficiency and safety of information acquisition during driving. Overall, this solution realizes closed-loop adaptive control of interaction strategies to cognitive states.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0019] Figure 1 This is a flowchart of the steps of an AI-based vehicle intelligent interaction method provided by one embodiment of the present invention;

[0020] Figure 2 This is a system structure diagram of an AI-based vehicle intelligent interaction system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0022] Figure 1 This is a flow chart of a method for intelligent vehicle interaction based on AI provided by one embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides an AI-based vehicle intelligent interaction method, the method comprising:

[0023] Step S10: Dynamically determine the cognitive load level in the current driving scenario through collaborative perception of the driver's state and the environment state.

[0024] Specifically, the perception rules of the driver's status include: collecting the driver's facial image, eye movement trajectory and head posture data; performing key point extraction and posture estimation processing on the facial image to generate a first type of status feature; performing time series pattern analysis on the eye movement trajectory to generate a second type of status feature; performing posture angle resolution operation on the head posture data to generate a third type of status feature; fusing the first, second and third types of status features into a driver status representation vector for characterizing the driver's status.

[0025] In an embodiment of the present invention, multi-source status data of the driver is collected, including facial images, eye movement trajectories, and head posture angle information. Facial images can be collected at a fixed frame rate by a visible light camera, and the image resolution should meet no less than the preset facial feature point extraction requirements. Eye movement trajectories are obtained by an infrared eye tracker or an eye movement detection algorithm based on image processing, and include continuous sequence data such as the two-dimensional coordinates of the gaze point, pupil diameter, blink frequency, and eye saccade speed. Head posture data can be obtained by calculating the position changes of facial key points in the image sequence, or by obtaining the three-axis angle information output by the posture sensing module to form an angle sequence including yaw angle, pitch angle, and roll angle.

[0026] The facial image is then processed for key point extraction and posture estimation. Key point extraction utilizes a 68-point facial structure detection algorithm based on a convolutional neural network. This algorithm obtains the coordinates of structural key points in each image frame, including eyebrows, eyes, mouth corners, nose tip, and chin, and calculates changes in their geometric relationships. Based on the positional changes of these points, static and dynamic facial metrics such as the eyelid opening / closing ratio (EAR), mouth opening / closing ratio (MAR), and micro-expression variation are further extracted to generate first-class state features that reflect facial behavioral patterns such as fatigue, yawning, frowning, and drowsiness.

[0027] Perform temporal pattern analysis on eye movement trajectories, including spatial distribution hotspots of fixation points, sequences of fixation durations, changes in saccade rate, and trends in pupil diameter over time. A sliding window structure is used to perform segmented statistics and pattern modeling on temporal eye movement data. This can be combined with temporal neural networks (such as LSTM) or frequency domain analysis methods to identify features such as distracted attention, frequent gaze switching, and gaze loss, generating second-class state features to characterize gaze stability and attention control ability.

[0028] Perform attitude angle calculation on head posture data. Differentiate the rate of change of angles in consecutive frames to extract dynamic indicators such as head turn frequency, average deflection angle, and pitch amplitude. This allows identification of behavioral characteristics such as frequent head lowering, frequent side glances, and prolonged periods of non-direct gaze. This generates third-category state features that reflect the consistency and stability of the driver's head direction control.

[0029] The first, second, and third category state features are normalized and concatenated into a unified vector structure according to the preset dimensions. Linear concatenation, weighted fusion, or dimensionality reduction based on feature selection can be used to form the final driver state representation vector. This vector serves as input to the cognitive state assessment model and is stable, semantically distinguishable, and temporally continuous.

[0030] This rule constructs a multi-dimensional driver behavior profile by collaboratively sensing facial, eye movement, and head movement signals. Compared to single-signal input methods, it more comprehensively captures typical driver behavior characteristics such as attention, fatigue, and head turning, effectively improving the accuracy and robustness of cognitive load estimation. Furthermore, the rule's clear structure and standardized processing facilitate replication and deployment across different hardware platforms.

[0031] Preferably, the perception rules of the environmental state include: collecting current road type, traffic density, weather conditions and vehicle speed information to construct an environmental data set; converting the environmental data set into a node set for constructing an environmental state graph structure; performing a graph embedding generation operation on the environmental state graph structure to extract an environmental feature vector representing the complexity of the scene; combining the geographic information system with real-time traffic event identification to adjust the contextual semantics of the environmental feature vector, and outputting the environmental feature vector after adjustment for characterizing the environmental state.

[0032] In this embodiment of the present invention, basic environmental information, such as the current vehicle's road type, traffic density, meteorological conditions, and the vehicle's own operating speed, is collected as the raw input for constructing an environmental data set. Road types can be obtained from high-precision maps and include categories such as urban main roads, expressways, ring roads, and rural roads. Traffic density is calculated by counting the number of targets captured by front, rear, and side radars or cameras, and can be converted into an indicator of vehicle traffic per unit time or vehicle density per unit area. Meteorological conditions, including precipitation, haze, humidity, temperature, and other parameters, can be obtained through on-board sensors or by accessing a weather API. Vehicle speed is read in real time by the on-board controller.

[0033] The above-mentioned multiple environmental data are structured into an environmental data set in a unified format, where each data item is represented as a triple containing a location label, a numerical field, and a timestamp. The environmental data set is node-based, that is, each type of environmental element (such as road type, traffic density, etc.) is abstracted into a node type in the graph structure, and each node contains corresponding state attributes. For example, "current road type" is a node, and its attributes may be "highway", "urban main road", etc.; "traffic density" is another node, and its attribute is a numerical density level. All nodes are connected by edges to form an environmental state graph, and the weight of the edge can be set as the correlation coefficient between the data or the spatial position association relationship.

[0034] Perform graph embedding on the environment state graph structure. Specifically, graph convolution methods within graph neural networks (such as GCN and GAT) can be used to aggregate features for each node, performing a weighted combination of the node's own attributes and those of its adjacent nodes. A low-dimensional vector representation of each node is generated through a multi-layer graph embedding process. Finally, the embedding vectors of all nodes in the graph are aggregated using global pooling methods (such as mean pooling or attention pooling), outputting an environment feature vector representing the complexity of the current scene.

[0035] The generated environmental feature vectors are aligned and semantically adjusted with geographic information system (GIS) data and real-time traffic event signs. GIS data provides contextual information such as historical congestion levels, high accident probability, and speed limit zones for the current road section. Real-time traffic events include emergency alerts such as construction sections, road occupation accidents, and temporary traffic restrictions. This information is encoded into auxiliary vectors and concatenated or weightedly fused with the environmental feature vectors to enhance the vector's ability to capture the real-world context.

[0036] The fused environmental feature vector is output as the final representation of the environmental state. This vector is comparable and discriminative across multiple driving scenarios and can serve as the environmental input for the cognitive load assessment model. Compared to traditional single-factor environmental input methods, this rule-based graph-based modeling approach captures the coupling relationships between multiple factors, enhancing the contextual awareness of environmental information. This makes cognitive load assessment more sensitive and adaptable to specific scenarios, helping to improve the accuracy and timeliness of strategic adjustments in complex or sudden traffic situations.

[0037] Furthermore, the judgment rule of the cognitive load level in the current driving scenario is: taking the driver state representation vector and the weighted environmental feature vector as input to perform the splicing of the driver state representation vector and the weighted environmental feature vector to obtain a joint input tensor as a cognitive state sample; inputting the cognitive state sample into the graph neural network model to perform cross-modal graph reasoning operations; wherein, during the reasoning process, the key path nodes are identified based on the attention mechanism, and their feature vectors are aggregated, the aggregated feature vectors are input into the classification module, and the current cognitive load level label is output.

[0038] In this embodiment of the present invention, a constructed driver state representation vector and environmental feature vector are obtained. The former represents the driver's behavioral characteristics after multimodal fusion, including dimensional information such as facial expression, eye movement patterns, and head posture; the latter represents the environmental state through graph structure modeling and graph embedding, including scene elements such as road type, traffic density, and weather conditions. These two vectors are concatenated in the order of feature dimensions to form a joint input tensor of unified dimension, which serves as a cognitive state sample. The sample format can be set as a vector of the form [1, D], where D represents the total feature dimension after fusion.

[0039] Next, the cognitive state sample is input into a pre-built graph neural network model. The model can be built based on a graph convolutional network (GCN), a graph attention network (GAT), or a dynamic graph neural network (DGN). The model structure includes at least two layers of graph feature propagation layers and an output classification layer. Within the model, the input sample is regarded as a super node in the graph and connected to multiple virtual nodes representing driver behavior sub-features and environmental sub-features to construct a graph structure for cross-modal information fusion. Each node contains an independent feature channel, and the edge weight can be set according to the similarity between features or a predefined coupling relationship.

[0040] During graph neural network inference, feature propagation is performed between nodes, and an attention mechanism is introduced. Specifically, at each propagation layer, for each target node, messages are obtained from its neighboring nodes and weighted summed based on attention weights. Attention weights are calculated using a trainable scoring function. Common methods include dot-product attention or feedforward network scoring mechanisms. This dynamically identifies which feature dimensions are most contributing to the current driving scenario. After multiple layers of propagation, a fused and enhanced aggregate feature vector is obtained.

[0041] Finally, this aggregated feature vector is fed into a classification module, which can consist of a set of fully connected neural network layers. The output is a multi-class predicted label representing the driver's current cognitive load level. This level can be categorized as low, medium, or high, or even more fine-grained based on actual training data. The classification output uses a softmax activation function to generate a probability distribution, ultimately selecting the class with the highest probability as the output label.

[0042] This judgment rule, based on the integration of information about the driver's individual state and the complexity of the external environment, introduces graph structure modeling and an attention mechanism. This effectively improves the ability to model the coupled relationships between multi-source heterogeneous information, enhances the dynamic response to key features, and thus improves the accuracy and robustness of cognitive load level judgment. Compared with traditional linear classification models, this method can more meticulously model the interactions between features and is suitable for complex and changing real-world driving scenarios.

[0043] Step S20: determining a target human-computer interaction strategy for the vehicle based on the cognitive load level.

[0044] Specifically, the target human-machine interaction strategy includes voice output mode, information density of screen display content, and feedback prompt form. The graph neural network model is constructed using a meta-learning approach. The corresponding construction rules include: constructing a task-level training sample set containing historical cognitive state sequences and load labels for multiple drivers; performing a meta-training process to optimize the model's initial parameters; and collecting target driver samples after deployment. Fine-tuning and updating these samples to obtain a graph neural network model suitable for the target driver.

[0045] Furthermore, the rules for executing the meta-training process include: collecting driver status data and environmental status data, and organizing them into a training sample sequence with a time series structure according to chronological order; based on the historical driving scenarios corresponding to the sample sequence, combined with expert scores, EEG signals and / or self-assessment scales, generating cognitive load level labels corresponding to the samples; using the training sample sequence and its labels as input, executing a training process based on the graph neural network model; during the training process, constructing a joint loss function, which includes a cross-entropy loss term for classification accuracy and an auxiliary loss term for feature attention enhancement; introducing a temporal smoothing regularization term in the loss function to constrain the consistency of the model output at adjacent moments; completing model training under the condition of minimizing the loss function, and outputting a deployable graph neural network model.

[0046] Specifically, the target human-computer interaction strategy of the vehicle is determined based on the cognitive load level, including: searching for a matching template in an interaction strategy template library according to the cognitive load level; parsing structural parameters related to voice output, information density, and feedback prompts in the template; performing parameter fine-tuning and optimization operations in combination with the current driving task type; constructing an interaction strategy profile, including voice broadcast frequency, interface display complexity, and feedback type instructions; and outputting the profile as the target human-computer interaction strategy.

[0047] In an embodiment of the present invention, the rules for determining the vehicle's target human-computer interaction strategy based on cognitive load level include: after completing the perception of the driver's state and the environmental state, obtaining a cognitive load level label for the current driving scenario, and retrieving the corresponding strategy template from the interaction strategy template library based on this label. Cognitive load levels are generally divided into three levels: low, medium, and high, and can also be set to more detailed levels based on actual application requirements. Each level corresponds to a set of interaction control parameters used to control voice broadcast behavior, screen interface complexity, and feedback mechanism type.

[0048] Specifically, when retrieving interaction strategy templates, the first step is to retrieve a template entry that matches the current level. This template contains a set of pre-set structured parameters. These include voice output parameters such as announcement frequency, speech rate, sentence length, and announcement trigger conditions; screen information density parameters such as the number of display modules in the interface, the number of fields within each module, and the ratio of graphics to text; and feedback parameters such as whether haptic feedback is enabled, the intensity level, and the combination of light, sound, and vibration. Each template is organized in JSON or a structured dictionary format and can be directly loaded and parsed.

[0049] To improve the adaptability of the strategy, after completing the template retrieval, the template parameters need to be fine-tuned in combination with the current driving task type. Driving task types can be divided into: normal cruising, lane change overtaking, low-speed following, on- and off-ramps, and driving in traffic congestion. Based on the task type, a weight factor is set to perform weighted adjustments on the parameters in the strategy template. For example, in the "high-speed straight-line cruising" task, the voice broadcast frequency and screen information density can be appropriately increased; while in the "on- and off-ramps" task, the output of non-critical information needs to be significantly reduced, retaining only the most core prompt content. The adjusted parameter set is encapsulated as an interaction strategy configuration file, which is used as the direct output of the target human-computer interaction strategy for subsequent interaction control module calls.

[0050] The graph neural network model used to calculate the current cognitive load level label is constructed using a meta-learning approach. This construction process consists of three stages: task-level training sample construction, meta-training parameter optimization, and adaptive fine-tuning of the target driver sample.

[0051] During the task-level training sample construction phase, historical driving data is collected from multiple drivers. A training task set is constructed, consisting of a sequence of driver states, a sequence of environmental states, and their corresponding cognitive load level labels. Each task consists of a sequence of samples from a continuous driving process, with the sequence structure maintaining temporal consistency. The sample length can be a fixed window length (e.g., 10 seconds), or a sliding window sampling strategy can be used to generate overlapping sequences.

[0052] Label generation requires combining driving scene reconstruction techniques with auxiliary evaluation signals to annotate training samples. This can be achieved through one or a combination of three methods: expert scoring (reference video + physiological data), subjective attention metrics obtained through wearable EEG devices, and self-administered driving task perception scales (such as the NASA-TLX rating scale). Label values are uniformly mapped to a predefined hierarchical classification space.

[0053] During the meta-training phase, the task set is fed into a meta-learning optimizer (e.g., one based on MAML or Reptile). During training, the "few-shot adaptation" scenario for new drivers is repeatedly simulated. Through multiple rounds of task-level inner and outer loop training, the initial model parameters are optimized to achieve rapid adaptation capabilities. This parameter set serves as the base weights for the graph neural network model and serves as a starting point for fine-tuning after deployment.

[0054] The graph neural network model structure includes an input encoding layer, a cross-modal feature construction layer, a graph message propagation layer, an attention feature aggregation layer, and an output classification layer. The input layer receives the concatenated driver state vector and environment feature vector and performs feature mapping and dimension alignment operations. The cross-modal construction layer constructs a heterogeneous node structure based on the input feature dimensions, with each node representing a class of sub-features, and edge connections representing attribute correlations or empirical weight relationships. The message propagation layer uses graph convolution or graph attention mechanisms to perform inter-node information fusion operations. The aggregation layer uses the attention mechanism to extract node features that have a significant impact on the final prediction. The output layer is a multi-classification layer that uses softmax activation to output cognitive load level labels.

[0055] The model's training rules are as follows: After collecting raw driver and environment state data, they are organized into time-series training samples; these samples are input into the graph neural network model, and forward propagation and gradient updates are performed. During training, a joint loss function is used for optimization, which consists of three parts:

[0056] 1) Cross entropy loss, used to improve classification accuracy.

[0057] 2) Feature attention auxiliary loss, which guides the model to focus on key nodes by constraining the sparsity and stability of attention weights.

[0058] 3) The temporal smoothing regularization term imposes continuity constraints on the prediction outputs at adjacent moments, reducing the abrupt changes in the classification results over time. The overall loss is a weighted sum of the three terms, with the weight coefficients set based on the performance of the validation set.

[0059] After the model is deployed, state samples of the target driver are collected from multiple short driving segments (e.g., 3-5 minutes of data) and rapidly fine-tuned based on the existing initialization parameters. The fine-tuning process uses a low learning rate for a limited number of updates, updating only some weight layers (such as the attention layer or classification head) to avoid overfitting the model to small samples and losing generalization ability. After fine-tuning, a graph neural network model specific to the target driver is output for use in the actual cognitive load level determination task.

[0060] After determining the cognitive load level, the entire strategic decision-making process outputs the target human-machine interaction strategy through template retrieval and parameter fine-tuning mechanisms. This strategy boasts real-time responsiveness and personalized adaptability, dynamically adjusting the density and format of information output based on the driver's cognitive load in complex driving tasks, effectively avoiding information overload or omissions. Compared to traditional fixed or rule-driven interaction strategies, this approach significantly improves information transmission efficiency and driving safety during driving. It is particularly suitable for complex scenarios, high-speed driving, and fatigue-prone environments, where interaction accuracy is critical.

[0061] Step S30: Determine corresponding triggering rules based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy.

[0062] Specifically, based on the structural differences between the current interaction strategy and the target interaction strategy, the amplitude of the interaction load change is calculated; combined with the direction of cognitive load change, the switching scenario type is judged as low load to high load or high load to low load; the corresponding trigger logic type is generated, including automatic triggering, authorization triggering or confirmation triggering; a trigger rule mapping table is established to record the binding relationship between each trigger logic and the system signal channel as the determined trigger rule.

[0063] In an embodiment of the present invention, the structural differences between the current human-computer interaction strategy and the target human-computer interaction strategy are analyzed. The current interaction strategy can be represented as a structured configuration file, which includes voice broadcast parameters, screen display density parameters, and feedback prompt control parameters. The target interaction strategy is also expressed in the same format. The various parameters of the two are compared to calculate their structural difference. The difference can be achieved through a weighted distance function, such as using a normalized difference for the voice broadcast frequency, using a structural similarity metric for the number of interface elements, and using a discrete state mapping distance for the feedback prompt type. The above-mentioned difference dimensions are merged into a unified interaction load change amplitude indicator, which can be set to a real value R in the range of [0, 1].

[0064] Next, we categorize the switching scenarios based on the direction of cognitive load change. If we identify a shift in cognitive load from a lower level to a higher level, we define the current scenario as "low-load to high-load switching"; otherwise, we define it as "high-load to low-load switching." This determination is based on a comparison of cognitive load levels at two consecutive moments. Combining the load change direction and the structural difference magnitude R, we can use the following rules to set the switching trigger type:

[0065] 1) If it is "low load to high load" and R exceeds the preset threshold (such as 0.3), the trigger type is set to automatic trigger and the switch is immediate without user confirmation.

[0066] 2) If it is "high load to low load" and R is in the medium range (such as 0.1 to 0.3), the trigger type is set to authorized trigger, and a prompt message is sent to the driver, and the driver actively confirms before switching;

[0067] 3) If it is "high load to low load" and R is lower than the lower threshold (such as 0.1), it is set as a confirmation trigger and a recommended switching option is provided, and the driver can choose whether to execute it.

[0068] After generating a trigger type, you need to create a trigger rule mapping table to define the signal response channels and operation priorities corresponding to different trigger types. The mapping table includes the trigger type, the required confirmation method (pop-up prompt / voice prompt), the bound signal channel (such as scheduled scan, status detection interrupt, voice interaction request), and whether to interrupt the current information output process. Each trigger rule is identified by a unique identifier in the mapping table for quick indexing.

[0069] Ultimately, the analysis results are output as trigger rules, and subsequent policy switching is executed according to these rules. This rule-based mechanism ensures that the human-machine interaction policy switching process not only considers cognitive state fluctuations but also fully weighs the degree of information interference. This ensures that information switching is timely and not excessively frequent, thereby improving the interactive system's responsiveness and fault tolerance to changes in the driver's cognitive state. In particular, in high cognitive load scenarios, the automated switching mechanism minimizes driver attention diversion and reduces potential risks. In low-load scenarios, however, it provides drivers with more autonomy and enhances the comfort and personalization of the interaction experience.

[0070] Step S40: In response to the trigger signal of the determined trigger rule, executing human-computer interaction strategy switching, and executing vehicle interaction based on the switched target human-computer interaction strategy.

[0071] Specifically, read the currently activated human-computer interaction strategy configuration file and the target interaction strategy configuration file to be switched, and compare the differences between the two in voice broadcast parameters, screen display content information density and feedback prompt mechanism; reconstruct and load the interaction-related modules according to the parameters in the target strategy configuration file. The voice broadcast parameters include the broadcast frequency, broadcast length and speech speed settings of the voice content. After loading, adjust the voice scheduler output interval and text template; information density adjustment involves the interface level refresh strategy, which updates the UI display logic according to the number of interface modules and content granularity set in the target configuration, including module visibility, content hierarchy and interactive element layout; the feedback prompt mechanism includes whether to enable vibration prompts, light signal frequency settings, voice prompt repetition times, etc., and reinitialize the corresponding feedback output control process according to the target strategy settings.

[0072] After the strategy switching is completed, the updated interaction strategy configuration is immediately enabled and the interaction execution phase begins. During the execution phase, the voice, interface, and prompt output modules begin to operate normally according to the target strategy settings. For example, when the cognitive load is at a high level, the interface display only retains the navigation path and necessary speed warning information, the voice prompt frequency is reduced and the content is limited to risk reminders, and all non-critical feedback prompts (such as entertainment push and schedule reminders) are suspended. Through real-time response and fine control of the strategy switching process, it can ensure that the vehicle interaction output content matches the driver's current cognitive carrying capacity, improve the timeliness and safety of the interaction, while reducing unnecessary information interference and enhancing the information access efficiency and driving stability during driving.

[0073] Figure 2 This is a system structure diagram of an AI-based vehicle intelligent interaction system provided by an embodiment of the present invention. Figure 2 As shown, an embodiment of the present invention provides an AI-based vehicle intelligent interaction system, which includes: a monitoring unit, which is used to dynamically judge the cognitive load level in the current driving scene through collaborative perception of the driver state and the environment state; a strategy determination unit, which is used to determine the target human-computer interaction strategy of the vehicle based on the cognitive load level; wherein the target human-computer interaction strategy includes the voice output mode, the information density of the screen display content, and the feedback prompt form; a rule determination unit, which is used to determine the corresponding trigger rule based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy; an execution unit, which is used to execute the human-computer interaction strategy switching in response to the trigger signal of the determined trigger rule, and execute the vehicle interaction based on the switched target human-computer interaction strategy.

[0074] An embodiment of the present invention further provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, enables the computer to execute the above-mentioned AI-based vehicle intelligent interaction method.

[0075] Those skilled in the art will appreciate that all or part of the steps in the methods of the aforementioned embodiments can be accomplished by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a single-chip microcomputer, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0076] The above describes in detail the optional embodiments of the present invention in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, a variety of simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the scope of protection of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will no longer describe the various possible combinations separately.

[0077] In addition, the various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. An AI-based vehicle intelligent interaction method, characterized in that: The method comprises: Dynamically judge the cognitive load level in the current driving scenario through collaborative perception of the driver's state and the environment; Determine the target human-computer interaction strategy of the vehicle based on the cognitive load level; wherein, The target human-computer interaction strategy includes the voice output mode, the information density of the screen display content, and the feedback prompt form; Based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy, the corresponding triggering rules are determined; In response to a trigger signal of the determined trigger rule, a human-machine interaction strategy switching is performed, and vehicle interaction is performed based on a target human-machine interaction strategy after the switching.

2. The method according to claim 1, characterized in that The driver status perception rules include: Collect driver's facial images, eye movement trajectory and head posture data; Performing key point extraction and posture estimation processing on the facial image to generate a first type of state feature; Perform temporal pattern analysis on the eye movement trajectories to generate the second type of state features; Performing posture angle calculation on the head posture data to generate a third type of state feature; The first, second and third category state features are fused into a driver state representation vector to characterize the driver state.

3. The method according to claim 1, characterized in that The perception rules of the environmental state include: Collect information on current road type, traffic density, weather conditions, and vehicle speed to construct an environmental data set; Converting the environment data set into a node set for constructing an environment state graph structure; Performing a graph embedding generation operation on the environment state graph structure to extract an environment feature vector representing scene complexity; The context semantics of the environmental feature vector are adjusted by combining the geographic information system with the real-time traffic event identification, and the environmental feature vector is output after the adjustment to represent the environmental state.

4. The method according to claim 2 or 3, characterized in that The judgment rules for the cognitive load level in the current driving scenario are: Taking the driver state representation vector and the weighted environment feature vector as input, performing concatenation of the driver state representation vector and the weighted environment feature vector to obtain a joint input tensor as a cognitive state sample; The cognitive state sample is input into the graph neural network model to perform cross-modal graph reasoning operations; wherein, During the reasoning process, the key path nodes are identified based on the attention mechanism, and their feature vectors are aggregated. The aggregated feature vectors are input into the classification module, and the current cognitive load level label is output.

5. The method according to claim 4, characterized in that The graph neural network model is constructed using a meta-learning method, and the corresponding construction rules include: Construct a task-level training sample set, which includes the historical cognitive state sequences and load labels of multiple drivers; Perform a meta-training process to optimize the model’s initial parameters; After deployment, target driver samples are collected, and fine-tuning and updating operations are performed based on the target driver samples to obtain a graph neural network model suitable for the target driver.

6. The method according to claim 5, characterized in that The rules for executing the meta-training process include: Collect driver status data and environment status data, and organize them into a training sample sequence with a time series structure according to the time sequence; Based on the historical driving scenarios corresponding to the sample sequence, combined with expert scores, EEG signals and / or self-assessment scales, a cognitive load level label corresponding to the sample is generated; Taking the training sample sequence and its label as input, executing a training process based on a graph neural network model; During the training process, a joint loss function is constructed, which includes a cross-entropy loss term for classification accuracy and an auxiliary loss term for feature attention enhancement; A temporal smoothing regularization term is introduced into the loss function to constrain the consistency of the model output at adjacent moments; The model training is completed under the condition of minimizing the loss function, and a deployable graph neural network model is output.

7. The method according to claim 1, characterized in that Determining a target human-machine interaction strategy for the vehicle based on the cognitive load level includes: Retrieving matching templates from the interaction strategy template library according to the cognitive load level; Analyzing structural parameters of the template regarding speech output, information density, and feedback prompts; Based on the current driving task type, perform parameter fine-tuning and optimization operations; Build interaction strategy profiles, including voice broadcast frequency, interface display complexity, and feedback type instructions; The configuration file is output as the target human-computer interaction strategy.

8. The method according to claim 1, characterized in that Based on the switching relationship between the current human-computer interaction strategy and the target human-computer interaction strategy, the corresponding triggering rules are determined, including: Based on the structural differences between the current interaction strategy and the target interaction strategy, the change amplitude of the interaction load is calculated; Combined with the direction of cognitive load change, determine whether the switching scenario type is from low load to high load or from high load to low load; Generate corresponding trigger logic types, including automatic triggering, authorization triggering, or confirmation triggering; A trigger rule mapping table is established to record the binding relationship between each trigger logic and the system signal channel as a determined trigger rule.

9. An AI-based vehicle intelligent interaction system, characterized in that: The system comprises: A monitoring unit, which is used to dynamically determine the cognitive load level in the current driving scenario through collaborative perception of the driver's state and the environment; A strategy determination unit is used to determine a target human-computer interaction strategy for the vehicle based on the cognitive load level; wherein, The target human-computer interaction strategy includes the voice output mode, the information density of the screen display content, and the feedback prompt form; A rule determination unit, configured to determine a corresponding trigger rule based on a switching relationship between a current human-computer interaction strategy and a target human-computer interaction strategy; The execution unit is used to execute human-computer interaction strategy switching in response to a trigger signal of a determined trigger rule, and execute vehicle interaction based on a target human-computer interaction strategy after switching.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the AI-based vehicle intelligent interaction method described in any one of claims 1 to 8.

Images