Human-computer interaction method and system for driving training applied to vehicle HUD

Abstract

The present invention provides a human-computer interaction method and system for driving training applied to a vehicle HUD, which relates to the field of intelligent driving technology, including obtaining physiological characteristic data of a driver and behavioral characteristic data of the driver, using a deep neural network to establish a driver cognitive load level prediction model and a driving behavior pattern recognition model to obtain the driver's current cognitive load level and driving behavior pattern; using a collaborative filtering algorithm to build a driver ability profile; extracting training knowledge points that match the driver's ability profile from a preset training knowledge base, using a reinforcement learning algorithm to sort the difficulty of the training knowledge points, and finally forming adaptive training content; using a visual saliency detection algorithm to enhance the adaptive training content in real time, and combining the driver's current cognitive load level, using an information entropy minimization algorithm to dynamically adjust the information density in the adaptive training content.

Description

Technical Field

[0001] The present invention relates to intelligent driving technology, and in particular to a human-computer interaction method and system for driving training applied to a vehicle HUD. Background Art

[0002] With the popularization of in-vehicle information systems and the improvement of their intelligence, the ways in which drivers interact with vehicles are becoming increasingly diverse. Among them, the Head-Up Display (HUD), with its unique design concept and display method, provides drivers with a more intuitive and efficient information presentation solution. Applying HUD technology to the field of driver training can effectively improve the pertinence and real-time nature of training, and accelerate drivers' acquisition and understanding of key information.

[0003] Traditional driver training mainly relies on classroom lectures and simulator exercises, which have limitations such as relatively rigid training content and inability to adjust dynamically. With the development of emerging technologies such as artificial intelligence and big data, intelligent driver training systems have gradually attracted attention. These systems adaptively generate personalized training content and interaction strategies by analyzing drivers' behavioral patterns and cognitive characteristics, thus achieving dynamic optimization of the training process. However, existing intelligent training systems still have deficiencies in information presentation and lack guidance on driver attention allocation, which can easily cause excessive cognitive load on drivers. Summary of the invention

[0004] The embodiments of the present invention provide a human-computer interaction method and system for driving training applied to a vehicle HUD, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention,

[0006] Provides a human-computer interaction method for driving training applied to vehicle HUD, including:

[0007] Acquire the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; use a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, and perform real-time prediction and recognition based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern;

[0008] Based on the driver's historical training data, the collaborative filtering algorithm is used to build a driver's ability profile to determine the driver's mastery of different knowledge points; the training knowledge points that match the driver's ability profile are extracted from the preset training knowledge base, and the training knowledge points are sorted by difficulty using the reinforcement learning algorithm, and combined with the driver's current cognitive load level, adaptive training content is finally formed;

[0009] According to the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time. Combined with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability.

[0010] In an optional embodiment,

[0011] A driver cognitive load level prediction model and a driving behavior pattern recognition model are established using a deep neural network, and real-time prediction and recognition are performed based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern, including:

[0012] The preprocessed physiological characteristic data are input into the cognitive load level prediction model based on the long short-term memory neural network. The temporal characteristics of the physiological characteristic data are extracted through the multi-layer long short-term memory neural network, and the attention mechanism is introduced to adaptively allocate the weights corresponding to the temporal characteristics at different moments. After the fully connected layer and activation function, the predicted value of the cognitive load level is obtained.

[0013] The behavior feature data are preprocessed, statistical values ​​of the behavior feature data are extracted, and a behavior heat map of the behavior feature data is constructed; the behavior feature data and the behavior heat map are input into a driving behavior pattern recognition model jointly constructed based on a convolutional neural network and a long short-term memory neural network for feature extraction and feature fusion to obtain fusion features; multi-classification is performed through the fully connected layer and the classification layer of the driving behavior pattern recognition model to obtain the probability distribution of the driving behavior pattern in each time window, and the driving behavior pattern with the highest probability distribution value is used as the driving behavior pattern of the current time window.

[0014] In an optional embodiment,

[0015] Extract training knowledge points that match the driver's ability profile from the preset training knowledge base, use the reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form an adaptive training content including:

[0016] Build a training knowledge base covering driving training, classify and label the knowledge points in the training knowledge base, and the attributes of the knowledge points include the category, difficulty level and prerequisite knowledge points; use the historical training data of drivers, explore the similarities between different drivers in the mastery of knowledge points, use collaborative filtering algorithms to predict the current driver's mastery of unlearned knowledge points, and determine the set of knowledge points that match the driver's ability;

[0017] Based on the knowledge point set that matches the driver's ability, a reinforcement learning algorithm is used to sort the knowledge point. The knowledge point sorting problem is modeled as a Markov decision process, where the state is the completed knowledge point sequence, the action is to select the next knowledge point, and the reward comprehensively considers the difficulty of the knowledge point, the matching degree of the driver's ability, and the dependency of the knowledge points. Through continuous learning, the optimal sorting strategy that maximizes the cumulative reward is found.

[0018] Based on the comparison between the driver's current cognitive load level and the preset reasonable cognitive load range, the difficulty of the training content is dynamically adjusted to form adaptive training content.

[0019] In an optional embodiment,

[0020] The method also includes dynamically adjusting the difficulty of the training content:

[0021] The Bayesian knowledge tracking model is used to track the probability of drivers mastering each knowledge point in the training content in real time, and dynamically adjust the presentation frequency and presentation time of each knowledge point. The presentation frequency and presentation time of knowledge points with a mastery probability higher than the preset threshold are reduced, and the presentation frequency and presentation time of knowledge points with a mastery probability lower than the preset threshold are increased.

[0022] According to the driver's current cognitive load level, the fuzzy control algorithm is used to dynamically adjust the difficulty gradient of the knowledge points.

[0023] In an optional embodiment,

[0024] According to the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time, and combined with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability, including:

[0025] The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, the saliency map is obtained by Gaussian filtering and inverse Fourier transform, and the saliency map is thresholded to generate a salient area mask;

[0026] In combination with the current cognitive load level of the driver, the information entropy of each information unit in the salient area mask is determined, and the information density of the salient area mask is dynamically adjusted using an information entropy minimization algorithm to match the real-time cognitive ability of the driver.

[0027] In an optional embodiment,

[0028] The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, and the saliency map is obtained by Gaussian filtering and inverse Fourier transform. The thresholding of the saliency map to generate a salient area mask includes:

[0029] Convert the color image of the training content to Lab color space and extract the L channel as the brightness image;

[0030] Performing a two-dimensional fast Fourier transform on the brightness image to obtain a frequency domain representation, and decomposing the frequency domain representation into an amplitude spectrum and a phase spectrum;

[0031] Calculating a residual spectrum of the amplitude spectrum, wherein the residual spectrum is obtained by subtracting a local average value of the amplitude spectrum after Gaussian filtering from the amplitude spectrum;

[0032] Recombining the residual spectrum and the phase spectrum into a frequency domain representation and performing a two-dimensional fast inverse Fourier transform to obtain a saliency map;

[0033] Normalizing, Gaussian filtering and thresholding the saliency map to generate a salient area mask;

[0034] Performing a logical AND operation on the salient area mask and the visual attention distribution data to find out the salient area where attention is insufficiently allocated, and obtaining an attention-deficient area mask;

[0035] According to the attention-deficit area mask, visual enhancement processing is performed on the area corresponding to the image in the training content to obtain dynamically optimized training content.

[0036] In an optional embodiment,

[0037] In combination with the current cognitive load level of the driver, determining the information entropy of each information unit in the salient area mask, and dynamically adjusting the information density of the salient area mask using an information entropy minimization algorithm includes:

[0038] Performing semantic segmentation on the salient region mask to obtain a set of information units, each of which is represented by a binary mask having the same size as the salient region mask;

[0039] Calculate the information entropy of each information unit, the information entropy is obtained by counting the probability distribution of pixels with different gray levels in the mask corresponding to the information unit, and the information entropy measures the amount of information contained in the information unit;

[0040] Taking the minimum sum of information entropy of all information units as the optimization goal and the constraint condition that the current cognitive load level of the driver does not exceed the preset constraint threshold as the constraint condition, the differential evolution algorithm is used to search for the optimal information presentation rhythm parameters, which include the display duration, display order and fade-in and fade-out effect of each information unit;

[0041] According to the optimized information presentation rhythm parameters, the information density of the salient region mask is dynamically adjusted.

[0042] According to a second aspect of the embodiments of the present invention,

[0043] Provide driving training human-computer interaction system for vehicle HUD, including:

[0044] The first unit is used to obtain the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; using a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, respectively based on the physiological characteristic data and the behavioral characteristic data for real-time prediction and recognition, to obtain the driver's current cognitive load level and driving behavior pattern;

[0045] The second unit is used to construct a driver capability profile based on the driver's historical training data using a collaborative filtering algorithm to determine the driver's mastery of different knowledge points; extract training knowledge points that match the driver's capability profile from the preset training knowledge base, use a reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form adaptive training content;

[0046] The third unit is used to enhance the adaptive training content in real time according to the driver's visual attention distribution data using the visual saliency detection algorithm, and to dynamically adjust the information density in the adaptive training content using the information entropy minimization algorithm in combination with the driver's current cognitive load level to match the driver's real-time cognitive ability.

[0047] According to a third aspect of the embodiments of the present invention,

[0048] An electronic device is provided, comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0052] A fourth aspect of the embodiments of the present invention is:

[0053] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0054] The real-time driver status monitoring module in this paper integrates multi-source heterogeneous data and dynamically evaluates the driver's status from the two dimensions of physiology and behavior. It can accurately predict the driver's cognitive load level and identify the driver's behavior pattern. This module uses advanced deep learning algorithms and introduces innovative designs such as attention mechanisms and dual-stream architectures to mine effective features in data in both time and space dimensions, making full use of the dynamic and correlation nature of the data and improving the accuracy of prediction and recognition.

[0055] The adaptive training content generation module integrates a variety of artificial intelligence technologies. Under the guidance of teaching theory, it achieves a dual match between training difficulty and trainee ability and real-time status, forming a personalized training model that teaches students in accordance with their aptitude and dynamically adjusts. After continuous iterative optimization, the module has initially acquired engineering application capabilities and can significantly improve the pertinence and effectiveness of driver training.

[0056] Dynamically adjusting the difficulty of training content is the core of adaptive learning. By integrating artificial intelligence methods such as BKT and fuzzy control, the system can comprehensively evaluate the driver's knowledge level and real-time cognitive state, and flexibly adjust the presentation method and difficulty gradient of knowledge points accordingly, truly teaching students in accordance with their aptitude and optimizing in real time, so that drivers are always immersed in the most appropriate learning area. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a process flow of a human-computer interaction method for driving training applied to a vehicle HUD according to an embodiment of the present invention;

[0058] Figure 2 The figure is a schematic diagram of the structure of a human-computer interaction system for driving training applied to a vehicle HUD according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0061] Figure 1 FIG. 1 is a flow chart of a human-computer interaction method for driving training applied to a vehicle HUD according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] S101. Acquire the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; use a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, and perform real-time prediction and recognition based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern;

[0063] S102. Based on the driver's historical training data, a collaborative filtering algorithm is used to construct a driver's ability profile and determine the driver's mastery of different knowledge points; training knowledge points that match the driver's ability profile are extracted from the preset training knowledge base, and the training knowledge points are sorted by difficulty using a reinforcement learning algorithm, and combined with the driver's current cognitive load level, an adaptive training content is finally formed;

[0064] S103. Based on the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time. In combination with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability.

[0065] In an optional embodiment,

[0066] A driver cognitive load level prediction model and a driving behavior pattern recognition model are established using a deep neural network, and real-time prediction and recognition are performed based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern, including:

[0067] The preprocessed physiological characteristic data are input into the cognitive load level prediction model based on the long short-term memory neural network. The temporal characteristics of the physiological characteristic data are extracted through the multi-layer long short-term memory neural network, and the attention mechanism is introduced to adaptively allocate the weights corresponding to the temporal characteristics at different moments. After the fully connected layer and activation function, the predicted value of the cognitive load level is obtained.

[0068] The behavior feature data are preprocessed, statistical values ​​of the behavior feature data are extracted, and a behavior heat map of the behavior feature data is constructed; the behavior feature data and the behavior heat map are input into a driving behavior pattern recognition model jointly constructed based on a convolutional neural network and a long short-term memory neural network for feature extraction and feature fusion to obtain fusion features; multi-classification is performed through the fully connected layer and the classification layer of the driving behavior pattern recognition model to obtain the probability distribution of the driving behavior pattern in each time window, and the driving behavior pattern with the highest probability distribution value is used as the driving behavior pattern of the current time window.

[0069] For example, in the intelligent driver training system, real-time monitoring of the driver's cognitive state and behavior pattern is crucial. This paper develops a real-time driver status monitoring module based on deep neural network. By modeling and analyzing the driver's physiological characteristics data and behavioral characteristics data, it dynamically predicts the driver's cognitive load level and identifies the driver's behavior pattern, providing a basis for subsequent training content adjustment and human-computer interaction optimization.

[0070] Cognitive load is an indicator of the total amount of cognitive resources a driver devotes to the driving task. A reasonable level of cognitive load helps maintain concentration and alertness, while too high or too low a cognitive load will affect driving performance. Therefore, accurately predicting the driver's cognitive load level is of great significance for optimizing the training process.

[0071] This module uses the long short-term memory (LSTM) neural network to build a cognitive load level prediction model. LSTM is a recurrent neural network that is particularly suitable for processing time series data and can capture long-term dependencies in the data. The input of the model is the driver's multi-channel physiological signal data, including electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG) and skin conductance (GSR). These physiological signals contain rich cognitive and emotional state information and can serve as an important basis for predicting cognitive load levels.

[0072] First, the collected raw physiological signal data is preprocessed. Power frequency interference and high-frequency noise are removed by bandpass filtering, and then baseline drift correction and normalization are performed to make the signal amplitude fall between 0 and 1. Then, the signal data is segmented with a fixed time window (such as 5 seconds), and the time domain, frequency domain, and time-frequency domain features in each time window, such as mean, variance, power spectral density, etc., are extracted to form a feature vector sequence.

[0073] The preprocessed feature vector sequence is input into the LSTM model. The basic LSTM unit contains three gating mechanisms: input gate, forget gate, and output gate, which can selectively store and update long and short-term memory. The model adopts a multi-layer LSTM stacked structure. The first layer of LSTM receives the feature vector sequence and extracts the underlying time series features. The subsequent layers of LSTM further extract high-level abstract features. At each time step, the LSTM unit processes the current moment input and combines the hidden state of the previous moment, updates the memory unit and outputs a new hidden state. The output vector sequence of the top LSTM contains the information in the entire time window.

[0074] In order to enhance the representation ability of the model, the attention mechanism is introduced after the LSTM layer. The traditional LSTM assigns equal importance to the hidden state of each time step, while the attention mechanism can adaptively evaluate the importance of features at different moments and assign higher weights to significant features. Specifically, the additive attention alignment function is used to calculate the attention weight of each item in the LSTM output vector sequence. The larger the weight, the more important the feature at that moment. Then, the LSTM output vector sequence is weighted summed using the attention weight to obtain the aggregated feature representation.

[0075] Finally, the aggregated features are passed through a fully connected layer and a Sigmoid activation function to output a prediction value between 0 and 1, indicating the driver’s current cognitive load level. For example, if the prediction value is 0.75, it means that the cognitive load level is at a high level.

[0076] In order to train and evaluate the model, it is necessary to construct a labeled cognitive load level dataset. By designing driving tasks of different difficulty and complexity (such as lane keeping, obstacle avoidance, traffic sign recognition, etc.), the physiological signals and subjective load scores (such as NASA-TLX scale) of the driver when performing these tasks are recorded simultaneously, and the continuous subjective scores are discretized into three levels: high, medium, and low. The collected physiological signal data and the corresponding load level labels are randomly divided into training and test sets in a ratio of 8:2.

[0077] The model was trained using the Adam optimization algorithm and the cross entropy loss function, with the batch size set to 64 and the initial learning rate set to 0.001. The model performance was evaluated on the validation set every 10 epochs, and the early stopping mechanism was triggered when it did not improve for five consecutive times. The optimal model was evaluated on the test set, and the classification accuracy was over 85%.

[0078] Here is an application example. A driver performs a lane change task on the simulator for 2 minutes. The system collects multi-channel physiological signal data during this period, and the sampling rate of each channel is 1000Hz. The data is divided into 24 time windows of 5 seconds, and 10 features are extracted in each time window. The shape of the feature vector sequence input to LSTM is (24,10). After 3 layers of LSTM and attention pooling, the output prediction value sequence is [0.45, 0.55, 0.62,...]. This sequence depicts the dynamic change trajectory of the driver's cognitive load level during the task. It can be seen that at the critical moment of lane change, the driver's cognitive load level increases significantly.

[0079] Driving behavior patterns refer to the stable operation and decision-making patterns that drivers display when completing a certain driving task. Common driving behavior patterns can be divided into different categories, such as standard, aggressive, conservative, and dangerous. Timely identification of driver behavior patterns helps to discover their bad driving habits and provide targeted guidance.

[0080] This module uses a combination of convolutional neural networks (CNN) and LSTM to identify driving behavior patterns. CNN is mainly used to extract spatial features from image data, and LSTM is used to extract temporal features from time series data. The input of the model is the behavior feature data, including the time series records of accelerator, brake, steering and other operation signals; the second is the behavior heat map, which is generated based on vehicle trajectory, speed and other information, and intuitively represents the driver's spatial driving mode in the road network.

[0081] For the preprocessing of behavioral feature data, similar to the cognitive load level prediction, it is segmented into fixed time windows, and statistical features such as mean, standard deviation, and change rate are extracted in each segment. The steps for generating the behavioral heat map are as follows:

[0082] Record the vehicle's GPS trajectory, speed, acceleration and other driving state variables within a period of time (such as 2 minutes).

[0083] The vehicle trajectory is mapped onto a gridded road network map, with each grid side length representing 5 meters.

[0084] The cumulative residence time and average speed of vehicles in each grid are counted to generate residence time heat map and average speed heat map respectively.

[0085] The two heat maps are element-wise multiplied to obtain a behavior heat map that takes time and speed into account. A higher value means that the driver has accumulated more time in the area or traveled at a higher speed.

[0086] The multi-dimensional behavior feature sequence and behavior heat map obtained by preprocessing are input into LSTM and CNN respectively. In order to fully exploit the spatiotemporal features, the model adopts a dual-stream architecture: the spatial stream takes the behavior heat map as input and extracts the spatial attention feature map through a three-layer CNN; the temporal stream takes the behavior feature sequence as input and extracts the temporal attention feature through a three-layer LSTM. Then, the two features are spliced ​​in the third dimension to form a unified spatiotemporal feature representation.

[0087] Next, the spatiotemporal feature representation is fed to two fully connected layers for nonlinear transformation and mapped to the probability distribution of driving behavior patterns through the Softmax activation function. The category with the highest probability value is used as the driving behavior pattern prediction result for the current time window. For example, if the probabilities of the four modes of standard, aggressive, conservative, and dangerous are 0.1, 0.7, 0.1, and 0.1 respectively, the driving behavior pattern of this time window is identified as aggressive.

[0088] The training and evaluation process of the model is similar to the cognitive load level prediction. First, multiple scenarios are designed in the simulator, including lane keeping, vehicle following, intersection passing, etc., and specific rules for standard and aggressive behavior modes are defined, such as speed threshold, vehicle distance threshold, number of violations, etc. Then, people with different driving styles are recruited for data collection, their driving process data is recorded, and the real behavior mode is marked according to the rules. Finally, the training and test sets are divided into 8:2 for model training.

[0089] Take the data of a user driving for 3 minutes as an example. A total of 18,000 frames of images and 10,800 sensor data points were collected, divided into 36 time windows of 5 seconds. The generated 36 behavior heat maps have a resolution of 100x100 and are input into the model together with 36 multi-dimensional behavior feature sequences. The output behavior pattern probability distribution sequence is [[0.05, 0.85, 0.05, 0.05], [0.7, 0.1, 0.1, 0.1], ...]. The results show that the user exhibits an obvious aggressive driving mode in the initial stage, and gradually transitions to a standard driving mode in the middle stage.

[0090] In summary, the real-time driver status monitoring module in this paper integrates multi-source heterogeneous data, dynamically evaluates the driver's status from the two dimensions of physiology and behavior, and can accurately predict the driver's cognitive load level and identify the driver's behavior pattern. This module adopts advanced deep learning algorithms, and through the introduction of innovative designs such as attention mechanism and dual-stream architecture, it mines effective features in the data in the two dimensions of time and space, fully utilizes the dynamic and correlation of the data, and improves the accuracy of prediction and recognition. Experiments show that the driver status monitoring module can achieve three-classification of cognitive load level and four-classification of driving behavior pattern with an accuracy rate of more than 85%, showing good performance.

[0091] In an optional embodiment,

[0092] Extract training knowledge points that match the driver's ability profile from the preset training knowledge base, use the reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form an adaptive training content including:

[0093] Build a training knowledge base covering driving training, classify and label the knowledge points in the training knowledge base, and the attributes of the knowledge points include the category, difficulty level and prerequisite knowledge points; use the historical training data of drivers, explore the similarities between different drivers in the mastery of knowledge points, use collaborative filtering algorithms to predict the current driver's mastery of unlearned knowledge points, and determine the set of knowledge points that match the driver's ability;

[0094] Based on the knowledge point set that matches the driver's ability, a reinforcement learning algorithm is used to sort the knowledge point. The knowledge point sorting problem is modeled as a Markov decision process, where the state is the completed knowledge point sequence, the action is to select the next knowledge point, and the reward comprehensively considers the difficulty of the knowledge point, the matching degree of the driver's ability, and the dependency of the knowledge points. Through continuous learning, the optimal sorting strategy that maximizes the cumulative reward is found.

[0095] Based on the comparison between the driver's current cognitive load level and the preset reasonable cognitive load range, the difficulty of the training content is dynamically adjusted to form adaptive training content.

[0096] For example, traditional driver training mostly adopts a "one-size-fits-all" approach, providing the same training content to everyone, ignoring the differences in ability and experience between individual drivers, making it difficult to achieve the effect of teaching students in accordance with their aptitude. In order to solve this problem, this paper develops an adaptive training content generation module, which customizes personalized training plans based on the driver's ability characteristics and real-time cognitive status, and achieves a dynamic match between training difficulty and driver level.

[0097] Building a comprehensive and detailed training knowledge base is the basis for achieving adaptive training. This module refers to the driving teaching syllabus and consults with senior coaches to carefully design a knowledge point system covering all aspects of driving training. Knowledge points are organized into different categories according to content similarity, such as basic control, traffic regulations, and dangerous situation handling, and the initial difficulty level is assigned according to the teaching sequence and cognitive difficulty. In addition, the dependency relationship between each knowledge point is also marked, that is, which knowledge points need to be mastered in advance to learn a certain knowledge point.

[0098] For example, the knowledge point "curved driving" belongs to the "basic control" category, with a difficulty level of 3 (elementary), and the prerequisite knowledge points include "throttle control" (difficulty 1) and "steering control" (difficulty 2). After systematic sorting, the training knowledge base finally constructed contains 500 atomic knowledge points, belonging to 12 categories, with difficulty levels covering 1 to 5, and an average of 2 prerequisite knowledge points for each knowledge point.

[0099] In order to recommend training content that matches the driver's ability level, we first need to accurately evaluate the driver's mastery of different knowledge points. Inspired by "collaborative filtering", we assume that different drivers have certain similarities in their mastery of knowledge points, that is, people with similar abilities have similar mastery of the same knowledge point. Based on this, we can use the historical training data of all drivers to mine the commonalities between drivers and predict the current driver's mastery of unlearned knowledge points. The specific implementation steps are as follows:

[0100] Collect the historical training data of all drivers participating in the training and construct a "driver-knowledge point" bipartite graph. If a driver has learned a certain knowledge point, record his or her mastery score (1 to 5 points) in the corresponding position, otherwise leave it blank.

[0101] Calculate the similarity between any two drivers. The similarity considers the difference in the scores of the two drivers for all the knowledge points they have learned together. The smaller the difference in scores, the higher the similarity. Common similarity metrics include Euclidean distance, Pearson correlation coefficient, etc.

[0102] For the current driver, find the K most similar drivers as "neighbors". Here K=10.

[0103] For each unlearned knowledge point of the current driver, predict the mastery level. The predicted value is equal to the weighted average of the ratings of the neighbors who have learned the knowledge point among the K most similar neighbors of the driver, and the weight is the similarity of the neighbors. The predicted value P can be expressed as P=sum(sim_i * rating_i) / sum(sim_i), where sim_i is the similarity of the i-th most similar neighbor, and rating_i is the rating of the i-th neighbor for the knowledge point.

[0104] All knowledge points are sorted according to the predicted value, and the knowledge points with high predicted values ​​are more suitable for the current driver's ability. The knowledge points with predicted values ​​greater than 3.5 points are selected as the candidate set, indicating that the driver is capable of learning these knowledge points.

[0105] Let's take an intuitive example. Assume that the current driver Xiao Ming has learned knowledge point A (score 4) and knowledge point B (score 3), and now we need to predict his mastery of knowledge point C. Through calculation, we find the three most similar neighbors to Xiao Ming: Xiao Hong (similarity 0.8), Xiao Gang (similarity 0.6), and Xiao Li (similarity 0.4). Among them, Xiao Hong and Xiao Gang have learned knowledge point C, and their scores are 4 and 5 respectively. Then Xiao Ming's prediction value for knowledge point C is (0.8*4+0.6*5) / (0.8+0.6)=4.43. It can be seen that although Xiao Ming himself has not learned knowledge point C, by analyzing the learning situation of Xiao Hong and Xiao Gang who are similar to him, the system predicts that Xiao Ming has a high probability of mastering knowledge point C.

[0106] After ability profile matching, about 100 knowledge points that match the current driver's abilities were selected from the knowledge base, providing a more personalized candidate pool for subsequent difficulty ranking.

[0107] After the set of knowledge points that match the driver's ability is determined, it is necessary to further arrange the learning order of the knowledge points. The ideal learning path should be gradual, the difficulty of the previous and subsequent knowledge points should change smoothly, the dependency between knowledge points should be considered, and there should be appropriate ups and downs to prevent the learning process from being boring. This is actually a combinatorial optimization problem. This module uses the Q-learning algorithm in reinforcement learning to find the optimal sorting strategy.

[0108] First, the knowledge point sorting problem is modeled as a Markov decision process:

[0109] State: A vector of the form (kp_1, kp_2, ..., kp_t), representing the sequence of knowledge points that have been learned at time t. The initial state is an empty sequence ().

[0110] Action: Select the next knowledge point kp to be learned. The action space in each state is different, and you must choose from the candidate knowledge points that have not yet been scheduled for learning.

[0111] State transition: From state (kp_1, kp_2, ..., kp_t) to (kp_1, kp_2, ..., kp_t, kp), it means adding knowledge point kp after the sequence has been completed. When the set of candidate knowledge points is empty, it enters the terminal state.

[0112] Reward: The immediate reward obtained after taking an action. When designing a reward function, the following points should be taken into consideration:

[0113] Moderate difficulty: The difficulty of the selected knowledge points matches the driver's ability. Knowledge points with a predicted difficulty value between 3.5 and 4.5 will receive higher rewards. Smooth difficulty change: The difficulty change from the previous knowledge point should not be too large. Jumping within 1 difficulty level will receive higher rewards. Pay attention to dependencies: Prioritizing the knowledge points that require prerequisites will receive higher rewards. Diverse content: Appropriately interspersing knowledge points of different categories will receive higher rewards.

[0114] Under the MDP framework, the knowledge point sorting problem is transformed into the problem of finding the optimal strategy, that is, selecting the action a that maximizes Q(s,a) in each state so that the cumulative reward obtained from this state to the terminal state is maximized.

[0115] The Q-learning algorithm updates the Q value table through continuous trial and error, and eventually converges to the optimal Q value. At the beginning of training, the Q value table is randomly initialized. Then it starts to iterate round by round. Each round generates a complete state-action sequence according to the current strategy, and calculates the temporal difference (TD) error of each step, and updates the Q value according to the TD error. The new Q value is a more accurate estimate of the cumulative reward. After multiple rounds of iteration, the Q value will converge, and the greedy strategy at this time is the optimal strategy.

[0116] It should be pointed out here that Q-learning is a model-free, different-strategy, and offline-updated temporal difference learning method. Compared with model-based planning methods such as value iteration and policy iteration, Q-learning does not require prior knowledge of state transition probabilities and can directly estimate the optimal action value function through sample data, making it more suitable for complex environments such as knowledge point sorting. The introduction of different strategies and offline updates is conducive to making full use of historical experience data and improving training efficiency.

[0117] After sorting the difficulty of knowledge points, the current driver has obtained a personalized training program that better matches his ability level. However, in the actual training process, the driver's cognitive state changes dynamically, and the constant content difficulty cannot guarantee that he is always at the appropriate cognitive load level. In order to further improve learning efficiency, it is necessary to fine-tune the content difficulty according to the driver's real-time cognitive load.

[0118] Educational psychology research shows that when learners' cognitive load is at a moderate level, learning is most effective. When cognitive load is too low, learners are easily distracted and lose focus; while too high cognitive load can cause anxiety and fatigue, making it difficult to deeply understand and remember. Therefore, the ideal cognitive load range is roughly between 0.4 and 0.7.

[0119] The driver status monitoring module mentioned above can predict the current driver's cognitive load level (0~1) in real time. Compare the predicted value with the ideal range (0.4~0.7). If it is lower than 0.4, it means that the current content is relatively simple and the difficulty should be increased; if it is higher than 0.7, it means that the content is relatively difficult and the difficulty should be reduced. The adjustment range is positively correlated with the degree of deviation. The greater the deviation, the more adjustments. For example, when a driver is learning a knowledge point of difficulty level 3, the average cognitive load for 3 consecutive minutes is 0.85, which is higher than 0.7, and the degree of deviation is about 0.15, which belongs to the second level. The system automatically adjusts the difficulty level of the next knowledge point from 4 to 3 to ensure that the cognitive load returns to a reasonable range.

[0120] The following takes the training process of a novice driver Xiao Zhang as an example to demonstrate the workflow of the adaptive training content generation module.

[0121] Xiao Zhang has some driving experience and is relatively proficient in basic driving, but lacks actual road driving experience. He recently completed the study of 10 knowledge points, including "throttle control" (score 4), "speed control" (score 4), "straight driving" (score 4), and "lane change" (score 3). The system first matches 50 candidate knowledge points from the knowledge base based on Xiao Zhang's historical performance, and the difficulty level is mostly level 3~4.

[0122] Next, the Q-learning algorithm was used to sort the candidate set and obtain a personalized learning path. Xiao Zhang began to train according to the recommended path. The learning process of the first three knowledge points was normal, and the cognitive load was maintained at around 0.6. However, when learning the fourth knowledge point "active avoidance" (difficulty 3), Xiao Zhang made several mistakes in succession, and the cognitive load rose sharply to 0.9. The system detected this change in time and lowered the difficulty of the next knowledge point "passing the intersection" from 4 to 2, and inserted two reinforcement exercises related to "active avoidance". The adjusted content is obviously more suitable for Xiao Zhang's current state. His cognitive load gradually recovered to 0.65, and he successfully completed this training.

[0123] In summary, the adaptive training content generation module integrates a variety of artificial intelligence technologies. Under the guidance of teaching theory, it achieves a dual match between training difficulty and trainee ability and real-time status, forming a personalized training model that teaches students in accordance with their aptitude and dynamically adjusts. After continuous iterative optimization, the module has initially acquired engineering application capabilities, which can significantly improve the pertinence and effectiveness of driver training.

[0124] In an optional embodiment,

[0125] The method also includes dynamically adjusting the difficulty of the training content:

[0126] The Bayesian knowledge tracking model is used to track the probability of drivers mastering each knowledge point in the training content in real time, and dynamically adjust the presentation frequency and presentation time of each knowledge point. The presentation frequency and presentation time of knowledge points with a mastery probability higher than the preset threshold are reduced, and the presentation frequency and presentation time of knowledge points with a mastery probability lower than the preset threshold are increased.

[0127] According to the driver's current cognitive load level, the fuzzy control algorithm is used to dynamically adjust the difficulty gradient of the knowledge points.

[0128] For example, in the adaptive training process, timely tracking the driver's mastery of each knowledge point and dynamically adjusting the presentation strategy and difficulty gradient of the knowledge point accordingly are key measures to promote efficient learning of drivers and reduce cognitive load. This application is based on Bayesian knowledge tracking and fuzzy control theory and elaborates on the technical solution to achieve the above functions.

[0129] Bayesian Knowledge Tracing (BKT) is a student modeling method widely used in the field of smart education. It uses a hidden Markov model to describe the dynamic changes of students' knowledge status and infers whether students have truly mastered a certain knowledge point by continuously observing their answers to the knowledge point. This module uses BKT to estimate the probability of the driver mastering each knowledge point in real time.

[0130] Specifically, the mastery of each knowledge point is simplified into a binary state: mastered (L) or not mastered (N). At the same time, the driver's answers to the knowledge points are also divided into two categories: correct (C) or wrong (W). The hidden Markov model is constructed, and the state transition and observation probability are as follows:

[0131] State transfer:

[0132] P(L->L) = 1, that is, the knowledge points that have been mastered will not be forgotten;

[0133] P(N->L) = P(T), that is, the probability that a knowledge point is actually learned in each practice, is called the learning probability;

[0134] Observation probability:

[0135] P(C|L) = 1-P(S), that is, after mastering the knowledge point, one may still make mistakes, and the probability of making a mistake is P(S), which is called the error probability;

[0136] P(C|N) = P(G), that is, it is possible to guess the correct answer even if you have not mastered the knowledge point. The probability of guessing the correct answer is P(G), which is called the guessing probability.

[0137] Among them, the learning probability P(T), the error probability P(S), and the guessing probability P(G) are prior parameters and can be set based on historical data or expert experience.

[0138] During the training process, the driver's answers to each knowledge point are recorded to form an observation sequence. The forward algorithm is used to recursively calculate the posterior probability that the driver is in the mastered state after each answer, that is, the mastery probability:

[0139] P(L_n|O_1, O_2, ..., O_n);

[0140] Among them, L_n represents the mastery status after the nth answer, and O_i represents the observation result (C or W) of the ith answer.

[0141] Taking the knowledge point "Lane Change" as an example, assuming that the prior parameters P(T)=0.2, P(S)=0.1, P(G)=0.2, and a driver's answer sequence is (C, W, W, C, C), the driver's mastery probability changes as follows:

[0142] The number of answers determines the probability; 0 0.00; 1 0.31; 2 0.14; 3 0.05; 4 0.21; 5 0.59;

[0149] It can be seen that errors accounted for the majority of the first three answers, and the probability of mastering the question showed a downward trend; starting from the fourth time, the correct answers were given continuously, and the probability of mastering the question gradually increased, eventually reaching 0.59.

[0150] To simplify the calculation, a threshold of the mastering probability can be set, such as 0.6. If the mastering probability exceeds 0.6 after multiple consecutive answers, it is determined that the driver has basically mastered this knowledge point, otherwise it is determined that the driver has not mastered it.

[0151] According to the real-time mastery probability of each knowledge point by the driver, the presentation strategy of the knowledge point can be dynamically adjusted to control the difficulty from two dimensions: time and frequency. For knowledge points with a mastery probability higher than a threshold (such as 0.6), it indicates that the driver has mastered them. In order to consolidate knowledge and prevent forgetting, the presentation frequency and duration can be appropriately reduced in subsequent training. The adjustment principle is:

[0152] The presentation frequency is reduced to 60%~80% of the original, that is, the knowledge point appears 6~8 times in every 10 exercises; the single presentation time is shortened by 20%~40%, that is, if it originally took an average of 5 minutes to practice this knowledge point each time, it will be adjusted to 3~4 minutes.

[0153] For knowledge points with a probability of mastering lower than the threshold, the driver has not yet fully mastered them and needs repeated intensive practice. Therefore, the presentation frequency and duration should be increased: the presentation frequency is increased to 120%~150% of the original, that is, the knowledge point appears 12~15 times in every 10 exercises; the single presentation duration is extended by 20%~50%, that is, if the original average practice of the knowledge point takes 5 minutes, it will be adjusted to 6~7.5 minutes.

[0154] By mastering the dynamic adjustment of probability drive, it is possible to highlight key knowledge and weaken non-key knowledge, allowing drivers to devote more time and energy to weak links, thereby teaching students in accordance with their aptitude and improving training efficiency.

[0155] In addition to adjusting the presentation strategy of knowledge points, it is also necessary to take into account the actual cognitive load of the driver and dynamically adjust the overall difficulty gradient of the knowledge points. Traditional difficulty adjustment mostly adopts deterministic rules and lacks flexibility. Considering the fuzziness and continuity of cognitive load, this module introduces fuzzy control theory to achieve flexible adjustment of difficulty.

[0156] First, the two continuous variables of cognitive load and difficulty adjustment amplitude were fuzzified. Cognitive load was divided into five levels: very low (VL), low (L), medium (M), high (H), and very high (VH). Similarly, the difficulty adjustment amplitude was divided into five levels: greatly reduced (DB), slightly reduced (DS), unchanged (M), slightly increased (IS), and greatly increased (IB).

[0157] For example, assuming that the current driver's cognitive load prediction value is 65%, the corresponding membership vector is (0, 0.25, 0.75, 0, 0), which matches rules 3 and 4. Through fuzzy reasoning (taking the smaller value) and defuzzification (center of gravity method), the difficulty adjustment range is calculated to be -5%. This means that the system will select new knowledge points that are about 5% less difficult than the current knowledge points, reduce the overall difficulty, and relieve the driver's cognitive pressure.

[0158] The advantage of the fuzzy control method is that the adjustment decision is more continuous and smooth, avoiding the "one-size-fits-all" of traditional rules, and can fully adapt to the driver's changing cognitive state. At the same time, the rule base can be modified and expanded according to the actual teaching situation, and has strong domain adaptability and interpretability.

[0159] After Xiao Zhang enters the "active avoidance" phase, the system starts running the dynamic difficulty adjustment strategy:

[0160] First, based on Xiao Zhang's incorrect operations on "active avoidance" in the previous exercises, the BKT model estimated that his mastery probability was only 0.3, far below the threshold of 0.6. Therefore, it decided to significantly increase the presentation frequency (150%) and single duration (50%) of subsequent exercises on this knowledge point until the mastery probability exceeds 0.6.

[0161] At the same time, difficulty monitoring found that Xiao Zhang's average cognitive load had reached 80%, corresponding to the membership vector (0, 0, 0.4, 0.6, 0), which mainly triggered Rules 4 and 5. After fuzzy reasoning, the system lowered the overall difficulty of the next stage of training content by 15% to prevent cognitive overload. Under the dual regulation of BKT and fuzzy control, Xiao Zhang quickly adapted to the difficulty of "active avoidance", and the probability of mastering it increased to more than 0.7, successfully completing the learning of this key knowledge point.

[0162] In summary, dynamically adjusting the difficulty of training content is the core link of adaptive learning. By integrating artificial intelligence methods such as BKT and fuzzy control, the system can comprehensively evaluate the driver's knowledge level and real-time cognitive state, and flexibly adjust the presentation method and difficulty gradient of knowledge points accordingly, truly teaching students in accordance with their aptitude and optimizing in real time, so that drivers are always immersed in the most appropriate learning area.

[0163] In an optional embodiment,

[0164] According to the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time, and combined with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability, including:

[0165] The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, the saliency map is obtained by Gaussian filtering and inverse Fourier transform, and the saliency map is thresholded to generate a salient area mask;

[0166] In combination with the current cognitive load level of the driver, the information entropy of each information unit in the salient area mask is determined, and the information density of the salient area mask is dynamically adjusted using an information entropy minimization algorithm to match the real-time cognitive ability of the driver.

[0167] For example, in the adaptive training process, in order to maximize the information acquisition efficiency of the driver, the training content needs to be optimized in real time according to their cognitive characteristics. This application focuses on two key factors: visual attention distribution and cognitive load level, and elaborates on the content enhancement method based on visual saliency detection and information entropy minimization.

[0168] Visual saliency detection aims to automatically extract the most eye-catching areas in an image or video, which usually have obvious visual differences from the surrounding environment, such as color, brightness, texture, etc. Introducing saliency analysis into adaptive training can predict the driver's attention distribution and highlight important information in a targeted manner. This module uses the spectral residual method for saliency detection.

[0169] The basic principle of the spectral residual method is that the salient area is represented by components with higher energy than the background in the frequency domain, and the residual spectrum obtained after removing these high-frequency components can reflect the saliency. The algorithm flow is as follows:

[0170] Input a frame of training video image, grayscale it, and retain the brightness information. Perform two-dimensional Fourier transform on the grayscale image to get the spectrum; decompose the spectrum into two parts: amplitude spectrum and phase spectrum. The amplitude spectrum represents the intensity of different frequency components, and the phase spectrum represents the phase information of different frequency components.

[0171] The local average of the amplitude spectrum is used to replace the original amplitude spectrum to obtain the residual spectrum. Specifically, a normalized Gaussian filter is used to convolve the amplitude spectrum, and the convolution result is used as the local average amplitude spectrum, and then the average spectrum is subtracted from the original amplitude spectrum to obtain the residual spectrum.

[0172] Reconstruct a new spectrum using the residual spectrum and the phase spectrum, where the real part is obtained by multiplying the cosine value of the residual spectrum and the phase spectrum, and the imaginary part is obtained by multiplying the sine value of the residual spectrum and the phase spectrum. Perform inverse Fourier transform on the reconstructed spectrum to obtain a visual saliency map. Gaussian filtering is performed on the visual saliency map to suppress high-frequency noise; the filtered saliency map is binarized. Set a saliency threshold, set pixels greater than or equal to the threshold to 1, indicating a salient area; set pixels less than the threshold to 0, indicating a background area. The selection of the threshold can be adjusted based on experience or experiments.

[0173] The final binary image is called a saliency mask map, in which the area with a value of 1 outlines the salient targets in the original image and provides a reference for subsequent information density adjustment.

[0174] For example, suppose a frame of 800×600 driving scene image is input, which contains multiple targets such as roads, road signs, and pedestrians. After being processed by the spectral residual method, the average saliency value of the road area in the obtained mask map is 0.2, the road sign area is 0.8, and the pedestrian area is 0.6. It can be seen that the road area has the lowest saliency, while the road signs and pedestrians have higher saliency, which is basically consistent with the actual visual preference of the driver. Based on this mask map, the visual enhancement strength of different areas can be further adjusted.

[0175] After obtaining the saliency distribution, it is necessary to consider how to control the information density of each salient area to adapt to the driver's real-time cognitive load. Too high information density is likely to cause cognitive overload, while too low information density may cause the driver to lose focus. Therefore, this application introduces the information entropy metric and combines it with the cognitive load level to achieve dynamic adjustment of information density.

[0176] First, the visual saliency map is divided into several regions of interest (ROIs), and the average saliency of the pixels in each ROI is calculated. Specifically, the pixel values ​​in each ROI are summed up and then divided by the total number of pixels in the region to obtain the average saliency value of the ROI.

[0177] Next, the information density of each ROI is defined as the amount of information contained in the region divided by its average saliency value. Here, the amount of information in the ROI can be roughly represented by the number of pixels in the original image it covers. The larger the information density value, the more information is contained under the unit saliency, and the driver needs to invest more attention resources to process the area.

[0178] To avoid overload, it is necessary to appropriately reduce the information density when the driver's cognitive load increases. Here, the information entropy minimization criterion is used to quantify the adjustment range of density. Specifically, the driver's cognitive load level is divided into several levels, and the higher the level, the greater the load. At the same time, considering the dynamic range of information density, an upper and lower bound of density is set.

[0179] Then, under the constraints of the upper and lower bounds of density, the optimal information density of each ROI after adjustment is solved by minimizing the overall information entropy. Using the Lagrange multiplier method, a closed-form solution for the optimal information density can be obtained, and then the adjustment coefficient of the density under different cognitive load levels can be determined.

[0180] Finally, the original ROI information volume is multiplied by the adjustment coefficient to obtain the adjusted information volume. After this adjustment process, the information density of each ROI is dynamically adapted to the driver's real-time cognitive level, while ensuring that the key points are highlighted and avoiding information overload or information shortage.

[0181] Assume that the saliency mask map is divided into three ROIs: road, road sign, and pedestrian. According to the average saliency statistics of each ROI, their original information density is calculated. The results show that the information density of the road area is the highest, followed by the road sign, and the pedestrian area is the lowest.

[0182] Assuming that drivers have three cognitive load levels of low, medium and high, and taking the dynamic range of information density as [5e4, 2e6], the overall information entropy minimization problem is solved to obtain the adjustment coefficients of information density under three load levels. It can be seen that the higher the load level, the smaller the adjustment coefficient, and the more the original information is compressed.

[0183] Next, assuming that the current driver is under medium cognitive load, the corresponding adjustment coefficients are applied to the original information of the three ROIs, that is, their adjusted information is obtained. It can be found that the information in high-density areas such as roads is greatly reduced, while the changes in low-density areas such as pedestrians are relatively small, so that the overall information distribution tends to be balanced.

[0184] Finally, the adjusted ROI information volume is mapped to the specific image enhancement algorithm parameters to generate a visually enhanced training picture. By comparing the original picture and the enhanced picture under different cognitive load levels, it can be intuitively seen that: as the cognitive load increases, the system adaptively reduces the information density of the road background and highlights key targets such as pedestrians and vehicles, which not only ensures the targeted training but also prevents cognitive overload.

[0185] In summary, this application proposes a training content optimization method that integrates visual saliency and information entropy. This method is based on the driver's visual attention distribution and dynamically adjusts the information density of each salient area to adapt it to the cognitive load level. While improving the efficiency of key information recognition, it avoids the waste of cognitive resources and provides strong support for adaptive training.

[0186] In an optional embodiment,

[0187] The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, and the saliency map is obtained by Gaussian filtering and inverse Fourier transform. The thresholding of the saliency map to generate a salient area mask includes:

[0188] Convert the color image of the training content to Lab color space and extract the L channel as the brightness image;

[0189] Performing a two-dimensional fast Fourier transform on the brightness image to obtain a frequency domain representation, and decomposing the frequency domain representation into an amplitude spectrum and a phase spectrum;

[0190] Calculating a residual spectrum of the amplitude spectrum, wherein the residual spectrum is obtained by subtracting a local average value of the amplitude spectrum after Gaussian filtering from the amplitude spectrum;

[0191] Recombining the residual spectrum and the phase spectrum into a frequency domain representation and performing a two-dimensional fast inverse Fourier transform to obtain a saliency map;

[0192] Normalizing, Gaussian filtering and thresholding the saliency map to generate a salient area mask;

[0193] Performing a logical AND operation on the salient area mask and the visual attention distribution data to find out the salient area where attention is insufficiently allocated, and obtaining an attention-deficient area mask;

[0194] According to the attention-deficit area mask, visual enhancement processing is performed on the area corresponding to the image in the training content to obtain dynamically optimized training content.

[0195] For example, the present application proposes a method for using the spectral residual method to detect visual saliency of images in adaptive training content, and combining visual attention distribution data to dynamically enhance salient areas. The method extracts the brightness information of the image and analyzes its energy distribution characteristics in the frequency domain to find out the salient areas that attract visual attention, and then performs targeted visual enhancement processing on the salient areas where attention is insufficient, thereby guiding the driver's attention distribution and improving the recognition efficiency of key information.

[0196] The spectral residual method is a visual saliency detection method based on frequency domain analysis. Its basic principle is that the salient area is represented by high-energy components in the amplitude spectrum in the frequency domain. The residual spectrum is obtained by removing these high-energy components. The residual spectrum is subjected to inverse Fourier transform to obtain a saliency map representing saliency. This application uses the spectral residual method to perform saliency detection on the training content image. The specific steps are as follows:

[0197] The input color image is converted from RGB color space to Lab color space, and the L channel is extracted as the brightness image, denoted as I. The Lab color space consists of a brightness channel L and two chromaticity channels a and b, where the L channel represents the brightness information of the image, which is closer to the brightness perception of the human eye.

[0198] Perform a two-dimensional fast Fourier transform (2D-FFT) on the brightness image I to obtain the frequency domain representation F. Fast Fourier transform is an algorithm for efficiently calculating discrete Fourier transform, which can transform the image from the spatial domain to the frequency domain. Decompose the frequency domain representation F into an amplitude spectrum A and a phase spectrum P. The amplitude spectrum represents the amplitude distribution of the image at different frequencies, and the phase spectrum represents the phase distribution of the image at different frequencies.

[0199] The normalized Gaussian filter G is convolved with the amplitude spectrum A to obtain the local average value Ā of A. The Gaussian filter can smooth the image and suppress the high-frequency noise in the amplitude spectrum. The residual spectrum R is calculated, and its value is the amplitude spectrum A minus the local average value Ā. The residual spectrum reflects the part of the amplitude spectrum that deviates from the local average value, which corresponds to the salient area in the image. The residual spectrum R and the phase spectrum P are recombined into the frequency domain representation F', where the real part of F' is obtained by multiplying the cosine values ​​of R and P, and the imaginary part is obtained by multiplying the sine values ​​of R and P. F' is subjected to a two-dimensional fast inverse Fourier transform (2D-IFFT) to obtain a saliency map S. The inverse Fourier transform can restore the frequency domain signal to the spatial domain signal, thereby converting the residual spectrum in the frequency domain into a saliency map in the spatial domain.

[0200] The saliency map S is normalized and its pixel values ​​are mapped to the interval [0,1]. Normalization can eliminate the scale difference of the saliency map and facilitate the subsequent thresholding operation.

[0201] Gaussian filtering is performed on the normalized saliency map to further suppress noise and details and highlight the main salient areas.

[0202] The filtered saliency map is thresholded, and the pixels greater than the threshold are set to 1, and the remaining pixels are set to 0, to generate a binary salient region mask M. The selection of the threshold can be adaptively determined according to the statistical characteristics of the saliency distribution, such as taking the mean or median of the saliency value.

[0203] After obtaining the salient area mask, it is necessary to further consider the actual visual attention distribution of the driver, find out the areas where the attention distribution is insufficient, and perform visual enhancement processing on these areas to guide the driver's attention distribution. This application adopts a salient area enhancement method based on visual attention distribution, and the specific steps are as follows:

[0204] Step 1: Analysis of attention deficit areas

[0205] Perform a logical AND operation on the salient area mask M and the visual attention distribution data A to obtain the attention-deficient area mask M'. Visual attention distribution data can be collected through an eye tracker or other equipment, where a pixel with a value of 1 indicates that it is being watched, and a pixel with a value of 0 indicates that it is not being watched.

[0206] The pixel with a value of 1 in the attention-deficient area mask M' represents a pixel that is both a salient area and not noticed, that is, an area with insufficient attention allocation.

[0207] Step 2: Enhancement of attention deficit areas;

[0208] According to the attention-deficient area mask M', the corresponding area in the original training content image I is visually enhanced to obtain the enhanced image I'. The visual enhancement processing method can be selected according to the specific application scenario and requirements. Common methods include contrast enhancement, sharpening, color enhancement, etc.

[0209] Taking contrast enhancement as an example, it can be achieved as follows: convert image I to HSV color space, extract V channel (brightness channel), record it as V. Linearly stretch the grayscale value of the pixel corresponding to M' in V to expand its dynamic range to [0,255], and obtain the enhanced brightness channel V'. Recombine the enhanced V' channel with the original H and S channels to obtain the contrast-enhanced color image I'.

[0210] Step 3: Fusion of visual enhancement results;

[0211] The contrast-enhanced image I' is weightedly fused with the original image I to obtain the final dynamic optimization result I". The weight of the weighted fusion can be adaptively determined according to the area ratio of the attention-deficient region in the entire image. The larger the area ratio, the greater the weight of the enhanced image. For example, if the area of ​​the attention-deficient region accounts for 30% of the total image area, the weight of the enhanced image can be set to 0.3 and the weight of the original image can be set to 0.7.

[0212] Suppose a traffic scene color image of size 800×600 is input, which contains multiple targets such as roads, road signs, pedestrians, and vehicles.

[0213] First, the color image is converted to Lab color space, the L channel is extracted as the brightness image, and the fast Fourier transform is performed on it to obtain the amplitude spectrum and phase spectrum. Then the amplitude spectrum is smoothed with a Gaussian filter, and the residual spectrum is calculated. The frequency domain representation is reconstructed using the residual spectrum and phase spectrum, and an inverse Fourier transform is performed to obtain a saliency map.

[0214] Then, the salient map is normalized, Gaussian filtered and thresholded to obtain the salient area mask. Through analysis, it is found that the salient values ​​corresponding to road signs and pedestrians are higher, while the salient values ​​of road areas are lower, which is consistent with the visual perception characteristics of drivers.

[0215] Then, the salient area mask is logically ANDed with the visual attention distribution data to find out the areas where attention is insufficiently allocated. Assuming that the current driver's vision is mainly focused on the vehicle and road area, and the attention to road signs and pedestrians is insufficient, the salient areas where road signs and pedestrians are located will be marked as attention-deficient areas.

[0216] Finally, the attention-deficient area is visually enhanced, and the enhanced result is weightedly fused with the original image to obtain the dynamically optimized training content image. By comparing the images before and after optimization, it can be found that the visual conspicuity of the optimized road signs and pedestrian areas is significantly improved, which helps to attract the driver's attention.

[0217] In order to quantitatively analyze the differences before and after optimization, the changes in the area ratio and significance value of the attention-deficient area can be statistically analyzed. For example, before optimization, the area ratio of the attention-deficient area was 20%, and the average significance value was 0.6; after optimization, the area ratio of the attention-deficient area was reduced to 5%, and the average significance value was increased to 0.85. This shows that the method of this application can effectively guide the driver's visual attention allocation and improve the recognition efficiency of key areas.

[0218] In summary, this application proposes a method for dynamic optimization of adaptive training content based on visual attention guidance. This method uses the spectral residual method to detect the saliency of the image, extracts the prior distribution of visual attention, and realizes the dynamic optimization of training content through attention loss area analysis and visual enhancement processing. Case analysis shows that this method can effectively guide the driver's visual attention allocation, improve the recognition efficiency of key information, and provide a new idea for realizing adaptive driving training.

[0219] In an optional embodiment,

[0220] In combination with the current cognitive load level of the driver, determining the information entropy of each information unit in the salient area mask, and dynamically adjusting the information density of the salient area mask using an information entropy minimization algorithm includes:

[0221] Performing semantic segmentation on the salient region mask to obtain a set of information units, each of which is represented by a binary mask having the same size as the salient region mask;

[0222] Calculate the information entropy of each information unit, the information entropy is obtained by counting the probability distribution of pixels with different gray levels in the mask corresponding to the information unit, and the information entropy measures the amount of information contained in the information unit;

[0223] Taking the minimum sum of information entropy of all information units as the optimization goal and the constraint condition that the current cognitive load level of the driver does not exceed the preset constraint threshold as the constraint condition, the differential evolution algorithm is used to search for the optimal information presentation rhythm parameters, which include the display duration, display order and fade-in and fade-out effect of each information unit;

[0224] According to the optimized information presentation rhythm parameters, the information density of the salient region mask is dynamically adjusted.

[0225] Exemplarily, the present application proposes a method for adaptively optimizing the information density of salient areas in combination with the driver's cognitive load level. The method first performs semantic segmentation on the salient area mask, extracts independent information units, and then calculates the information entropy of each information unit, and takes minimizing the information entropy as the optimization goal, takes the driver's current cognitive load level as the constraint condition, and uses the differential evolution algorithm to search for the optimal information presentation rhythm parameters, thereby achieving dynamic adjustment of the information density of the salient area and avoiding the problem of cognitive overload or cognitive deficiency of the driver.

[0226] In order to achieve refined control of the information density of the salient region, it is necessary to first perform semantic segmentation on the salient region mask and extract independent information units. This application uses a semantic segmentation algorithm based on region growing to segment the salient region mask into multiple semantically consistent regions, each corresponding to an information unit. The specific steps are as follows:

[0227] Selection of regional growing seed points;

[0228] Connected domain analysis is performed on the salient region mask to extract all connected regions, each of which corresponds to a candidate seed point. The local variance of each candidate seed point is calculated, and the pixel with the largest variance is selected as the initial seed point. The local variance reflects the degree of grayscale change in the area around the pixel point. The larger the variance, the higher the significance of the pixel point area, and the more suitable it is as the starting point for region growth.

[0229] Regional growth process;

[0230] Taking the initial seed point as the center, the pixels with grayscale values ​​close to the seed point are expanded to the surrounding areas and included in the current growth area. Grayscale similarity can be measured by Euclidean distance or Mahalanobis distance. When the grayscale distance between the pixel point and the seed point is less than the preset threshold, the two are considered to belong to the same semantic area.

[0231] Repeat the above process for the newly added pixels until the current growth area can no longer be expanded.

[0232] The next growth starting point is selected from the remaining candidate seed points, and the region growing process is repeated until all pixels are assigned to a semantic region.

[0233] Information unit extraction;

[0234] The semantic segmentation results are post-processed, including region merging, hole filling, small area filtering, etc., to improve the completeness and accuracy of the segmentation results.

[0235] A binary mask is generated for each semantic region, and the size of the mask is the same as the size of the original saliency map. The pixel value of the corresponding semantic region is 1, and the other pixel values ​​are 0. Each semantic region mask represents an independent information unit.

[0236] After extracting the information unit, it is necessary to measure the amount of information contained in each information unit in order to optimize the control of information density later. This application uses information entropy to measure the amount of information in the information unit. The calculation steps of information entropy are as follows:

[0237] Perform grayscale statistics on the binary mask corresponding to each information unit to obtain the frequency of occurrence of different grayscales. Here, only two grayscales need to be counted, namely 0 and 1. Let the number of pixels with grayscale 0 be n0, and the number of pixels with grayscale 1 be n1.

[0238] Calculate the probability of each gray level appearing, that is, divide the frequency of the corresponding gray level by the total number of pixels in the mask.

[0239] According to the definition of Shannon entropy, the information entropy of the information unit is calculated. Substitute the gray level probability into the Shannon entropy formula to obtain the information entropy of the information unit. The information entropy is normalized and mapped to the interval [0,1]. Normalization can eliminate the dimensional difference of information entropy between different information units, which is convenient for subsequent optimization calculations.

[0240] After obtaining the information entropy of each information unit, it is necessary to further consider how to present these information units to the driver so that they can maximize the transmission of effective information without causing cognitive overload. This application models this problem as a constrained optimization problem, with the minimization of information entropy as the optimization goal and the current cognitive load level of the driver as the constraint condition, and optimizes the presentation rhythm parameters of the information unit to obtain the optimal information presentation strategy.

[0241] Specifically, this application uses a differential evolution algorithm to solve the optimization problem, and the optimization objective function and constraint conditions are set as follows:

[0242] Objective function: minimize the sum of information entropy of all information units; Constraint: the driver's current cognitive load level CL does not exceed the preset constraint threshold. The cognitive load level can be estimated from physiological signals such as heart rate and breathing rate, and its value range is [0,1], indicating the degree of cognitive load. The constraint threshold can be adaptively adjusted according to the individual differences of the driver and the complexity of the driving environment, and is usually 0.6~0.8.

[0243] Optimization variables: Information presentation rhythm parameters, including the following three parameters:

[0244] The display duration ti of the information unit indicates the duration of the display of the i-th information unit on the screen; the display order oi of the information unit indicates the order of the i-th information unit in the display queue; the fade-in and fade-out effect parameter ei of the information unit indicates the fade-in and fade-out time of the i-th information unit when it is displayed and disappears.

[0245] Variable value range:

[0246] The display duration ti ranges from [1,10], in seconds; the display order oi ranges from [1,M], indicating the order in M ​​information units; the fade-in and fade-out effect parameter ei ranges from [0,1], indicating the speed of fade-in and fade-out, 0 indicates no fade-in and fade-out effect, and 1 indicates the softest fade-in and fade-out effect.

[0247] The differential evolution algorithm is used to solve the above constrained optimization problem and obtain the optimal information presentation rhythm parameters. The differential evolution algorithm is a heuristic optimization algorithm based on swarm intelligence. It simulates the evolutionary mechanism of "survival of the fittest" in nature, iteratively optimizes a group of randomly generated candidate solutions, and finally obtains the global optimal solution.

[0248] After obtaining the optimal information presentation rhythm parameters, the information density of the salient area mask can be dynamically adjusted. The specific steps are as follows:

[0249] According to the optimal display order and display duration, each information unit is presented on the screen in turn. For each information unit, the corresponding area in the original salient area image is first extracted according to its corresponding binary mask, and then the fade-in and fade-out effect parameters are used to control the fade-in and fade-out speed of the area when it appears and disappears, creating a smooth and natural display effect. After one information unit is displayed, the next information unit begins to be displayed until all information units are displayed.

[0250] While dynamically displaying information units, it is necessary to supplement the remaining blank areas in the salient area mask to avoid the problem of insufficient driver cognition caused by too sparse information density. The blank area is mean filtered to obtain a smoothed background image, which is then superimposed on the original salient area image to form the final optimized salient map.

[0251] Through adaptive optimization of information density, when viewing the optimized saliency map, the driver can obtain the maximum key information under the premise of controllable cognitive load, which helps to improve the driver's perception and judgment of road conditions. In addition, this method fully considers the individual differences of drivers and the dynamic changes of the driving environment, has strong adaptability and real-time performance, and can be applied to information perception enhancement in different driving scenarios.

[0252] In summary, this application proposes an adaptive optimization method for salient area information density based on cognitive load perception. This method extracts information units through semantic segmentation and calculates the information entropy of the information units. Then, the information entropy minimization is used as the optimization goal, and the driver's cognitive load is used as the constraint condition. The differential evolution algorithm is used to optimize the information presentation rhythm parameters, and finally the dynamic adjustment of the salient area information density is achieved. Case analysis shows that this method can present key information to the maximum extent while ensuring that the driver's cognitive load is controllable. It has the potential to improve the driver's perception ability and has important reference value for the design and development of assisted driving systems.

[0253] Figure 2 FIG. 1 is a schematic diagram of a human-computer interaction system for driving training applied to a vehicle HUD according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0254] The first unit is used to obtain the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; using a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, respectively based on the physiological characteristic data and the behavioral characteristic data for real-time prediction and recognition, to obtain the driver's current cognitive load level and driving behavior pattern;

[0255] The second unit is used to construct a driver capability profile based on the driver's historical training data using a collaborative filtering algorithm to determine the driver's mastery of different knowledge points; extract training knowledge points that match the driver's capability profile from the preset training knowledge base, use a reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form adaptive training content;

[0256] The third unit is used to enhance the adaptive training content in real time according to the driver's visual attention distribution data using the visual saliency detection algorithm, and to dynamically adjust the information density in the adaptive training content using the information entropy minimization algorithm in combination with the driver's current cognitive load level to match the driver's real-time cognitive ability.

[0257] According to a third aspect of the embodiments of the present invention,

[0258] An electronic device is provided, comprising:

[0259] processor;

[0260] a memory for storing processor-executable instructions;

[0261] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0262] A fourth aspect of the embodiments of the present invention is:

[0263] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0264] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0265] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human-computer interaction method for driving training applied to vehicle HUD, characterized in that: include: Acquire the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; use a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, and perform real-time prediction and recognition based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern; Based on the driver's historical training data, the collaborative filtering algorithm is used to build a driver's ability profile to determine the driver's mastery of different knowledge points; the training knowledge points that match the driver's ability profile are extracted from the preset training knowledge base, and the training knowledge points are sorted by difficulty using the reinforcement learning algorithm, and combined with the driver's current cognitive load level, adaptive training content is finally formed; According to the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time, and combined with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability; According to the driver's visual attention distribution data, the visual saliency detection algorithm is used to enhance the adaptive training content in real time, and combined with the driver's current cognitive load level, the information entropy minimization algorithm is used to dynamically adjust the information density in the adaptive training content to match the driver's real-time cognitive ability, including: The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, the saliency map is obtained by Gaussian filtering and inverse Fourier transform, and the saliency map is thresholded to generate a salient area mask; In combination with the current cognitive load level of the driver, the information entropy of each information unit in the salient area mask is determined, and the information density of the salient area mask is dynamically adjusted using an information entropy minimization algorithm to match the real-time cognitive ability of the driver; The spectral residual method is used to perform visual saliency detection on the images in the adaptive training content, the brightness image is extracted and Fourier transformed, the amplitude spectrum is replaced by the local average to obtain the residual spectrum, and the saliency map is obtained by Gaussian filtering and inverse Fourier transform. The thresholding of the saliency map to generate a salient area mask includes: Convert the color image of the training content to Lab color space and extract the L channel as the brightness image; Performing a two-dimensional fast Fourier transform on the brightness image to obtain a frequency domain representation, and decomposing the frequency domain representation into an amplitude spectrum and a phase spectrum; Calculating a residual spectrum of the amplitude spectrum, wherein the residual spectrum is obtained by subtracting a local average value of the amplitude spectrum after Gaussian filtering from the amplitude spectrum; Recombining the residual spectrum and the phase spectrum into a frequency domain representation and performing a two-dimensional fast inverse Fourier transform to obtain a saliency map; Normalizing, Gaussian filtering and thresholding the saliency map to generate a salient area mask; Performing a logical AND operation on the salient area mask and the visual attention distribution data to find out the salient area where attention is insufficiently allocated, and obtaining an attention-deficient area mask; According to the attention-deficit area mask, visual enhancement processing is performed on the area corresponding to the image in the training content to obtain dynamically optimized training content; In combination with the current cognitive load level of the driver, determining the information entropy of each information unit in the salient area mask, and dynamically adjusting the information density of the salient area mask using an information entropy minimization algorithm includes: Performing semantic segmentation on the salient region mask to obtain a set of information units, each of which is represented by a binary mask having the same size as the salient region mask; Calculate the information entropy of each information unit, the information entropy is obtained by counting the probability distribution of pixels with different gray levels in the mask corresponding to the information unit, and the information entropy measures the amount of information contained in the information unit; Taking the minimum sum of information entropy of all information units as the optimization goal and the constraint condition that the current cognitive load level of the driver does not exceed the preset constraint threshold as the constraint condition, the differential evolution algorithm is used to search for the optimal information presentation rhythm parameters, which include the display duration, display order and fade-in and fade-out effect of each information unit; According to the optimized information presentation rhythm parameters, the information density of the salient region mask is dynamically adjusted.

2. The method according to claim 1, characterized in that A driver cognitive load level prediction model and a driving behavior pattern recognition model are established using a deep neural network, and real-time prediction and recognition are performed based on the physiological characteristic data and the behavioral characteristic data, respectively, to obtain the driver's current cognitive load level and driving behavior pattern, including: The preprocessed physiological characteristic data are input into the cognitive load level prediction model based on the long short-term memory neural network. The temporal characteristics of the physiological characteristic data are extracted through the multi-layer long short-term memory neural network, and the attention mechanism is introduced to adaptively allocate the weights corresponding to the temporal characteristics at different moments. After the fully connected layer and activation function, the predicted value of the cognitive load level is obtained. The behavior feature data are preprocessed, statistical values ​​of the behavior feature data are extracted, and a behavior heat map of the behavior feature data is constructed; the behavior feature data and the behavior heat map are input into a driving behavior pattern recognition model jointly constructed based on a convolutional neural network and a long short-term memory neural network for feature extraction and feature fusion to obtain fusion features; multi-classification is performed through the fully connected layer and the classification layer of the driving behavior pattern recognition model to obtain the probability distribution of the driving behavior pattern in each time window, and the driving behavior pattern with the highest probability distribution value is used as the driving behavior pattern of the current time window.

3. The method according to claim 1, characterized in that Extract training knowledge points that match the driver's ability profile from the preset training knowledge base, use the reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form an adaptive training content including: Build a training knowledge base covering driving training, classify and label the knowledge points in the training knowledge base, and the attributes of the knowledge points include the category, difficulty level and prerequisite knowledge points; use the historical training data of drivers, explore the similarities between different drivers in the mastery of knowledge points, use collaborative filtering algorithms to predict the current driver's mastery of unlearned knowledge points, and determine the set of knowledge points that match the driver's ability; Based on the knowledge point set that matches the driver's ability, a reinforcement learning algorithm is used to sort the knowledge point. The knowledge point sorting problem is modeled as a Markov decision process, where the state is the completed knowledge point sequence, the action is to select the next knowledge point, and the reward comprehensively considers the difficulty of the knowledge point, the matching degree of the driver's ability, and the dependency of the knowledge points. Through continuous learning, the optimal sorting strategy that maximizes the cumulative reward is found. Based on the comparison between the driver's current cognitive load level and the preset reasonable cognitive load range, the difficulty of the training content is dynamically adjusted to form adaptive training content.

4. The method according to claim 3, characterized in that The method also includes dynamically adjusting the difficulty of the training content: The Bayesian knowledge tracking model is used to track the probability of drivers mastering each knowledge point in the training content in real time, and dynamically adjust the presentation frequency and presentation time of each knowledge point. The presentation frequency and presentation time of knowledge points with a mastery probability higher than the preset threshold are reduced, and the presentation frequency and presentation time of knowledge points with a mastery probability lower than the preset threshold are increased. According to the driver's current cognitive load level, the fuzzy control algorithm is used to dynamically adjust the difficulty gradient of the knowledge points.

5. A driving training human-computer interaction system applied to a vehicle HUD, used to implement the method described in any one of claims 1 to 4, characterized in that: include: The first unit is used to obtain the driver's physiological characteristic data, including at least one of EEG, heart rate, and eye movement, and the driver's behavioral characteristic data, including at least one of accelerator, brake, and steering operation behavior data and visual attention distribution data; using a deep neural network to establish a driver's cognitive load level prediction model and a driving behavior pattern recognition model, respectively based on the physiological characteristic data and the behavioral characteristic data for real-time prediction and recognition, to obtain the driver's current cognitive load level and driving behavior pattern; The second unit is used to construct a driver capability profile based on the driver's historical training data using a collaborative filtering algorithm to determine the driver's mastery of different knowledge points; extract training knowledge points that match the driver's capability profile from the preset training knowledge base, use a reinforcement learning algorithm to sort the difficulty of the training knowledge points, and combine the driver's current cognitive load level to finally form adaptive training content; The third unit is used to enhance the adaptive training content in real time according to the driver's visual attention distribution data using the visual saliency detection algorithm, and to dynamically adjust the information density in the adaptive training content using the information entropy minimization algorithm in combination with the driver's current cognitive load level to match the driver's real-time cognitive ability.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images