Intervention strategy determination method, apparatus and device

Abstract

The application discloses an intervention strategy determination method, device and equipment, the method comprises the following steps: in the process that a target object drives a target vehicle, target physiological signals of the target object and target driving data of the target vehicle are acquired and input to a multidimensional information evaluation model for cross-modal fusion processing to obtain target multi-modal fusion features; the target multi-modal fusion features are orthogonally decoupled and projected to obtain target cognitive load features, target fatigue degree features and target emotional stability features; the cognitive load result of the target object is predicted according to the target cognitive load features, the fatigue degree result of the target object is predicted according to the target fatigue degree features, and the emotional stability result of the target object is predicted according to the target emotional stability features; and the target intervention strategy is determined according to the cognitive load result, the fatigue degree result and the emotional stability result. The application improves the accuracy of the target intervention strategy.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method, apparatus and device for determining intervention strategies. Background Technology

[0002] Current driver monitoring systems (DMS) mainly rely on in-vehicle cameras for facial recognition. They use indicators such as eyelid closure and yawning frequency to determine whether the driver is in an abnormal state such as fatigue, and trigger alarms using fixed thresholds after detecting an abnormal state.

[0003] After multiple physiological signals are fused, a single risk value or classification label is output. The different types of state information reflected by each signal source are mixed and compressed during the fusion process, so the intervention strategy can only be determined based on a single-dimensional assessment indicator, resulting in a single intervention strategy and low accuracy. Summary of the Invention

[0004] This invention provides a method, apparatus, and device for determining intervention strategies, which can improve the accuracy of target intervention strategies.

[0005] On one hand, the present invention provides a method for determining intervention strategies, the method comprising: During the process of the target object driving the target vehicle, the target object's target physiological signals and the target vehicle's target driving data are acquired. The target physiological signals and the target driving data are input into a multidimensional information evaluation model for cross-modal fusion processing to obtain the target multimodal fusion features; Orthogonal uncoupling projection is performed on the multimodal fusion features of the target to obtain the target cognitive load features, target fatigue level features, and target emotional stability features; The cognitive load result of the target object is predicted based on the target cognitive load characteristics, the fatigue result of the target object is predicted based on the target fatigue degree characteristics, and the emotional stability result of the target object is predicted based on the target emotional stability characteristics. The cognitive load result, the fatigue degree result, and the emotional stability result are then output. Based on the cognitive load results, the fatigue level results, and the emotional stability results, a targeted intervention strategy is determined for the target subject.

[0006] On the other hand, an intervention strategy determination device is provided, the device comprising: The data acquisition module is used to acquire the target physiological signals of the target object and the target driving data of the target vehicle during the process of the target object driving the target vehicle; The feature fusion module is used to input the target physiological signal and the target driving data into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain the target multi-modal fusion features; The feature parsing module is used to perform orthogonal uncoupling projection on the multimodal fusion features of the target to obtain the target cognitive load features, the target fatigue level features, and the target emotional stability features. The prediction module is used to predict the cognitive load result of the target object based on the target cognitive load characteristics, predict the fatigue result of the target object based on the target fatigue characteristics, and predict the emotional stability result of the target object based on the target emotional stability characteristics, and output the cognitive load result, the fatigue result, and the emotional stability result; The intervention strategy determination module is used to determine the target intervention strategy for the target object based on the cognitive load result, the fatigue level result, and the emotional stability result.

[0007] In one exemplary embodiment, the model training module is further configured to: determine first loss data based on the difference between the sample cognitive load result and the sample cognitive load label; determine second loss data based on the difference between the sample fatigue level result and the sample fatigue level label; determine third loss data based on the difference between the sample emotional stability result and the sample emotional stability label; determine target loss data based on the first loss data, the second loss data, and the third loss data; and adjust the model parameters of the preset model based on the target loss data until the training termination condition is met, thereby obtaining the multidimensional information evaluation model.

[0008] In one exemplary embodiment, the model training module is further configured to: determine multidimensional loss data based on the first loss data, the second loss data, and the third loss data; determine orthogonalized constraint loss data based on the first weight corresponding to the sample cognitive load, the second weight corresponding to the sample fatigue level, and the third weight corresponding to the sample emotional stability; determine statistically independent constraint loss data based on the sample cognitive load result, the sample fatigue level result, and the sample emotional stability result; and determine the target loss data based on the multidimensional loss data, the orthogonalized constraint loss data, and the statistically independent constraint loss data.

[0009] In one exemplary embodiment, the model training module is further configured to: determine a first preset weight of the orthogonalized constraint loss data and a second preset weight of the statistically independent constraint loss data; calculate the product of the orthogonalized constraint loss data and the first preset weight to obtain first weighted loss data; calculate the product of the statistically independent constraint loss data and the second preset weight to obtain second weighted loss data; and calculate the sum of the first weighted loss data, the second weighted loss data, and the multidimensional loss data to obtain the target loss data.

[0010] In one exemplary embodiment, the intervention strategy determination module is further configured to construct bandpasses corresponding to the three-dimensional results output by the multidimensional information assessment model, wherein each bandpass includes a preset trigger threshold corresponding to the intervention strategy; the three-dimensional results include the cognitive load result, the fatigue level result, and the emotional stability result; based on the bandpass, any one of the cognitive load result, the fatigue level result, and the emotional stability result is used as the current prediction result; the preset trigger threshold of the bandpass corresponding to the current prediction result is obtained to obtain the current trigger threshold; if the current prediction result is greater than the current trigger threshold, the current prediction result is determined to be an abnormal result, and the current intervention strategy corresponding to the dimension of the current prediction result is obtained; based on the current intervention strategies corresponding to the cognitive load result, the fatigue level result, and the emotional stability result, the target intervention strategy is constructed and executed.

[0011] In one exemplary embodiment, the bandpass converter further includes a context feature vector and probability distribution parameters of the intervention strategy. The intervention strategy determination module is further configured to obtain at least two initial intervention strategies corresponding to the dimension of the current prediction result; adjust the probability distribution parameters of each initial intervention strategy according to the context feature vector to obtain adjusted probability distribution parameters of each initial intervention strategy; perform a random sampling on each initial intervention strategy according to the adjusted probability distribution parameters to obtain a sampled value of each initial intervention strategy; and select the strategy with the highest sampled value from the at least two initial intervention strategies as the current intervention strategy.

[0012] In one exemplary embodiment, the apparatus further includes: The update result acquisition module is used to start timing when the execution of the target intervention strategy is detected to have ended. If the cumulative duration reaches a preset duration threshold, it acquires updated cognitive load results, updated fatigue level results, and updated emotional stability results. The first reward calculation module is used to calculate the first reward data of the cognitive load dimension based on the difference between the cognitive load result and the updated cognitive load result; and update the probability distribution parameters of the cognitive load dimension based on the first reward data. The second reward calculation module is used to calculate second reward data for the fatigue level dimension based on the difference between the fatigue level result and the updated fatigue level result; and to update the probability distribution parameters of the fatigue level dimension based on the second reward data. The third reward calculation module is used to calculate the third reward data of the emotion stability dimension based on the difference between the emotion stability result and the updated emotion stability result; and to update the probability distribution parameters of the emotion stability dimension based on the third reward data.

[0013] In one exemplary embodiment, the apparatus further includes: The threshold increase module is used to increase the preset trigger threshold corresponding to the current dimension if the number of times the abnormal result of the current dimension occurs within a unit time is greater than the target number; the current dimension is at least one of the cognitive load dimension, fatigue level dimension, and emotional stability dimension. The threshold reduction module is used to reduce the preset trigger threshold corresponding to the current dimension if the ratio of the number of occurrences of abnormal results in the current dimension per unit time to the target number is less than a preset ratio.

[0014] In one exemplary embodiment, the target intervention strategy includes a first intervention strategy based on the cognitive load dimension, a second intervention strategy based on the fatigue level dimension, and a third intervention strategy based on the emotional stability dimension. The device further includes: An intervention level acquisition module is used to acquire the first intervention level of the first intervention strategy, the second intervention level of the second intervention strategy, and the third intervention level of the third intervention strategy; The target level determination module is used to determine an initial intervention level based on the orthogonal result of the first intervention level and the second intervention level; and to revise the initial intervention level based on the third intervention level and the emotional stability result to obtain the target intervention level. The intervention action execution module is used to obtain the intervention content corresponding to the target intervention level and execute the intervention action corresponding to the intervention content.

[0015] In one exemplary embodiment, the apparatus further includes: The sample data acquisition module is used to acquire the physiological signals of the sample subjects during the driving of the sample vehicle and the driving data of the sample vehicle; the sample subjects include sample cognitive load labels, sample fatigue level labels and sample emotional stability labels. The sample input module is used to input the sample physiological signals and the sample driving data into a preset model, and output the sample cognitive load results, sample fatigue results, and sample emotional stability results of the sample object. The model training module is used to train the preset model based on the differences between the sample cognitive load results and the sample cognitive load labels, the differences between the sample fatigue results and the sample fatigue labels, and the differences between the sample emotional stability results and the sample emotional stability labels, so as to obtain the multidimensional information evaluation model.

[0016] In one exemplary embodiment, the sample input module is further configured to input the sample physiological signal and the sample driving data into a preset model, perform feature encoding processing on the sample physiological signal to obtain sample physiological signal encoded features, and perform feature encoding processing on the sample driving data to obtain sample driving data encoded features; perform cross-modal fusion processing on the sample physiological signal encoded features and the sample driving data encoded features to obtain sample multimodal fusion features; perform orthogonal decoupling projection on the sample multimodal fusion features to obtain sample cognitive load features, sample fatigue level features, and sample emotional stability features; predict the sample cognitive load result based on the sample cognitive load features, predict the sample fatigue level result based on the sample fatigue level features, and predict the sample emotional stability result based on the sample emotional stability features.

[0017] In one exemplary embodiment, the feature parsing module is further configured to determine a first weight corresponding to the sample cognitive load, a second weight corresponding to the sample fatigue level, and a third weight corresponding to the sample emotional stability; determine the sample cognitive load feature based on the sample multimodal fusion feature and the first weight; determine the sample fatigue level feature based on the sample multimodal fusion feature and the second weight; and determine the sample emotional stability feature based on the sample multimodal fusion feature and the third weight.

[0018] In one exemplary embodiment, the apparatus further includes: The bias term determination module is used to determine the first initial bias term corresponding to the cognitive load of the sample, the second initial bias term corresponding to the fatigue level of the sample, and the third initial bias term corresponding to the emotional stability of the sample. The feature parsing module is further configured to calculate the first product of the sample multimodal fusion feature and the first weight, and determine the sample cognitive load feature based on the sum of the first product and the first initial bias term; calculate the second product of the sample multimodal fusion feature and the second weight, and determine the sample fatigue level feature based on the sum of the second product and the second initial bias term; calculate the third product of the sample multimodal fusion feature and the third weight, and determine the sample emotional stability feature based on the sum of the third product and the third initial bias term.

[0019] On the other hand, an intervention strategy determination device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the intervention strategy determination method as described above.

[0020] On the other hand, a computer storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the intervention strategy determination method as described above.

[0021] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the intervention strategy determination method as described above.

[0022] The intervention strategy determination method, apparatus, and equipment provided by this invention have the following technical effects: This invention acquires the target physiological signals of the target subject and the target driving data of the target vehicle during the driving process of the target subject. Then, the target physiological signals and the target driving data are input into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain target multi-modal fusion features. Next, orthogonal uncoupling projection is performed on the target multi-modal fusion features to obtain three-dimensional features: target cognitive load features, target fatigue level features, and target emotional stability features. This facilitates further multi-dimensional evaluation results. Then, the cognitive load result of the target subject is predicted based on the target cognitive load features, the fatigue level result is predicted based on the target fatigue level features, and the emotional stability result is predicted based on the target emotional stability features. The cognitive load result, the fatigue level result, and the emotional stability result are output, thus obtaining evaluation results in three dimensions: cognitive load, fatigue level, and emotional stability. Finally, based on the cognitive load result, the fatigue level result, and the emotional stability result, a target intervention strategy is formulated for the target subject from the three dimensions of cognitive load, fatigue level, and emotional stability, improving the accuracy of the target intervention strategy. Attached Figure Description

[0023] To more clearly illustrate the technical solutions and advantages in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an intervention strategy determination method provided in the embodiments of this specification; Figure 2 This is an overall architecture diagram of an intervention strategy determination system provided in the embodiments of this specification; Figure 3 This is a schematic diagram of a three-dimensional evaluation result provided in the embodiments of this specification; Figure 4 This is a flowchart of MMST fusion and orthogonal decoupling provided in the embodiments of this specification; Figure 5 This is a schematic diagram of the learning process for three bandpasses corresponding to three dimensions, provided in the embodiments of this specification. Figure 6 This is a schematic diagram of a five-level progressive intervention strategy provided in the embodiments of this specification; Figure 7 This is a schematic diagram illustrating online learning and personalized optimization of intervention effects provided in the embodiments of this specification; Figure 8 This is a safety linkage flowchart for autonomous driving provided in the embodiments of this specification; Figure 9 This is a schematic diagram of the structure of an intervention strategy determination device provided in the embodiments of this specification; Figure 10 This is a schematic diagram of the structure of a server provided in the embodiments of this specification. Detailed Implementation

[0025] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0027] In this embodiment of the invention, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0028] The following describes a method for determining intervention strategies according to the present invention. Figure 1This is a flowchart illustrating an intervention strategy determination method provided in an embodiment of this specification. This specification provides the operational steps of the method described in the embodiment or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiment or drawings. Specifically, as... Figure 1 As shown, the method may include: S101: During the process of the target object driving the target vehicle, acquire the target object's target physiological signals and the target vehicle's target driving data. In the embodiments of this specification, the target object can be a driver, the target physiological signal of the target object can be a multimodal physiological signal, which may include, but is not limited to, head and face signals, grip force signals on the vehicle steering wheel, and the target driving data of the target vehicle may include, but is not limited to, the vehicle speed, acceleration, steering wheel angle, lane departure, and other data.

[0029] The method in this embodiment can be applied to scenarios such as advanced driver assistance systems (ADAS), driver monitoring systems (DMS), and safe takeover of autonomous driving systems.

[0030] S103: Input the target physiological signal and the target driving data into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain the target multi-modal fusion features.

[0031] In the embodiments of this specification, the target physiological signal and the target driving data can be input into a multidimensional information evaluation model, the target physiological signal can be processed by feature encoding to obtain target physiological signal encoded features, and the target driving data can be processed by feature encoding to obtain target driving data encoded features; then the target physiological signal encoded features and the target driving data encoded features can be processed by cross-modal fusion to obtain target multimodal fusion features.

[0032] S105: Perform orthogonal uncoupling projection on the target multimodal fusion features to obtain target cognitive load features, target fatigue level features, and target emotional stability features.

[0033] In the embodiments of this specification, the multi-modal fusion features of the target can be orthogonally decoupled and projected based on the multi-dimensional information evaluation model to obtain three-dimensional target cognitive load features, target fatigue level features, and target emotional stability features. Among them, the target cognitive load features are used to predict the cognitive load result, the target fatigue level features are used to predict the fatigue level result, and the target emotional stability features are used to predict the emotional stability result.

[0034] S107: Predict the cognitive load result of the target object based on the target cognitive load characteristics, predict the fatigue result of the target object based on the target fatigue characteristics, and predict the emotional stability result of the target object based on the target emotional stability characteristics, and output the cognitive load result, the fatigue result, and the emotional stability result.

[0035] In the embodiments of this specification, the multidimensional information assessment model can predict the cognitive load result of the target object based on the target cognitive load characteristics, predict the fatigue result of the target object based on the target fatigue characteristics, and predict the emotional stability result of the target object based on the target emotional stability characteristics, and output the cognitive load result, the fatigue result, and the emotional stability result, wherein the cognitive load result, the fatigue result, and the emotional stability result can all be scores in the range of 0-1. If the score of at least one of the cognitive load result, the fatigue result, or the emotional stability result is too high, it will affect the driving safety of the target object.

[0036] S109: Based on the cognitive load results, the fatigue level results, and the emotional stability results, determine the target intervention strategy for the target object.

[0037] In the embodiments of this specification, a target intervention strategy for the target object can be determined from three dimensions—cognitive load, fatigue level, and emotional stability—based on the cognitive load result, the fatigue level result, and the emotional stability result. For example, a cognitive load safety threshold, a fatigue level safety threshold, and an emotional stability safety threshold can be set respectively. If at least one of the following conditions is met—that the cognitive load result exceeds the cognitive load safety threshold, the fatigue level result exceeds the fatigue level safety threshold, or the emotional stability result exceeds the emotional stability safety threshold—a target intervention strategy needs to be developed for the target object to ensure driving safety.

[0038] In this embodiment, during the process of a target subject driving a target vehicle, target physiological signals of the target subject and target driving data of the target vehicle are acquired. Then, the target physiological signals and target driving data are input into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain target multi-modal fusion features. Next, orthogonal decoupling projection is performed on the target multi-modal fusion features to obtain three-dimensional features: target cognitive load features, target fatigue level features, and target emotional stability features. This facilitates further multi-dimensional evaluation results. Then, based on the target cognitive load features, the target subject's cognitive load result is predicted; based on the target fatigue level features, the target subject's fatigue level result is predicted; and based on the target emotional stability features, the target subject's emotional stability result is predicted. The cognitive load result, the fatigue level result, and the emotional stability result are then output, thus obtaining evaluation results in three dimensions: cognitive load, fatigue level, and emotional stability. Finally, based on the cognitive load result, the fatigue level result, and the emotional stability result, a target intervention strategy is formulated for the target subject from the three dimensions of cognitive load, fatigue level, and emotional stability, improving the accuracy of the target intervention strategy.

[0039] For example, such as Figure 2 As shown, Figure 2 The overall architecture diagram of an intervention strategy determination system is shown below. It includes a multimodal signal acquisition module, a feature extraction and fusion module, a three-dimensional state assessment module, a five-level progressive intervention module, an online learning module, and a safety linkage module for formulating intervention strategies. The functions of each module are described in detail below. The multimodal signal acquisition module may include a facial camera, a steering wheel grip force sensor, a heart rate sensor, and a vehicle dynamics sensor. The feature extraction and fusion module receives data from each sensor, extracts features, fuses them using a multimodal Transformer fusion model (MMST), and then assesses the state of the fused features using the three-dimensional state assessment module, obtaining three-dimensional assessment results. Based on the three-dimensional assessment results, the five-level progressive intervention module formulates intervention strategies, resulting in multiple intervention strategies of different levels. The highest-level intervention strategy is selected and executed through the safety linkage module. During the execution of the action, the online learning module collects effect feedback records and performs personalized optimization of the intervention action. The optimization results can then be used to fine-tune the multimodal Transformer fusion model (MMST).

[0040] In some embodiments, acquiring the target physiological signals of the target object and the target driving data of the target vehicle includes: The first physiological signal is obtained by collecting the eye features, mouth features, head posture, and facial micro-expressions of the target object. The grip strength, grip strength distribution information, and grip strength change frequency of the target object's two hands are collected to obtain the second physiological signal; Collect the heart rate information and heart rate variability of the target object to obtain the third physiological signal; The target physiological signal of the target object is determined based on at least one of the first physiological signal, the second physiological signal, and the third physiological signal; The target vehicle's speed, acceleration, steering wheel angle, and lane departure are collected to obtain the target vehicle's target driving data.

[0041] In the embodiments of this specification, target physiological signals of the target object and target driving data of the target vehicle can be acquired through sensors. Specifically, a first physiological signal can be obtained by acquiring the target object's eye features, mouth features, head posture, and facial micro-expressions through a facial camera; a second physiological signal can be obtained by acquiring the target object's hand grip force, grip force distribution information, and grip force change frequency through a steering wheel capacitance / pressure sensor array; and a third physiological signal can be obtained by acquiring the target object's heart rate information and heart rate variability through a non-contact heart rate sensor. Then, the target physiological signal of the target object is determined based on at least one of the first, second, and third physiological signals. For example, the first, second, and third physiological signals can all be used as the target physiological signal to obtain a multimodal physiological signal. Furthermore, the target vehicle's speed, acceleration, steering wheel angle, and lane departure can be acquired through a vehicle dynamics sensor to obtain the target vehicle's target driving data.

[0042] For example, sensor components can be deployed in the cockpit of the target vehicle, as shown in Table 1 below: Table 1

[0043] All sensor signals are aligned with a uniform time axis, and features can be extracted using preset parameters (e.g., a 2-second sliding window and a 0.5-second step size). This allows for cross-modal fusion processing of the target physiological signals and the target driving data to obtain target multimodal fusion features, which can be a multimodal fusion feature matrix M(t).

[0044] In some embodiments, the training method of the multidimensional information evaluation model includes: The sample physiological signals of the sample subjects during the driving of the sample vehicle and the sample driving data of the sample vehicle are obtained; the sample subjects include sample cognitive load labels, sample fatigue level labels and sample emotional stability labels. The sample physiological signals and the sample driving data are input into a preset model, and the sample cognitive load results, sample fatigue results, and sample emotional stability results of the sample object are output. Based on the differences between the sample cognitive load results and the sample cognitive load labels, the differences between the sample fatigue level results and the sample fatigue level labels, and the differences between the sample emotional stability results and the sample emotional stability labels, the preset model is trained to obtain the multidimensional information evaluation model.

[0045] In the embodiments of this specification, a multidimensional information evaluation model can be obtained through model training. The sample object and the target object can be of the same type, the sample physiological signal and the target physiological signal can be of the same type, and the sample driving data and the target driving data can be of the same type. The sample object includes labels in three dimensions: sample cognitive load label, sample fatigue level label, and sample emotional stability label. The sample cognitive load label can be labeled using gaze entropy, n-back task performance, etc. The n-back task is one of the most classic experimental paradigms in cognitive psychology and neuroscience for assessing working memory capacity and cognitive control; its performance directly reflects an individual's ability to refresh, maintain, and suppress interference under high cognitive load. The sample fatigue level label can be labeled using PERCLOS, blink duration, etc. PERCLOS refers to the proportion of time occupied by eyelid closure ≥70% (or 80%) per unit time; it is usually calculated using a 1-minute / 30-second / 60-second sliding window. The sample emotional stability label can be labeled using facial motor unit intensity, skin conductance fluctuations, etc.

[0046] Then, based on the differences between the sample cognitive load result and the sample cognitive load label, the differences between the sample fatigue level result and the sample fatigue level label, and the differences between the sample emotional stability result and the sample emotional stability label, the target loss data is determined. Then, the parameters of the preset model are adjusted according to the target loss data until the training termination condition is met, thereby training a multidimensional information evaluation model that can predict the evaluation results in three dimensions. The training termination condition can be determined based on at least one of the target loss data and the number of training iterations. For example, the training termination condition can be determined as the target loss data being less than a preset value.

[0047] The acquisition of the sample physiological signals of the sample object during the driving of the sample vehicle and the sample driving data of the sample vehicle includes: The eye features, mouth features, head posture, and facial micro-expressions of the sample object are collected to obtain the physiological signal of the first sample; The grip strength, grip strength distribution information, and grip strength change frequency of the sample subjects were collected to obtain the physiological signals of the second sample. Collect heart rate information and heart rate variability of the sample subjects to obtain the physiological signal of the third sample; The sample physiological signal of the sample object is determined based on at least one of the first sample physiological signal, the second sample physiological signal, and the third sample physiological signal; The vehicle speed, acceleration, steering wheel angle, and lane departure of the sample vehicle are collected to obtain sample driving data of the sample vehicle.

[0048] In the embodiments of this specification, the first sample physiological signal is the same type of signal as the first physiological signal of the target object, the second sample physiological signal is the same type of signal as the second physiological signal of the target object, and the third sample physiological signal is the same type of signal as the third physiological signal of the target object. For example, the first, second, and third sample physiological signals can all be used as the sample physiological signals of the sample object to obtain multimodal physiological signals, thereby improving the diversity of sample data and facilitating the improvement of the accuracy of the model's predictive multidimensional evaluation results during subsequent model training.

[0049] In some embodiments, the step of inputting the sample physiological signals and the sample driving data into a preset model and outputting the sample cognitive load results, sample fatigue level results, and sample emotional stability results of the sample object includes: The sample physiological signal and the sample driving data are input into a preset model. The sample physiological signal is processed by feature encoding to obtain the sample physiological signal encoding feature, and the sample driving data is processed by feature encoding to obtain the sample driving data encoding feature. Cross-modal fusion processing is performed on the physiological signal encoding features and driving data encoding features of the samples to obtain multimodal fusion features of the samples; Orthogonal uncoupling projection is performed on the multimodal fusion features of the samples to obtain the sample cognitive load features, sample fatigue level features, and sample emotional stability features; The cognitive load result of the sample is predicted based on the cognitive load characteristics of the sample, the fatigue level result of the sample is predicted based on the fatigue level characteristics of the sample, and the emotional stability result of the sample is predicted based on the emotional stability characteristics of the sample.

[0050] In the embodiments of this specification, the preset model may include a feature encoding network, a cross-modal Transformer fusion network, and three prediction networks (a cognitive load prediction network, a fatigue level prediction network, and an emotional stability prediction network). After inputting the sample physiological signals and the sample driving data into the preset model, the feature encoding network can perform feature encoding processing on the sample physiological signals to obtain sample physiological signal encoded features, and perform feature encoding processing on the sample driving data to obtain sample driving data encoded features. Then, orthogonal decoupling projection is performed on the sample multimodal fusion features to obtain sample cognitive load features, sample fatigue level features, and sample emotional stability features. The sample cognitive load features are then input into the cognitive load prediction network to output the sample cognitive load result; the sample fatigue level features are input into the fatigue level prediction network to output the sample fatigue level result; and the sample emotional stability features are input into the emotional stability prediction network to output the sample emotional stability result. This allows for the accurate extraction of features in three dimensions: sample cognitive load features, sample fatigue level features, and sample emotional stability features. Thus, prediction results in three dimensions—sample cognitive load results, sample fatigue level results, and sample emotional stability results—are obtained, facilitating the training of the model's three-dimensional evaluation result prediction capability.

[0051] For example, the sample physiological signals include a first sample physiological signal, a second sample physiological signal, and a third sample physiological signal. These three samples, along with sample driving data, can be input into a pre-defined model. The pre-defined model can be a multimodal Transformer fusion model (MMST), where the original signals from the four modalities are mapped to a unified latent space using dedicated encoders: facial signals via a convolutional neural network (CNN), grip strength signals via a one-dimensional convolutional neural network (1D-CNN), heart rate signals via a long short-term memory network (LSTM), and vehicle dynamics signals via a multilayer perceptron (MLP). The encoded feature vectors are then fused across modalities using a multi-head cross-attention mechanism to output the sample multimodal fusion features, i.e., the shared latent state H(t) ∈ R. d .

[0052] In some embodiments, the orthogonal uncoupling projection of the multimodal fusion features of the samples to obtain sample cognitive load features, sample fatigue level features, and sample emotional stability features includes: Determine the first weight corresponding to the cognitive load of the sample, the second weight corresponding to the fatigue level of the sample, and the third weight corresponding to the emotional stability of the sample; Based on the multimodal fusion features of the sample and the first weight, the cognitive load features of the sample are determined; Based on the multimodal fusion features of the samples and the second weight, the fatigue level features of the samples are determined; Based on the multimodal fusion features of the sample and the third weight, the emotional stability features of the sample are determined.

[0053] In the embodiments of this specification, a first weight corresponding to the cognitive load of the sample, a second weight corresponding to the fatigue level of the sample, and a third weight corresponding to the emotional stability of the sample can be set respectively. The first, second, and third weights can all be linear projection matrices with a dimension of (1×d), mapping the shared latent state (multimodal fusion features of the sample) to scalar values ​​of the three evaluation dimensions, respectively. The three weights can be randomly initialized before model training begins, and automatically learned through backpropagation during training. The final convergence value is jointly determined by the training loss function (including orthogonalization constraints); thus, the importance of the three dimensional features can be accurately determined.

[0054] In some embodiments, the method further includes: Determine the first initial bias term corresponding to the cognitive load of the sample, the second initial bias term corresponding to the fatigue level of the sample, and the third initial bias term corresponding to the emotional stability of the sample; The step of determining the cognitive load feature of the sample based on the multimodal fusion feature of the sample and the first weight includes: calculating the first product of the multimodal fusion feature of the sample and the first weight, and determining the cognitive load feature of the sample based on the sum of the first product and the first initial bias term; The step of determining the sample fatigue degree feature based on the sample multimodal fusion feature and the second weight includes: calculating the second product of the sample multimodal fusion feature and the second weight, and determining the sample fatigue degree feature according to the sum of the second product and the second initial bias term; The step of determining the emotional stability feature of the sample based on the multimodal fusion feature of the sample and the third weight includes: calculating the third product of the multimodal fusion feature of the sample and the third weight, and determining the emotional stability feature of the sample based on the sum of the third product and the third initial bias term.

[0055] In the embodiments of this specification, a first initial bias term, a second initial bias term, and a third initial bias term can be initialized and adjusted during model training. This allows the bias terms to correct the features in the three dimensions, thereby improving the accuracy of the sample cognitive load features, sample fatigue level features, and sample emotional stability features.

[0056] In the embodiments described in this specification, such as Figure 3 As shown, Figure 3This is a schematic diagram of a three-dimensional evaluation result; continuous values ​​of the three evaluation dimensions can be calculated from the shared hidden state H(t) through three independent linear projection heads: CL(t) = σ(W_CL •H(t) + b_CL) Cognitive load ∈ [0,1] FL(t) = σ(W_FL •H(t) + b_FL) Fatigue level ∈ [0,1] ES(t) = σ(W_ES •H(t) + b_ES) Emotional stability ∈ [0,1] CSI(t) = [CL(t), FL(t), ES(t)] Wherein, (W_CL •H(t) + b_CL) corresponds to the sample cognitive load feature, (W_FL •H(t) + b_FL) corresponds to the sample fatigue level feature, and (W_ES •H(t) + b_ES) corresponds to the sample emotional stability feature. CL(t) is the sample cognitive load result, FL(t) is the sample fatigue level result, and ES(t) is the sample emotional stability result.

[0057] H(t): The shared hidden state vector output by MMST, with dimension d, which encodes the comprehensive features after multimodal fusion; W_CL, W_FL, W_ES: These correspond to the first, second, and third weights, respectively, and are three independent weight matrices with a dimension of (1×d). They are randomly initialized by the Xavier method before model training begins, automatically learned during training through the backpropagation algorithm, and the final convergence value is determined by the training loss function (including orthogonalization constraints).

[0058] b_CL, b_FL, b_ES: Three independent bias terms (scalars), corresponding to the first initial bias term, the second initial bias term, and the third initial bias term, respectively; initialized to 0, automatically learned by gradient descent during training, with no explicit constraints on the range of values, and the final value is determined by the overall loss function.

[0059] σ(•): Sigmoid activation function, which compresses the output to the interval [0, 1].

[0060] In some embodiments, training the preset model based on the differences between the sample cognitive load result and the sample cognitive load label, the differences between the sample fatigue level result and the sample fatigue level label, and the differences between the sample emotional stability result and the sample emotional stability label to obtain the multidimensional information evaluation model includes: The first loss data is determined based on the difference between the sample cognitive load results and the sample cognitive load labels; The second loss data is determined based on the difference between the sample fatigue level result and the sample fatigue level label; The third loss data is determined based on the difference between the sample emotional stability results and the sample emotional stability labels; The target loss data is determined based on the first loss data, the second loss data, and the third loss data; The model parameters of the preset model are adjusted according to the target loss data until the training termination condition is met, thereby obtaining the multidimensional information evaluation model.

[0061] In the embodiments of this specification, a first loss data can be determined by the difference between the sample cognitive load result and the sample cognitive load label; a second loss data can be determined by the difference between the sample fatigue level result and the sample fatigue level label; a third loss data can be determined by the difference between the sample emotional stability result and the sample emotional stability label; then, a target loss data can be determined based on the sum of the first loss data, the second loss data, and the third loss data; or, weights can be set for the first loss data, the second loss data, and the third loss data respectively, and then the weighted sum of the first loss data, the second loss data, and the third loss data can be calculated to obtain the target loss data; then, the model parameters of the preset model can be adjusted based on the target loss data until the training termination condition is met, thereby determining the target loss data based on the evaluation results of the three dimensions, and thus training the preset model to obtain a multidimensional information evaluation model that can accurately predict the evaluation results of the three dimensions.

[0062] In some embodiments, determining the target loss data based on the first loss data, the second loss data, and the third loss data includes: Based on the first loss data, the second loss data, and the third loss data, determine the multidimensional loss data; Based on the first weight corresponding to the cognitive load of the sample, the second weight corresponding to the fatigue level of the sample, and the third weight corresponding to the emotional stability of the sample, the orthogonalization constraint loss data of the weights is determined. Based on the sample cognitive load results, the sample fatigue level results, and the sample emotional stability results, statistically independent constrained loss data are determined. The target loss data is determined based on the multidimensional loss data, the orthogonalized constraint loss data, and the statistically independent constraint loss data.

[0063] In the embodiments described in this specification, to ensure that the three output dimensions are statistically independent, two constraint loss data can be introduced: orthogonalization constraint loss data and statistical independence constraint loss data. Specifically, these two constraint terms are added to the total loss function of the model, and they are optimized through training to approach zero.

[0064] In some embodiments, determining the target loss data based on the multidimensional loss data, the orthogonalization constraint loss data, and the statistical independence constraint loss data includes: Determine the first preset weight of the orthogonalized constraint loss data and the second preset weight of the statistically independent constraint loss data; Calculate the product of the orthogonalized constraint loss data and the first preset weight to obtain the first weighted loss data; Calculate the product of the statistically independent constraint loss data and the second preset weight to obtain the second weighted loss data; The target loss data is obtained by summing the first weighted loss data, the second weighted loss data, and the multidimensional loss data.

[0065] In the embodiments of this specification, a first preset weight λ1 for the orthogonalization constraint loss data and a second preset weight λ2 for the statistically independent constraint loss data can be set. For example, the values ​​are λ1∈[0.01, 0.1] and λ2∈[0.01, 0.1], specifically determined through grid search on the validation set. The setting principle is to minimize the two constraint losses while ensuring normal convergence of the supervised losses in each dimension. If the coefficients are too large, the orthogonalization constraint will suppress the supervised signal, leading to a decrease in prediction accuracy; if the coefficients are too small, the orthogonalization decoupling effect will be insufficient. When tuning the parameters, the constraint condition is that the prediction accuracy loss of each dimension on the validation set does not exceed 2%, and the coefficients are gradually increased until the orthogonality requirement is met (the absolute value of the Pearson correlation coefficient between any two dimensions after training is less than 0.15). Therefore, the weighted loss data of the two loss data can be calculated based on the two weights, thereby accurately obtaining the target loss data.

[0066] For example, such as Figure 4 As shown, Figure 4This is a flowchart of MMST fusion and orthogonal decoupling. First, sensors collect eye features, mouth features, head posture, and facial micro-expressions of the sample subject to obtain the first sample physiological signal. Then, the hand grip strength, grip strength distribution, and grip strength change frequency of the sample subject are collected to obtain the second sample physiological signal. Next, heart rate information and heart rate variability of the sample subject are collected to obtain the third sample physiological signal. Finally, vehicle speed, acceleration, steering wheel angle, and lane departure are collected to obtain the sample vehicle's driving data. Based on the first, second, and third sample physiological signals and the driving data, a multimodal signal is constructed. Different types of signal data are then input into corresponding feature encoding networks for feature encoding processing to obtain four types of encoded features. These four encoded features are then input into a multimodal shared feature network for feature fusion, followed by orthogonal decoupling projection to obtain sample cognitive load features, sample fatigue level features, and sample emotional stability features. Finally, corresponding evaluation results are output based on each feature, thus obtaining the sample cognitive load results, sample fatigue level results, and sample emotional stability results for the sample subject.

[0067] For example, the formula for calculating the target loss data is as follows:

[0068] in, The first loss function corresponds to the first loss data. The second loss function corresponds to the second loss data. The third loss function corresponds to the third loss data; The loss function corresponding to the orthogonalization constraint of the weights is the loss function for the loss data. This is the loss function corresponding to the statistically independent constraint loss data.

[0069] λ1 and λ2 are the weights corresponding to the two constraint losses, i.e., the balance coefficients. The sum of the two is not required to be equal to 1, and they take values ​​independently.

[0070] Constraint 1: Orthogonalization constraint on the loss function — Constraints at the parameter level The three weight matrices W_CL, W_FL, and W_ES are each responsible for extracting information from the shared hidden state. If the information extracted by the three matrices overlaps, the output dimensions will be coupled. The purpose is to constrain the three matrices to be "each independent and non-overlapping," which mathematically means that the inner product of each pair of the three matrices tends to zero.

[0071] Where W_CL, W_FL, and W_ES correspond to the first, second, and third weights, respectively, three independent weight matrices with a dimension of (1×d). ||·|_F represents the Frobenius norm (the square root of the sum of the squares of all elements of the matrix), and T represents the matrix transpose. When = 0, the three matrices are pairwise orthogonal, and the output of each dimension depends only on the non-overlapping feature components in the hidden state.

[0072] Constraint 2: Statistical Independent Constraint Loss Function — Constraints from the output level Even if the weights are orthogonal, the output may still have statistical correlations due to the data distribution during training. Its purpose is to constrain the outputs of the three dimensions to be statistically uncorrelated. Specifically, it calculates the covariance matrix Σ of the three dimensions of the output within a training batch and constrains its off-diagonal elements to tend to zero.

[0073] The covariance matrix Σ is a 3×3 matrix with the following meanings: Diagonal elements (e.g., Var(CL)): The variance of each dimension itself, reflecting the degree of fluctuation of that dimension, which does not require constraints.

[0074] Off-diagonal elements (e.g., Cov(CL,FL)): Covariance between two dimensions, measuring whether they "change together". Cov = 0 indicates that the two dimensions are uncorrelated.

[0075] Sum the squares of all off-diagonal elements (i.e., the covariance between each pair of dimensions), and minimize this value during training to force the outputs of the three dimensions to become uncorrelated:

[0076] The relationship between the two constraint terms is as follows: Constraining the projector to extract different information at the "model parameter" level The "output results" constraint ensures that the outputs are independent of each other, and that the two complement each other and are indispensable.

[0077] Orthogonality maintenance during the reasoning phase: When the system is running online, it calculates the Pearson correlation coefficient matrix of the CSI sequences within the window at preset time intervals (such as every 5 minutes) to detect whether there is a correlation between any two dimensions.

[0078] The formula for calculating the Pearson correlation coefficient is:

[0079] Where n is the number of sampling points within the window, and These are the means of the two dimensions within the current window, respectively. The value of r ranges from [-1, 1]: r = 0 indicates that the two dimensions are completely uncorrelated (ideal state), and the closer |r| is to 1, the more correlated they are. The system sets a threshold of 0.3, meaning that if |r| > 0.3 for any two dimensions, it indicates that the orthogonalization during the training phase has shifted during inference, and the current CSI output needs to be transformed into a decorrelation form using Cholesky whitening to restore the independence between dimensions. Cholesky whitening (Square-Root Whitening) is a whitening method that uses the Cholesky decomposition of a symmetric positive definite covariance matrix to convert correlated multidimensional random variables into whitening variables with a mean of 0 and a covariance of an identity matrix (uncorrelated and homoscedastic).

[0080] In the embodiments described in this specification, the features of the three dimensions are orthogonally decoupled and become independent of each other, allowing changes in each dimension to be independently observed and attributed. Based on this characteristic, each CSI dimension is equipped with an independent Thompson Sampling context bandpass, which performs state detection, intervention decision-making, and effect learning for that dimension. The learning processes of the three bandpasses do not interfere with each other, as the reward signal of each bandpass depends only on the CSI change value of the corresponding dimension and is unaffected by changes in other dimensions.

[0081] A bandpass for each dimension d ∈ {CL, FL, ES} consists of the following parts: Personalized trigger threshold θ_d(t): The initial value is set to the statistical quantile of that dimension on the training set (e.g., p75), and is adaptively adjusted according to the intervention frequency during online operation.

[0082] Beta probability distribution B(αd,a, βd,a): For each optional intervention action a ∈ A_d, a set of Beta distribution parameters is maintained. α and β record the historical number of positive rewards and negative rewards for the action, respectively. The larger α is, the better the action has historically performed.

[0083] Context feature vector x_d(t): contains the current CSI dimension value CSI_d(t) and the first difference. CSI_d(t) (reflects the rate of change), second-order difference ²CSI_d(t) (reflects changing acceleration), normalized vehicle speed, and time interval since the last intervention.

[0084] In some embodiments, determining the target intervention strategy for the target subject based on the cognitive load result, the fatigue level result, and the emotional stability result includes: Construct bandpasses corresponding to the three-dimensional results output by the multidimensional information assessment model, wherein each bandpass includes a preset trigger threshold corresponding to the intervention strategy; the three-dimensional results include the cognitive load result, the fatigue level result, and the emotional stability result. Based on the bandpass filter, any one of the cognitive load result, the fatigue level result, and the emotional stability result is used as the current prediction result; Obtain the preset trigger threshold of the bandpass corresponding to the current prediction result, and thus obtain the current trigger threshold; If the current prediction result is greater than the current trigger threshold, the current prediction result is determined to be an abnormal result, and the current intervention strategy for the dimension corresponding to the current prediction result is obtained; Based on the current intervention strategies corresponding to the cognitive load results, the fatigue level results, and the emotional stability results, the target intervention strategy is constructed and executed.

[0085] In the embodiments of this specification, if the current prediction result is less than or equal to the current trigger threshold, the current prediction result is determined to be a normal result. If the cognitive load result, the fatigue level result, and the emotional stability result are all normal results, then no intervention is required for the target object, nor is it necessary to determine an intervention strategy. If at least one of the cognitive load result, the fatigue level result, and the emotional stability result is an abnormal result, then a corresponding intervention strategy is formulated based on the abnormal result. Preset trigger thresholds can be set for each of the three dimensions of cognitive load result, fatigue level result, and emotional stability result, corresponding to a bandpass transducer. The preset trigger thresholds for each bandpass transducer can be the same or different. By comparing the current prediction result in each bandpass transducer with the preset trigger threshold, it is determined whether an intervention strategy needs to be triggered. The intervention strategy can include at least one intervention action. Therefore, based on the cognitive load result, the fatigue level result, the emotional stability result, and the preset trigger thresholds corresponding to each of the three dimensions, it is determined whether to trigger an intervention. If it is determined that an intervention needs to be triggered, a target intervention strategy matching the determination result is determined, thereby achieving accurate determination of the target intervention strategy by combining the three dimensions of cognitive load result, fatigue level result, and emotional stability result.

[0086] In some embodiments, the bandpass filter further includes a context feature vector and probability distribution parameters of the intervention strategy, and obtaining the current intervention strategy corresponding to the dimension of the current prediction result includes: Obtain at least two initial intervention strategies corresponding to the dimension of the current prediction result; The probability distribution parameters of each initial intervention strategy are adjusted according to the context feature vector to obtain the adjusted probability distribution parameters of each initial intervention strategy. Then, a random sample is performed on each initial intervention strategy according to the adjusted probability distribution parameters to obtain the sampled value of each initial intervention strategy. The strategy with the highest sample value is selected from the at least two initial intervention strategies as the current intervention strategy.

[0087] In the embodiments described in this specification, the context feature vector x_d(t) includes the current CSI dimension value CSI_d(t) and the first-order difference. CSI_d(t) (reflects the rate of change), second-order difference ²CSI_d(t) (reflecting changing acceleration), normalized vehicle speed, and the time interval since the last intervention. The probability distribution parameters of each initial intervention strategy can be adjusted based on the context feature vector to obtain the adjusted probability distribution parameters for each initial intervention strategy. Specifically, an offset is added to the historical statistical parameters α and β based on the current driving state (e.g., vehicle speed, time since the last intervention), and a value s_CL,a is sampled from the adjusted posterior distribution Beta(αCL,a,βCL,a). The action with the largest s_CL,a is selected as the action a_CL* to be executed in this round. In this way, the same intervention action will have different selection probabilities in different driving scenarios (e.g., a tendency to select actions with greater force when driving at high speeds). Finally, the strategy with the highest sampled value is selected from the at least two initial intervention strategies as the current intervention strategy, thereby improving the accuracy of intervention strategy selection.

[0088] In Thompson sampling, the system maintains a reward distribution based on historical data for each action.

[0089] Parameter meaning: α_CL,a usually represents the cumulative number of successful actions (or rewards), while β_CL,a represents the cumulative number of failures (or the number of times no reward was obtained).

[0090] The uncertainty reflects that as the observed data accumulates (i.e., the sum of α + β increases), the variance of the Beta distribution gradually decreases. This means that the uncertainty in our estimate of the true success rate of this action is decreasing, and the confidence level is increasing.

[0091] When faced with multiple candidate actions, the algorithm does not directly compare their average expected values. Instead, it performs a random sampling from the posterior distribution Beta(α_CL,a, β_CL,a) corresponding to each action to obtain its respective sampled value s_CL,a. For actions that perform well but have limited data, their Beta distribution is wider, and the sampled values ​​may fluctuate significantly.

[0092] For actions with stable performance and large datasets, their Beta distribution is highly concentrated, and the sampled values ​​remain very stable around their true mean. After all actions have completed one round of sampling, the system compares these sampled values ​​s_CL,a and selects the action with the highest sampled value for execution.

[0093] This mechanism makes the system tend to select the preferred action (taking advantage of known advantages); at the same time, due to the randomness of sampling, actions with small data volume and high uncertainty still have a chance to be selected due to a higher sample value by chance (maintaining the exploration of potential better options).

[0094] After executing the selected action and receiving actual environmental feedback (success or failure), the system records the result and increments the corresponding action parameter α_CL,a or β_CL,a by one. Subsequently, the next round of decision-making will resample and select based on this updated posterior distribution, iterating in this way until convergence to the optimal policy.

[0095] In some embodiments, the method further includes: Timing begins when the execution of the target intervention strategy is detected to have ended. If the cumulative duration reaches a preset duration threshold, updated cognitive load results, updated fatigue level results, and updated emotional stability results are obtained. Based on the difference between the cognitive load result and the updated cognitive load result, calculate the first reward data for the cognitive load dimension; and update the probability distribution parameters of the cognitive load dimension based on the first reward data. Based on the difference between the fatigue level result and the updated fatigue level result, calculate the second reward data for the fatigue level dimension; and update the probability distribution parameters of the fatigue level dimension based on the second reward data. Based on the difference between the emotional stability result and the updated emotional stability result, a third reward data for the emotional stability dimension is calculated; and the probability distribution parameters of the emotional stability dimension are updated based on the third reward data.

[0096] In the embodiments of this specification, reward data can be calculated separately for the evaluation results of each dimension. Specifically, first reward data for the cognitive load dimension can be calculated, and the probability distribution parameters of the cognitive load dimension can be updated based on the first reward data. Second reward data for the fatigue level dimension can be calculated, and the probability distribution parameters of the fatigue level dimension can be updated based on the second reward data. Third reward data for the emotional stability dimension can be calculated, and the probability distribution parameters of the emotional stability dimension can be updated based on the third reward data. For example, after the intervention is performed, a preset observation window Δt (30 seconds) is waited, and a new CSI value CSI(t+Δt) is obtained. The specific reward r_CL for the cognitive load dimension is calculated as r_CL = CL(t) - CL(t+Δt). This reward value only reflects the change in the cognitive load dimension. Since orthogonal decoupling ensures the independence of CL from FL and ES, changes in FL and ES will not be mixed into r_CL. After binarizing r_CL, update the Beta distribution parameters of action a_CL: α_CL,a increments by 1 when there is a positive reward (r_CL>δ), and β_CL,a* increments by 1 when there is a negative reward; thus, continue to the next loop and re-perform anomaly detection.

[0097] In an exemplary embodiment, taking the cognitive load dimension as an example, the processing flow for each decision cycle is as follows: Step 1 — Anomaly Detection: Compare CL(t) with θCL(t). If CL(t) ≤ θCL(t), do not intervene and end the current round of processing for this dimension. If CL(t) > θCL(t), proceed to Step 2.

[0098] Step 2 — Action Selection: For each intervention action a∈A_CL applicable to the cognitive load dimension (including reducing information density, simplifying HUD display, ACC automatic speed reduction, etc.), firstly, the Beta distribution parameters of the action are conditionally adjusted based on the current context feature vector x_d(t)—that is, an offset is added to the historical statistical parameters α and β according to the current driving state (such as vehicle speed, time since the last intervention, etc.). Then, a value s_CL,a is sampled from the adjusted posterior distribution Beta(αCL,a,βCL,a), and the action with the largest s_CL,a is selected as the action a_CL* to be executed in this round. In this way, the same intervention action will have different probabilities of being selected in different driving scenarios (for example, when driving at high speed, a stronger action is preferred).

[0099] Step 3 — Joint Execution: The actions selected by the three dimensions that trigger the anomaly at the same time are combined and executed according to the preset priority (priority order: ES>FL>CL, and the final intervention intensity is the highest level selected by each dimension).

[0100] Step 4 — Reward Observation: After the intervention is implemented, wait for a preset observation window Δt (30 seconds) and obtain a new CSI value CSI(t+Δt). Calculate the specific reward for the cognitive load dimension r_CL = CL(t) - CL(t+Δt). This reward value only reflects changes in the cognitive load dimension. Due to orthogonal decoupling, CL is independent of FL and ES, and changes in FL and ES will not be mixed into r_CL.

[0101] Step 5 — Parameter Update: After binarizing r_CL, update the Beta distribution parameters of action a_CL: α_CL,a increments by 1 when there is a positive reward (r_CL>δ), and β_CL,a* increments by 1 when there is a negative reward.

[0102] For example, the intervention strategies (intervention actions) for each dimension are shown in Table 2 below: Table 2

[0103] In some embodiments, the method further includes: If the number of abnormal results detected in the current dimension within a unit of time exceeds the target number, the preset trigger threshold corresponding to the current dimension is increased; the current dimension is at least one of the cognitive load dimension, fatigue level dimension, and emotional stability dimension. If the ratio of the number of occurrences of abnormal results in the current dimension to the target number within a unit of time is less than a preset ratio, the preset trigger threshold corresponding to the current dimension is reduced.

[0104] In the embodiments of this specification, for any one of the cognitive load dimension, fatigue level dimension, and emotional stability dimension, the number of times abnormal results occur in the current dimension within a unit of time is detected. If the number of occurrences is greater than the target number, the preset trigger threshold corresponding to the current dimension is increased; if the ratio of the number of occurrences to the target number is less than the preset ratio, the preset trigger threshold corresponding to the current dimension is decreased; that is, the threshold is updated independently for each bandpass. If the trigger frequency of a certain dimension exceeds the preset target value (e.g., 3 times per hour) within a unit of time, the threshold is increased by 5%; if the trigger frequency is less than 50% of the target value, the threshold is decreased by 5%.

[0105] For example, such as Figure 5 As shown, Figure 5 The diagram illustrates the learning process of three bandpasses corresponding to three dimensions. Independent bandpasses are set for the three dimensions of cognitive load, fatigue, and emotional stability, and priorities are set for each dimension. Emotional stability has a higher priority than fatigue, and fatigue has a higher priority than emotional stability. Orthogonality constraints are used to ensure that each bandpass learns independently.

[0106] In some embodiments, the targeted intervention strategy includes a first intervention strategy based on the cognitive load dimension, a second intervention strategy based on the fatigue level dimension, and a third intervention strategy based on the emotional stability dimension. The method further includes: Obtain the first intervention level of the first intervention strategy, the second intervention level of the second intervention strategy, and the third intervention level of the third intervention strategy; The initial intervention level is determined based on the orthogonal results of the first intervention level and the second intervention level; Based on the third intervention level and the emotional stability results, the initial intervention level is revised to obtain the target intervention level; Obtain the intervention content corresponding to the target intervention level and execute the intervention action corresponding to the intervention content.

[0107] In this embodiment, an initial intervention level can be determined first based on the orthogonal results of the first intervention level of the first intervention strategy and the second intervention level of the second intervention strategy. Then, the initial intervention level is corrected based on the third intervention level of the emotional stability dimension and the emotional stability results to obtain the target intervention level, thereby improving the accuracy of the target intervention level. In this embodiment, the intervention level is jointly determined by three orthogonal CSI dimensions, rather than by a single-dimensional threshold comparison. The base level is determined by the value ranges of the CL and FL dimensions through a rule matrix: Rule matrix: CL range × FL range → Base intervention level. As shown in Table 3 below: Table 3

[0108] In the embodiments described in this specification, the ES dimension is used as a correction factor to adjust the base level. ES stands for Emotional Stability Score, with a value range of [0, 1], where a higher value indicates greater emotional stability.

[0109] When ES ≥ 0.6: The level remains unchanged (emotional stability). When ES ∈ [0.3, 0.6): Base level +1 (emotional fluctuations, requiring stronger intervention) When ES < 0.3: Base level +2 (emotional breakdown, requiring strong intervention) When ES < 0.1: directly enter Level 4 (emergency situation).

[0110] In some embodiments, such as Figure 6 As shown, Figure 6This diagram illustrates a five-level progressive intervention strategy, which includes a gradual escalation rule: a maximum of one level can be increased per assessment cycle. Each level must last for at least two assessment cycles (4 seconds). If any assessment shows that the CSI value falls below the trigger range of the current level, the escalation process is paused. A rapid descent rule applies: when all three CSI dimensions simultaneously return to the normal range (each dimension < 0.3) and remain so for 30 seconds, the level is directly reduced to Level 0. Due to the orthogonality of the three dimensions, simultaneous return to normal across all dimensions confirms a full recovery of the driver's condition. Dimension-specific downgrading occurs when only some dimensions return to normal, stopping only the intervention for those dimensions while continuing the selected interventions for the other dimensions. The overall level is recalculated based on the dimensions still in an abnormal state. The conditions and intervention content corresponding to each of the five intervention strategies are shown in Table 4 below.

[0111] Table 4

[0112] For example, such as Figure 7 As shown, Figure 7 This diagram illustrates online learning and personalized optimization of intervention effects. The system records complete data for each intervention event (pre-intervention CSI, post-intervention CSI changes, and driver behavior changes). Through online learning, it optimizes personalized CSI thresholds, intervention method preferences, minimizes false alarm rates, and ensures safety redundancy. A reward function can be set for each intervention strategy (intervention event), and the optimal intervention action can be determined using this reward function.

[0113] For example, such as Figure 8 As shown, Figure 8 This is a safety linkage flowchart for autonomous driving; when the system enters Level 3 or Level 4, it automatically sends a safety status signal (Safety_Signal) to the autonomous driving control system:

[0114] in, The field identifies the primary CSI dimension that triggers the current intervention. The autonomous driving system executes anomaly-specific intervention strategies (response strategies) based on this field; for example, as shown in Table 5 below, Table 5 shows the correspondence between various anomaly dimensions and intervention strategies.

[0115] Table 5

[0116] As can be seen from the technical solutions provided in the embodiments of this specification above, these embodiments decompose the driver's state into three statistically independent continuous value evaluation dimensions: cognitive load, fatigue level, and emotional stability, through weighted orthogonal constraints and covariance diagonal constraints. Each orthogonally decoupled CSI dimension is equipped with an independent Thompson Sampling context bandpass, and the reward signal of each bandpass depends only on the CSI change value of the corresponding dimension. Orthogonal decoupling ensures the independent observability of changes in each dimension, making the dimension-specific reward signal unaffected by cross-dimensional coupling interference, and enabling correct convergence of online learning. A two-dimensional interval rule matrix using CL and FL is used to determine the basic intervention level, with ES as a correction factor for adjustment. Compared to the one-dimensional mapping from a single threshold to the intervention level in existing solutions, the three-dimensional joint rule matrix of this solution supports more refined intervention decisions and supports dimension-specific degradation—when some dimensions return to normal, only the corresponding intervention is stopped, while other dimensions continue to be executed. The system automatically learns each driver's personalized threshold and intervention strategy preferences by recording the effect data of each intervention. An abnormal dimension identification field is added to the safety signal, utilizing the dimension-specific information from orthogonal decoupling to enable the autonomous driving system to make targeted responses. Among them, the multimodal fusion method reduces the risk of single sensor failure and improves the robustness of driver status monitoring; orthogonal decoupling eliminates interdimensional coupling, making changes in each dimension independently observable and attributable; dimension-specific online learning rewards pure signals, with each learner not interfering with the others, and convergence efficiency is better than that of coupled systems; the three-dimensional joint rule matrix supports refined intervention decisions and dimension-specific degradation; and abnormal dimension identification makes autonomous driving linkage response more targeted.

[0117] This specification also provides an intervention strategy determination device, such as... Figure 9 As shown, the device includes: The data acquisition module 910 is used to acquire the target physiological signals of the target object and the target driving data of the target vehicle during the process of the target object driving the target vehicle; The feature fusion module 920 is used to input the target physiological signal and the target driving data into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain the target multi-modal fusion features; The feature parsing module 930 is used to perform orthogonal uncoupling projection on the target multimodal fusion features to obtain target cognitive load features, target fatigue level features and target emotional stability features; The prediction module 940 is used to predict the cognitive load result of the target object based on the target cognitive load characteristics, predict the fatigue result of the target object based on the target fatigue characteristics, and predict the emotional stability result of the target object based on the target emotional stability characteristics, and output the cognitive load result, the fatigue result, and the emotional stability result; The intervention strategy determination module 950 is used to determine the target intervention strategy for the target object based on the cognitive load result, the fatigue level result and the emotional stability result.

[0118] In one exemplary embodiment, the data acquisition module is further configured to collect the eye features, mouth features, head posture, and facial micro-expressions of the target object to obtain a first physiological signal; collect the grip strength, grip strength distribution information, and grip strength change frequency of the target object's hands to obtain a second physiological signal; collect the heart rate information and heart rate variability of the target object to obtain a third physiological signal; determine the target physiological signal of the target object based on at least one of the first physiological signal, the second physiological signal, and the third physiological signal; and collect the vehicle speed, acceleration, steering wheel angle, and lane departure of the target vehicle to obtain the target driving data of the target vehicle.

[0119] In one exemplary embodiment, the apparatus further includes: The sample data acquisition module is used to acquire the physiological signals of the sample subjects during the driving of the sample vehicle and the driving data of the sample vehicle; the sample subjects include sample cognitive load labels, sample fatigue level labels and sample emotional stability labels. The sample input module is used to input the sample physiological signals and the sample driving data into a preset model, and output the sample cognitive load results, sample fatigue results, and sample emotional stability results of the sample object. The model training module is used to train the preset model based on the differences between the sample cognitive load results and the sample cognitive load labels, the differences between the sample fatigue results and the sample fatigue labels, and the differences between the sample emotional stability results and the sample emotional stability labels, so as to obtain the multidimensional information evaluation model.

[0120] In one exemplary embodiment, the sample input module is further configured to input the sample physiological signal and the sample driving data into a preset model, perform feature encoding processing on the sample physiological signal to obtain sample physiological signal encoded features, and perform feature encoding processing on the sample driving data to obtain sample driving data encoded features; perform cross-modal fusion processing on the sample physiological signal encoded features and the sample driving data encoded features to obtain sample multimodal fusion features; perform orthogonal decoupling projection on the sample multimodal fusion features to obtain sample cognitive load features, sample fatigue level features, and sample emotional stability features; predict the sample cognitive load result based on the sample cognitive load features, predict the sample fatigue level result based on the sample fatigue level features, and predict the sample emotional stability result based on the sample emotional stability features.

[0121] In one exemplary embodiment, the feature parsing module is further configured to determine a first weight corresponding to the sample cognitive load, a second weight corresponding to the sample fatigue level, and a third weight corresponding to the sample emotional stability; determine the sample cognitive load feature based on the sample multimodal fusion feature and the first weight; determine the sample fatigue level feature based on the sample multimodal fusion feature and the second weight; and determine the sample emotional stability feature based on the sample multimodal fusion feature and the third weight.

[0122] In one exemplary embodiment, the apparatus further includes: The bias term determination module is used to determine the first initial bias term corresponding to the cognitive load of the sample, the second initial bias term corresponding to the fatigue level of the sample, and the third initial bias term corresponding to the emotional stability of the sample. The feature parsing module is further configured to calculate the first product of the sample multimodal fusion feature and the first weight, and determine the sample cognitive load feature based on the sum of the first product and the first initial bias term; calculate the second product of the sample multimodal fusion feature and the second weight, and determine the sample fatigue level feature based on the sum of the second product and the second initial bias term; calculate the third product of the sample multimodal fusion feature and the third weight, and determine the sample emotional stability feature based on the sum of the third product and the third initial bias term.

[0123] In one exemplary embodiment, the model training module is further configured to: determine first loss data based on the difference between the sample cognitive load result and the sample cognitive load label; determine second loss data based on the difference between the sample fatigue level result and the sample fatigue level label; determine third loss data based on the difference between the sample emotional stability result and the sample emotional stability label; determine target loss data based on the first loss data, the second loss data, and the third loss data; and adjust the model parameters of the preset model based on the target loss data until the training termination condition is met, thereby obtaining the multidimensional information evaluation model.

[0124] In one exemplary embodiment, the model training module is further configured to: determine multidimensional loss data based on the first loss data, the second loss data, and the third loss data; determine orthogonalized constraint loss data based on the first weight corresponding to the sample cognitive load, the second weight corresponding to the sample fatigue level, and the third weight corresponding to the sample emotional stability; determine statistically independent constraint loss data based on the sample cognitive load result, the sample fatigue level result, and the sample emotional stability result; and determine the target loss data based on the multidimensional loss data, the orthogonalized constraint loss data, and the statistically independent constraint loss data.

[0125] In one exemplary embodiment, the model training module is further configured to: determine a first preset weight of the orthogonalized constraint loss data and a second preset weight of the statistically independent constraint loss data; calculate the product of the orthogonalized constraint loss data and the first preset weight to obtain first weighted loss data; calculate the product of the statistically independent constraint loss data and the second preset weight to obtain second weighted loss data; and calculate the sum of the first weighted loss data, the second weighted loss data, and the multidimensional loss data to obtain the target loss data.

[0126] In one exemplary embodiment, the intervention strategy determination module is further configured to construct bandpasses corresponding to the three-dimensional results output by the multidimensional information assessment model, wherein each bandpass includes a preset trigger threshold corresponding to the intervention strategy; the three-dimensional results include the cognitive load result, the fatigue level result, and the emotional stability result; based on the bandpass, any one of the cognitive load result, the fatigue level result, and the emotional stability result is used as the current prediction result; the preset trigger threshold of the bandpass corresponding to the current prediction result is obtained to obtain the current trigger threshold; if the current prediction result is greater than the current trigger threshold, the current prediction result is determined to be an abnormal result, and the current intervention strategy corresponding to the dimension of the current prediction result is obtained; based on the current intervention strategies corresponding to the cognitive load result, the fatigue level result, and the emotional stability result, the target intervention strategy is constructed and executed.

[0127] In one exemplary embodiment, the bandpass converter further includes a context feature vector and probability distribution parameters of the intervention strategy. The intervention strategy determination module is further configured to obtain at least two initial intervention strategies corresponding to the dimension of the current prediction result; adjust the probability distribution parameters of each initial intervention strategy according to the context feature vector to obtain adjusted probability distribution parameters of each initial intervention strategy; perform a random sampling on each initial intervention strategy according to the adjusted probability distribution parameters to obtain a sampled value of each initial intervention strategy; and select the strategy with the highest sampled value from the at least two initial intervention strategies as the current intervention strategy.

[0128] In one exemplary embodiment, the apparatus further includes: The update result acquisition module is used to start timing when the execution of the target intervention strategy is detected to have ended. If the cumulative duration reaches a preset duration threshold, it acquires updated cognitive load results, updated fatigue level results, and updated emotional stability results. The first reward calculation module is used to calculate the first reward data of the cognitive load dimension based on the difference between the cognitive load result and the updated cognitive load result; and update the probability distribution parameters of the cognitive load dimension based on the first reward data. The second reward calculation module is used to calculate second reward data for the fatigue level dimension based on the difference between the fatigue level result and the updated fatigue level result; and to update the probability distribution parameters of the fatigue level dimension based on the second reward data. The third reward calculation module is used to calculate the third reward data of the emotion stability dimension based on the difference between the emotion stability result and the updated emotion stability result; and to update the probability distribution parameters of the emotion stability dimension based on the third reward data.

[0129] In one exemplary embodiment, the apparatus further includes: The threshold increase module is used to increase the preset trigger threshold corresponding to the current dimension if the number of times the abnormal result of the current dimension occurs within a unit time is greater than the target number; the current dimension is at least one of the cognitive load dimension, fatigue level dimension, and emotional stability dimension. The threshold reduction module is used to reduce the preset trigger threshold corresponding to the current dimension if the ratio of the number of occurrences of abnormal results in the current dimension per unit time to the target number is less than a preset ratio.

[0130] In one exemplary embodiment, the target intervention strategy includes a first intervention strategy based on the cognitive load dimension, a second intervention strategy based on the fatigue level dimension, and a third intervention strategy based on the emotional stability dimension. The device further includes: An intervention level acquisition module is used to acquire the first intervention level of the first intervention strategy, the second intervention level of the second intervention strategy, and the third intervention level of the third intervention strategy; The target level determination module is used to determine an initial intervention level based on the orthogonal result of the first intervention level and the second intervention level; and to revise the initial intervention level based on the third intervention level and the emotional stability result to obtain the target intervention level. The intervention action execution module is used to obtain the intervention content corresponding to the target intervention level and execute the intervention action corresponding to the intervention content.

[0131] In one exemplary embodiment, the sample data acquisition module is further configured to collect eye features, mouth features, head posture, and facial micro-expressions of the sample object to obtain a first sample physiological signal; collect the grip strength, grip strength distribution information, and grip strength change frequency of the sample object to obtain a second sample physiological signal; collect the heart rate information and heart rate variability of the sample object to obtain a third sample physiological signal; determine the sample physiological signal of the sample object based on at least one of the first sample physiological signal, the second sample physiological signal, and the third sample physiological signal; and collect the vehicle speed, acceleration, steering wheel angle, and lane departure of the sample vehicle to obtain sample driving data of the sample vehicle.

[0132] The apparatus and method embodiments described herein are based on the same inventive concept.

[0133] This specification provides an intervention strategy determination device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the intervention strategy determination method provided in the above method embodiments.

[0134] Embodiments of the present invention also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing an intervention strategy determination method in the method embodiments. The at least one instruction or at least one program is loaded and executed by the processor to implement the intervention strategy determination method provided in the above method embodiments.

[0135] Embodiments of the present invention also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the intervention strategy determination method provided in the above-described method embodiments.

[0136] Optionally, in the embodiments of this specification, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0137] The memory described in the embodiments of this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0138] The intervention strategy determination method embodiments provided in this specification can be executed on a mobile terminal, computer terminal, server, or similar computing device. Taking a server as an example... Figure 10This is a hardware structure block diagram of a server for an intervention strategy determination method provided in an embodiment of this specification. Figure 10 As shown, the server 1000 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1010 (CPUs 1010 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1030 for storing data, and one or more storage media 1020 (e.g., one or more mass storage devices) for storing application programs 1023 or data 1022. The memory 1030 and storage media 1020 may be temporary or persistent storage. The program stored in the storage media 1020 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 1010 may be configured to communicate with the storage media 1020 and execute the series of instruction operations in the storage media 1020 on the server 1000. Server 1000 may also include one or more power supplies 1060, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1040, and / or one or more operating systems 1021, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0139] The input / output interface 1040 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1000. In one example, the input / output interface 1040 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 1040 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0140] Those skilled in the art will understand that Figure 10 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 1000 may also include... Figure 10 The more or fewer components shown, or having the same Figure 10 The different configurations shown.

[0141] As can be seen from the embodiments of the intervention strategy determination method, apparatus, device, or storage medium provided by the present invention, during the process of a target object driving a target vehicle, the present invention acquires the target physiological signals of the target object and the target driving data of the target vehicle; then, the target physiological signals and the target driving data are input into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain target multi-modal fusion features; then, the target multi-modal fusion features are orthogonally decoupled and projected to obtain three-dimensional features: target cognitive load features, target fatigue level features, and target emotional stability features; thus facilitating further multi-dimensional evaluation results; next, the cognitive load result of the target object is predicted based on the target cognitive load features, the fatigue level result of the target object is predicted based on the target fatigue level features, and the emotional stability result of the target object is predicted based on the target emotional stability features, and the cognitive load result, the fatigue level result, and the emotional stability result are output, thereby obtaining evaluation results of three dimensions: cognitive load, fatigue level, and emotional stability; finally, based on the cognitive load result, the fatigue level result, and the emotional stability result, a target intervention strategy is formulated for the target object from the three dimensions of cognitive load, fatigue level, and emotional stability, improving the accuracy of the target intervention strategy.

[0142] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0143] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0144] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining intervention strategies, characterized in that, The method includes: During the process of the target object driving the target vehicle, the target object's target physiological signals and the target vehicle's target driving data are acquired. The target physiological signals and the target driving data are input into a multidimensional information evaluation model for cross-modal fusion processing to obtain the target multimodal fusion features; Orthogonal uncoupling projection is performed on the multimodal fusion features of the target to obtain the target cognitive load features, target fatigue level features, and target emotional stability features; The cognitive load result of the target object is predicted based on the target cognitive load characteristics, the fatigue result of the target object is predicted based on the target fatigue degree characteristics, and the emotional stability result of the target object is predicted based on the target emotional stability characteristics. The cognitive load result, the fatigue degree result, and the emotional stability result are then output. Based on the cognitive load results, the fatigue level results, and the emotional stability results, a targeted intervention strategy is determined for the target subject.

2. The method according to claim 1, characterized in that, The step of determining a targeted intervention strategy for the target subject based on the cognitive load results, the fatigue level results, and the emotional stability results includes: Construct bandpasses corresponding to the three-dimensional results output by the multidimensional information assessment model, wherein each bandpass includes a preset trigger threshold corresponding to the intervention strategy; the three-dimensional results include the cognitive load result, the fatigue level result, and the emotional stability result. Based on the bandpass filter, any one of the cognitive load result, the fatigue level result, and the emotional stability result is used as the current prediction result; Obtain the preset trigger threshold of the bandpass corresponding to the current prediction result, and thus obtain the current trigger threshold; If the current prediction result is greater than the current trigger threshold, the current prediction result is determined to be an abnormal result, and the current intervention strategy for the dimension corresponding to the current prediction result is obtained; Based on the current intervention strategies corresponding to the cognitive load results, the fatigue level results, and the emotional stability results, the target intervention strategy is constructed and executed.

3. The method according to claim 2, characterized in that, The bandpass also includes the context feature vector and probability distribution parameters of the intervention strategy, and obtaining the current intervention strategy corresponding to the dimension of the current prediction result includes: Obtain at least two initial intervention strategies corresponding to the dimension of the current prediction result; The probability distribution parameters of each initial intervention strategy are adjusted according to the context feature vector to obtain the adjusted probability distribution parameters of each initial intervention strategy. Then, a random sample is performed on each initial intervention strategy according to the adjusted probability distribution parameters to obtain the sampled value of each initial intervention strategy. The strategy with the highest sample value is selected from the at least two initial intervention strategies as the current intervention strategy.

4. The method according to claim 2, characterized in that, The method further includes: Timing begins when the execution of the target intervention strategy is detected to have ended. If the cumulative duration reaches a preset duration threshold, updated cognitive load results, updated fatigue level results, and updated emotional stability results are obtained. Based on the difference between the cognitive load result and the updated cognitive load result, calculate the first reward data for the cognitive load dimension; and update the probability distribution parameters of the cognitive load dimension based on the first reward data. Based on the difference between the fatigue level result and the updated fatigue level result, calculate the second reward data for the fatigue level dimension; and update the probability distribution parameters of the fatigue level dimension based on the second reward data. Based on the difference between the emotional stability result and the updated emotional stability result, a third reward data for the emotional stability dimension is calculated; and the probability distribution parameters of the emotional stability dimension are updated based on the third reward data.

5. The method according to claim 4, characterized in that, The method further includes: If the number of abnormal results detected in the current dimension within a unit of time exceeds the target number, the preset trigger threshold corresponding to the current dimension is increased; the current dimension is at least one of the cognitive load dimension, fatigue level dimension, and emotional stability dimension. If the ratio of the number of occurrences of abnormal results in the current dimension to the target number within a unit of time is less than a preset ratio, the preset trigger threshold corresponding to the current dimension is reduced.

6. The method according to claim 2, characterized in that, The targeted intervention strategies include a first intervention strategy based on the cognitive load dimension, a second intervention strategy based on the fatigue level dimension, and a third intervention strategy based on the emotional stability dimension. The method further includes: Obtain the first intervention level of the first intervention strategy, the second intervention level of the second intervention strategy, and the third intervention level of the third intervention strategy; The initial intervention level is determined based on the orthogonal results of the first intervention level and the second intervention level; Based on the third intervention level and the emotional stability results, the initial intervention level is revised to obtain the target intervention level; Obtain the intervention content corresponding to the target intervention level and execute the intervention action corresponding to the intervention content.

7. The method according to claim 1, characterized in that, The training method for the multidimensional information evaluation model includes: The sample physiological signals of the sample subjects during the driving of the sample vehicle and the sample driving data of the sample vehicle are obtained; the sample subjects include sample cognitive load labels, sample fatigue level labels and sample emotional stability labels. The sample physiological signals and the sample driving data are input into a preset model, and the sample cognitive load results, sample fatigue results, and sample emotional stability results of the sample object are output. Based on the differences between the sample cognitive load results and the sample cognitive load labels, the differences between the sample fatigue level results and the sample fatigue level labels, and the differences between the sample emotional stability results and the sample emotional stability labels, the preset model is trained to obtain the multidimensional information evaluation model.

8. The method according to claim 7, characterized in that, The process of inputting the sample physiological signals and the sample driving data into a preset model and outputting the sample cognitive load results, sample fatigue level results, and sample emotional stability results of the sample subjects includes: The sample physiological signal and the sample driving data are input into a preset model. The sample physiological signal is processed by feature encoding to obtain the sample physiological signal encoding feature, and the sample driving data is processed by feature encoding to obtain the sample driving data encoding feature. Cross-modal fusion processing is performed on the physiological signal encoding features and driving data encoding features of the samples to obtain multimodal fusion features of the samples; Orthogonal uncoupling projection is performed on the multimodal fusion features of the samples to obtain the sample cognitive load features, sample fatigue level features, and sample emotional stability features; The cognitive load result of the sample is predicted based on the cognitive load characteristics of the sample, the fatigue level result of the sample is predicted based on the fatigue level characteristics of the sample, and the emotional stability result of the sample is predicted based on the emotional stability characteristics of the sample.

9. The method according to claim 8, characterized in that, The orthogonal uncoupling projection of the multimodal fusion features of the samples yields sample cognitive load features, sample fatigue level features, and sample emotional stability features, including: Determine the first weight corresponding to the cognitive load of the sample, the second weight corresponding to the fatigue level of the sample, and the third weight corresponding to the emotional stability of the sample; Based on the multimodal fusion features of the sample and the first weight, the cognitive load features of the sample are determined; Based on the multimodal fusion features of the samples and the second weight, the fatigue level features of the samples are determined; Based on the multimodal fusion features of the sample and the third weight, the emotional stability features of the sample are determined.

10. An intervention strategy determination device, characterized in that, The device includes: The data acquisition module is used to acquire the target physiological signals of the target object and the target driving data of the target vehicle during the process of the target object driving the target vehicle; The feature fusion module is used to input the target physiological signal and the target driving data into a multi-dimensional information evaluation model for cross-modal fusion processing to obtain the target multi-modal fusion features; The feature parsing module is used to perform orthogonal uncoupling projection on the multimodal fusion features of the target to obtain the target cognitive load features, the target fatigue level features, and the target emotional stability features. The prediction module is used to predict the cognitive load result of the target object based on the target cognitive load characteristics, predict the fatigue result of the target object based on the target fatigue characteristics, and predict the emotional stability result of the target object based on the target emotional stability characteristics, and output the cognitive load result, the fatigue result, and the emotional stability result; The intervention strategy determination module is used to determine the target intervention strategy for the target object based on the cognitive load result, the fatigue level result, and the emotional stability result.

11. An intervention strategy determination device, characterized in that, The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the intervention strategy determination method as described in any one of claims 1-9.

12. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the intervention strategy determination method as described in any one of claims 1-9.

13. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the intervention strategy determination method according to any one of claims 1-9.

Images