System for predicting one or more specific cognitive states based on non-neurophysiological data

Abstract

A system for predicting one or more particular cognitive states of an individual based on non-neurophysiological data collected by one or more non-neurophysiological sensors includes one or more controllers in electronic communication with the one or more non-neurophysiological sensors. The one or more controllers include a physiological data-based neural network, a neural data-based neural network, and an encoder-decoder that learns a transformation between a hidden layer of the physiological data-based neural network and a hidden layer of the neural data-based neural network during a system training phase.

Description

Technical Field

[0001] This disclosure relates to a system for predicting one or more specific cognitive states of an individual based solely on non-neurophysiological data, wherein the system is trained on both non-neurophysiological and neural data. Background Technology

[0002] Direct neural data recorded from the human brain is the most reliable way to detect an individual's cognitive state. The most common and least invasive method for recording neural data is electroencephalography (EEG). An EEG cap consists of multiple electrodes attached to the individual's scalp to monitor neural signals generated by the brain. Another method for detecting an individual's cognitive state relies on sensors that monitor non-neurophysiological data, such as, but not limited to, heart rate, eye tracking, blood flow, and skin conductance. Current state-of-the-art methods for detecting an individual's cognitive state may rely solely on neural data, solely on non-neurophysiological data, or a combination thereof.

[0003] While studies relying solely on EEG data have shown relatively high accuracy in predicting an individual's cognitive state and stress, EEG caps are invasive and impractical for everyday use. Although collecting non-neurophysiological data is less invasive, studies relying solely on non-neurophysiological data are often limited, potentially only detecting extreme stress or no stress at all, and may lack the ability to achieve relatively high accuracy in more nuanced predictions. Studies relying on both EEG and non-neurophysiological data share the same drawbacks as studies relying solely on EEG data and requiring additional physiological sensors, and produce only relatively small performance improvements compared to studies relying solely on EEG data. Attempts have been made to determine the conversion between non-neurophysiological and EEG data; however, these attempts have generally been unsuccessful.

[0004] Therefore, while current methods for detecting an individual's cognitive state have achieved their intended purpose, there is still a need in the field for an improved method to determine the conversion between non-neurophysiological data and EEG data. Summary of the Invention

[0005] According to several aspects, a system for predicting one or more specific cognitive states of an individual based on non-neurophysiological data collected by one or more non-neurophysiological sensors is disclosed. The system includes a controller that electronically communicates with one or more non-neurophysiological sensors. The one or more controllers include one or more processors that execute instructions to receive non-neurophysiological data by a data-based neural network, wherein the data-based neural network is trained based on non-neurophysiological training data collected by the one or more non-neurophysiological sensors during a system training phase. In response to receiving non-neurophysiological data from the one or more non-neurophysiological sensors, the data-based neural network predicts one or more intermediate specific cognitive states of the individual based on the non-neurophysiological data. Hidden layers of the data-based neural network, created when the data-based neural network predicts one or more intermediate specific cognitive states of the individual, are received via an encoder-decoder. The encoder-decoder predicts hidden layers of the neural data-based neural network based on the hidden layers of the data-based neural network for one or more intermediate specific cognitive states of the individual. The encoder-decoder transmits the hidden layers of the neural data-based neural network to the neural data-based neural network. Predicting one or more specific cognitive states of an individual through the hidden layers of a neural network based on neural data, wherein the neural network is trained on neural training data, and the encoder-decoder learns the transition between the hidden layers of the physiological data-based neural network and the hidden layers of the neural data-based neural network during the system training phase.

[0006] On one hand, the hidden layers of a physiological data-based neural network are projected into the latent space of the encoder-decoder by one or more feedforward layers of the encoder portion of the encoder-decoder, so as to transform the hidden layers of the physiological data-based neural network into latent vectors, thereby enabling one or more processors of one or more controllers to perform the system training phase.

[0007] On the other hand, the system training phase is performed by one or more processors of one or more controllers, which reconstruct the hidden layers of a neural network based on latent vectors using one or more feedforward layers of the encoder-decoder decoder.

[0008] On the other hand, the hidden layers of neural networks based on physiological data are matched with the hidden layers of neural networks based on neural data.

[0009] On one hand, neural training data is received from one or more electroencephalogram (EEG) sensors worn by the individual during the systematic training phase.

[0010] In another respect, neural networks based on physiological data and neural networks based on neural data are feedforward neural networks.

[0011] On the other hand, neural networks based on physiological data and neural networks based on neural data are called convolutional neural networks (CNNs).

[0012] On one hand, the encoder-decoder is a variational autoencoder (VAE).

[0013] On the other hand, one or more processors of one or more controllers perform separate training phases for each unrelated cognitive state of an individual.

[0014] On the other hand, one or more controllers of the system communicate electronically with one or more controllers that are part of one or more vehicle systems.

[0015] On one hand, one or more processors of one or more controllers execute instructions to transmit one or more specific cognitive states of an individual to one or more controllers that are respective parts of one or more vehicle systems, wherein the behavior of one or more vehicle systems is modified based on one or more specific cognitive states of the individual.

[0016] On the other hand, one or more vehicle systems include one or more of the following: audio system, infotainment system, automated driving system (ADS), advanced driver assistance system (ADAS), and navigation system.

[0017] In one aspect, a method is disclosed for predicting one or more specific cognitive states of an individual based on non-neurophysiological data collected by one or more non-neurophysiological sensors. The method includes receiving non-neurophysiological data by a physiological data-based neural network of one or more controllers, wherein the physiological data-based neural network is trained based on non-neurophysiological training data collected by one or more non-neurophysiological sensors during a system training phase. In response to receiving non-neurophysiological data from one or more non-neurophysiological sensors, the method includes predicting one or more intermediate specific cognitive states of an individual based on the non-neurophysiological data by the physiological data-based neural network of one or more controllers. The method includes receiving a hidden layer of the physiological data-based neural network by an encoder-decoder of one or more controllers, the hidden layer being created when the physiological data-based neural network predicts one or more intermediate specific cognitive states of an individual. The method includes predicting a hidden layer of a neural data-based neural network by an encoder-decoder of one or more controllers based on the hidden layer of the physiological data-based neural network of one or more intermediate specific cognitive states of an individual. The method also includes transmitting the hidden layer of the neural data-based neural network to the neural data-based neural network via the encoder-decoder of one or more controllers. Finally, the method includes predicting one or more specific cognitive states of an individual through the hidden layers of a neural network based on neural data of one or more controllers, wherein the neural network based on neural data is trained based on neural training data, and wherein the encoder-decoder learns the transformation between the hidden layers of the physiological data-based neural network and the hidden layers of the neural data-based neural network during the system training phase.

[0018] In another aspect, the method includes transmitting one or more specific cognitive states of an individual to one or more controllers that are respective parts of one or more vehicle systems, wherein the behavior of one or more vehicle systems is modified based on the one or more specific cognitive states of the individual.

[0019] In another aspect, a system for predicting one or more specific cognitive states of an individual based on non-neurophysiological data is disclosed. The system includes one or more non-neurophysiological sensors for monitoring non-neurophysiological measurements of each individual, and one or more controllers in electronic communication with the one or more non-neurophysiological sensors. The one or more controllers include one or more processors executing instructions to receive non-neurophysiological data via a physiological data-based neural network, wherein the physiological data-based neural network is trained based on non-neurophysiological training data acquired by the one or more non-neurophysiological sensors during a system training phase. In response to receiving non-neurophysiological data from the one or more non-neurophysiological sensors, one or more intermediate specific cognitive states of the individual are predicted by the physiological data-based neural network based on the non-neurophysiological data. Hidden layers of the neural data-based neural network are predicted by an encoder-decoder based on the hidden layers of the physiological data-based neural network for the one or more intermediate specific cognitive states of the individual. The encoder-decoder predicts hidden layers of the neural data-based neural network based on the hidden layers of the physiological data-based neural network for the one or more intermediate specific cognitive states of the individual. The encoder-decoder transmits the hidden layers of the neural data-based neural network to the neural data-based neural network. Predicting one or more specific cognitive states of an individual through the hidden layers of a neural network based on neural data, wherein the neural network is trained on neural training data, and the encoder-decoder learns the transition between the hidden layers of the physiological data-based neural network and the hidden layers of the neural data-based neural network during the system training phase.

[0020] On the other hand, one or more processors of one or more controllers perform the system training phase by projecting the hidden layers of a physiological data-based neural network into the latent space of the encoder-decoder by one or more feedforward layers, which are part of the encoder of the encoder-decoder, to transform the hidden layers of the physiological data-based neural network into latent vectors.

[0021] In another aspect, one or more processors of one or more controllers perform the system training phase by reconstructing the hidden layers of a neural network based on latent vectors using one or more feedforward layers of the encoder-decoder's decoder.

[0022] On the one hand, the hidden layers of neural networks based on physiological data are matched with the hidden layers of neural networks based on neural data.

[0023] On the other hand, neural training data is received from one or more electroencephalogram (EEG) sensors worn by the individual during the systematic training phase.

[0024] In another aspect, one or more non-neurophysiological sensors include one or more of the following: eye-tracking sensors, thermal imagers for measuring blood flow in specific areas of an individual's body, camera systems for recording an individual's facial expressions and body postures, wearable sensors, and functional near-infrared spectroscopy (FNIRS) sensors.

[0025] Further areas of application will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0026] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0027] Figure 1 This is a schematic diagram of a vehicle including the disclosed system for predicting one or more specific cognitive states of an individual, according to an exemplary embodiment, wherein the system includes one or more controllers that are in electronic communication with one or more non-neurophysiological sensors.

[0028] Figure 2 According to exemplary embodiments, in Figure 1 A block diagram of the software architecture of one or more controllers is shown, wherein the one or more controllers receive training data from neural sensors and non-neurophysiological sensors; and

[0029] Figure 3 This is a process flowchart of a method for predicting one or more specific cognitive states of an individual based on non-neurophysiological data collected by non-neurophysiological sensors, according to an exemplary embodiment. Detailed Implementation

[0030] The following description is merely exemplary in nature and is not intended to limit this disclosure, its application, or its uses.

[0031] refer to Figure 1 The illustration depicts a vehicle 10 including an exemplary system 12 for predicting one or more specific cognitive states of an individual 14. The system 12 includes one or more controllers 20 in electronic communication with one or more non-neurophysiological sensors 22, each monitoring non-neurophysiological measurements of the individual 14. The one or more controllers 20 of the system 12 also communicate electronically with one or more controllers 30, which are parts of one or more vehicle systems, wherein the behavior of one or more vehicle systems is modified based on the predicted cognitive state of the individual 14. Some examples of vehicle systems modified based on the predicted cognitive state of the individual 14 include, but are not limited to, audio systems that play music or other types of audio files, infotainment systems, automated driving systems (ADS), advanced driver assistance systems (ADAS), or navigation systems. While Figure 1 System 12 is shown as part of a vehicle, but it should be understood that system 12 is not limited to vehicles and can be used in a variety of other applications that take into account the specific cognitive states of an individual. By way of example only, in another embodiment, system 12 is used to predict one or more specific cognitive states of an assembly line operator.

[0032] The non-neurophysiological sensor 22 includes any type of sensor for monitoring non-neurophysiological data of individual 14. In a non-limiting embodiment, the non-neurophysiological sensor 22 includes, but is not limited to, eye-tracking sensors, thermal imagers for measuring blood flow in specific areas of individual 14's body and for respiration, camera systems for recording facial expressions and body posture of individual 14, wearable sensors, and functional near-infrared spectroscopy (FNIRS) sensors for measuring non-neurophysiological data, such as respiration in the prefrontal cortex of individual 14 and the concentrations of oxyhemoglobin (O2Hb) and deoxyhemoglobin (Hhb). It should be understood that system 12 may also include other types of non-neurophysiological sensors 22. For example, radar sensors may be used to detect respiration and heart rate, red, green, and blue (RGB) cameras may be used for remote heart rate sensing, accelerometers may be used to measure heart rate and respiration, and pressure sensors may be used to measure heart rate, body posture, and respiration.

[0033] Wearable sensors are worn on individual 14's body and measure non-neurophysiological data, such as pulse, heart rate variability, and skin conductance. Eye-tracking sensors determine eye movement tracking, such as individual 14's gaze position, pupil dilation, eye distance, gaze velocity, and gaze acceleration in the x and y directions. Thermal imaging determines blood flow in individual 14's facial regions, such as the nose, tip of the nose, forehead, and left and right cheeks. A camera system records facial expressions indicating individual 14's engagement, valence, and attention, and records facial expressions such as frowning, raising eyebrows, raising inner eyebrows, closing eyes, widening eyes, cheek lifting, and pursing of the lips.

[0034] Figure 2 A block diagram of the software architecture of one or more controllers 20 of system 12 is shown. The one or more controllers 20 include a physiological data-based neural network 40, a neural data-based neural network 42, and an encoder-decoder 44. In a non-limiting embodiment, both the physiological data-based neural network 40 and the neural data-based neural network 42 are convolutional neural networks (CNNs). However, it should be understood that any type of feedforward neural network can be used instead, such as a multilayer perceptron (MLP) neural network. In one embodiment, the encoder-decoder 44 is a variational autoencoder (VAE); however, other types of encoder-decoders with generative capabilities, such as generative adversarial networks (GANs) or another type of VAE, can also be used.

[0035] Figure 2 The diagram illustrates a physiological data-based neural network 40 and a neural data-based neural network 42 receiving training data 50 during the training phase of system 12, with the data flow of the training data shown as dashed lines. Specifically, the physiological data-based neural network 40 receives non-neurophysiological training data from one or more non-neurophysiological sensors 22, and the neural data-based neural network 42 receives neural training data from one or more electroencephalogram (EEG) sensors 52 worn by the individual 14. Figure 2 In the embodiment shown, the EEG sensor 52 is an EEG cap 54 worn by the individual 14. The EEG cap 54 includes a plurality of electrodes 56 attached to the scalp 58 of the individual 14.

[0036] It should be understood that the physiological data-based neural network 40 is trained on non-neurophysiological training data collected by one or more non-neurophysiological sensors 22, and the neural data-based neural network 42 is trained on neural training data from one or more EEG sensors 52. As explained below, during the training phase of system 12, both the physiological data-based neural network 40 and the neural data-based neural network 42 independently predict one or more specific cognitive states of individual 14, while the encoder-decoder 44 learns the transformations between the hidden layers of the physiological data-based neural network 40 and the hidden layers of the neural data-based neural network 42. During deployment, system 12 only receives non-neurophysiological training collected by one or more non-neurophysiological sensors 22 to predict one or more specific cognitive states of individual 14.

[0037] The transformation between the hidden layers of the physiological data-based neural network 40 and the neural data-based neural network 42, determined during the training phase of System 12, will now be described. It should be understood that the encoder-decoder 44 can learn the transformation between any hidden layer of the physiological data-based neural network 40 and any hidden layer of the neural data-based neural network 42 during the training phase of System 12. For example, the encoder-decoder 44 can learn the transformation between the final hidden layer of the physiological data-based neural network 40 and the corresponding final hidden layer of the neural data-based neural network 42 during the training phase of System 12.

[0038] The encoder-decoder 44 includes an encoder 46, a decoder 48, and a latent space 60, wherein the latent space 60 is located between the encoder 46 and the decoder 48. The encoder 46 of the encoder-decoder 44 includes one or more feedforward layers that project the hidden layers of the physiological data-based neural network 40 into the latent space 60 to transform the hidden layers of the physiological data-based neural network 40 into latent vectors. It should be understood that the latent vectors have a smaller matrix dimension compared to the matrix dimension of the hidden layers of the physiological data-based neural network 40. By way of example only, in one embodiment, the hidden layers of the physiological data-based neural network 40 have a matrix dimension of [1×2320], and the latent vectors have a matrix dimension of [1×100].

[0039] The latent vector comprises multiple latent variables, where the matrix dimension of the latent vector indicates the total number of latent variables. In the current example, the matrix dimension of the latent vector is [1×100], which results in one hundred latent variables. Each latent variable in the latent vector is sampled from a probability distribution of possible values. In a non-limiting embodiment, the probability distribution is a normal distribution; however, it should be understood that other types of probability distributions may also be used. The decoder 48 of the encoder-decoder 44 receives the latent vector. The decoder 48 of the encoder-decoder 44 includes one or more feedforward layers that reconstruct the hidden layers of the neural network 42 based on the latent vector. It should be understood that the hidden layers of the neural network 40 based on physiological data are matched with the hidden layers of the neural network 42 based on neural data.

[0040] System 12 is trained to predict one or more specific cognitive states of individual 14, rather than all cognitive states that individual 14 may experience. In one embodiment, one or more specific cognitive states of individual 14 include a subset of relevant cognitive states of individual 14. The subset of relevant cognitive states represents cognitive states that lead to similar physiological effects exhibited by the individual. In the described example, the subset of relevant cognitive states includes stress and cognitive load for individual 14. As another example, the subset of relevant cognitive states may include excitement and fear, both of which lead to similar physiological effects, such as a high heart rate and a high level of alertness in the individual. In yet another example, the subset of relevant cognitive states may include fatigue and sadness. It should be understood that one or more specific cognitive states may include any other cognitive states exhibited by the human body, such as fear, joy, depression, and fatigue. Furthermore, one or more controllers 20 perform separate training phases for unrelated cognitive states of individual 14. For example, in the described embodiment, once stress and cognitive load are predicted, another training phase is required to determine fear in individual 14. It should be understood that System 12 focuses on predicting one or more specific cognitive states that are related to each other, rather than trying to predict a variety of different, unrelated cognitive states. This simplifies and narrows the range of transformations learned by encoder-decoder 44.

[0041] In one embodiment, one or more specific cognitive states indicate high or low stress and three distinct levels of cognitive load (low, medium, and high). Non-neurophysiological and neurophysiological training data were obtained by having individual 14 complete three sets of different mathematical problems 62, with progressively increasing difficulty to simulate three different levels of cognitive load. The mathematical problems were solved in two different scenarios: a first, untimed scenario with no external feedback, representing a low-stress condition; and a second, timed scenario where individual 14 received less time than required for all three mathematical problems and received continuous feedback comparing their performance to that of their peers, representing a high-stress condition.

[0042] Once the training phase is complete, system 12 can predict one or more specific cognitive states of individual 14 during real-time operation based solely on non-neurophysiological training collected by one or more non-neurophysiological sensors 22. Figure 3 A flowchart illustrating a method 300 for predicting one or more specific cognitive states in individual 14 is provided. (Reference) Figure 2 and Figure 3Method 300 may begin at decision box 302. In decision box 302, a physiological data-based neural network 40 of one or more controllers 20 continues to monitor one or more non-neurophysiological sensors 22 until non-neurophysiological data is received. As described above, the physiological data-based neural network is trained based on non-neurophysiological training data collected by one or more non-neurophysiological sensors 22 during the training phase of system 12. In response to receiving non-neurophysiological data, method 300 may proceed to box 304.

[0043] In block 304, in response to receiving non-neurophysiological data from one or more non-neurophysiological sensors 22, a neural network 40 of one or more controllers 20, based on physiological data, predicts one or more intermediate specific cognitive states of individual 14 based on the non-neurophysiological data. Method 300 can then proceed to block 306.

[0044] In box 306, encoder-decoder 44 receives a hidden layer of physiological data-based neural network 40, wherein the hidden layer of physiological data-based neural network 40 is created when the physiological data-based neural network 40 predicts one or more specific cognitive states of individual 14 based on non-neurophysiological data as described in box 304. Method 300 can then proceed to box 306.

[0045] In block 306, in response to receiving the hidden layer of the physiologically based neural network 40, the encoder-decoder 44 predicts the hidden layer of the neural network 42 based on the hidden layer of the physiologically based neural network 40 for one or more intermediate specific cognitive states of individual 14. Method 300 can then proceed to block 308.

[0046] In box 308, encoder-decoder 44 transmits the hidden layer of neural data-based neural network 42 to neural data-based neural network 42. Method 300 can then proceed to box 310.

[0047] In box 310, the neural network 42 based on neural data predicts one or more specific cognitive states of individual 14 based on the hidden layers of the neural network 42 received from encoder-decoder 44. Method 300 can then proceed to box 312.

[0048] In box 312, a neural network 42 based on neural data from one or more controllers 20 transmits one or more specific cognitive states of individual 14 to one or more controllers 30, which are respective parts of one or more vehicle systems. The behavior of one or more vehicle systems is modified based on one or more predicted cognitive states of individual 14. Method 300 can then terminate.

[0049] refer to Figure 2 The modification of the behavior of one or more vehicle systems based on one or more specific cognitive states of individual 14 will now be described. In one example, when one or more specific cognitive states of individual 14 indicate a high level of cognitive load and high stress, this indicates that individual 14 may be driving vehicle 10 under challenging conditions. Therefore, the behavior of one or more vehicle systems is modified to reduce driver distraction and notify individual 14's various contacts. An example of challenging conditions is maneuvering vehicle 10 along a steep mountain road under conditions of limited visibility, such as during rain or fog. In this example, one or more controllers 30 corresponding to the audio system can turn off the music so that individual 14 can pay more attention to the road, controllers 30 corresponding to the infotainment system can notify the individual's emergency contacts of the vehicle's location, and one or more controllers 30 corresponding to ADS or ADAS can prompt the driver to take over control of vehicle 10.

[0050] In another example, when one or more specific cognitive states of individual 14 indicate moderate cognitive load and high stress, this indicates that individual 14 is paying sufficient attention to the driving situation. Therefore, the behavior of one or more vehicle systems is modified based on individual 14's preferences. For example, if traffic congestion delays the estimated arrival time, one or more controllers 30 corresponding to the navigation system may ask individual 14 if he or she is interested in taking an alternative route; one or more controllers 30 corresponding to the infotainment system may send a text message to one of the individual's contacts to let them know that individual 14 may be late due to the delayed estimated arrival time; and one or more controllers 30 corresponding to the audio system may lower the volume of the audio being played.

[0051] In another example, when one or more specific cognitive states of an individual indicate a low level of cognitive load and low stress, this indicates that individual 14 may be falling asleep. Therefore, one or more controllers 30 corresponding to the audio system can create a notification instructing individual 14 to pay attention to the road. Alternatively, when one or more specific cognitive states of an individual indicate a low level of cognitive load and high stress, this indicates that individual 14 may be distracted and thinking about other issues (e.g., a sick family member in the hospital). Therefore, one or more controllers 30 corresponding to the audio system can play a favorite music playlist to soothe individual 14.

[0052] Referring generally to the accompanying drawings, the disclosed system for predicting one or more specific cognitive states of an individual offers various technical effects and benefits. Specifically, the disclosed system provides a method for learning transformations between hidden layers of a neural network based on physiological data and hidden layers of a neural network based on neural data. Therefore, the system can predict one or more specific cognitive states of an individual during real-time operation based solely on non-neurophysiological data, without utilizing invasive EEG sensors. It should be understood that the disclosed system improves accuracy compared to systems trained only on non-neurophysiological data. For example, in one embodiment, the disclosed system achieves approximately 87% accuracy. In contrast, a system trained only using non-neurophysiological data achieves 76% accuracy. It should also be understood that the physiological data-based neural network preprocesses the non-neurophysiological data before sending it to the encoder-decoder, which in turn denoises the non-neurophysiological data and narrows the encoder-decoder's focus to only task-relevant signals that are crucial for predicting one or more specific cognitive states of an individual.

[0053] A controller can refer to electronic circuitry, combinational logic circuitry, a field-programmable gate array (FPGA), a processor (shared, dedicated, or grouped) that executes code, or a combination of some or all of the above, such as in a system-on-a-chip. Alternatively, the controller can be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor can operate under the control of an operating system residing in memory. The operating system can manage computer resources so that computer program code embodied as one or more computer software applications (such as applications residing in memory) can have instructions that the processor can execute. In alternative embodiments, the processor can directly execute the application, in which case the operating system can be omitted.

[0054] The descriptions in this disclosure are merely exemplary in nature, and variations thereof that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A system for predicting one or more specific cognitive states of an individual, the prediction being based on non-neurophysiological data collected by one or more non-neurophysiological sensors, the system comprising: One or more controllers, which communicate electronically with the one or more non-neurophysiological sensors, the one or more controllers including one or more processors, the processors executing instructions to: The non-neurophysiological data is received by a neural network based on physiological data, wherein the neural network based on physiological data is trained based on non-neurophysiological training data collected by the one or more non-neurophysiological sensors during the system training phase; In response to receiving the non-neurophysiological data from the one or more non-neurophysiological sensors, the individual's one or more intermediate specific cognitive states are predicted based on the non-neurophysiological data using the physiological data-based neural network. The hidden layers of the physiological data-based neural network are received by an encoder-decoder, the hidden layers of the physiological data-based neural network being created by the physiological data-based neural network when predicting one or more intermediate specific cognitive states of the individual; The encoder-decoder predicts the hidden layers of the neural network based on the physiological data based layers of the individual's one or more intermediate specific cognitive states. The encoder-decoder sends the hidden layers of the neural data-based neural network to the neural data-based neural network; and The individual's one or more specific cognitive states are predicted through the hidden layers of the neural data-based neural network, wherein the neural data-based neural network is trained based on neural training data, and wherein the encoder-decoder learns the transition between the hidden layers of the physiological data-based neural network and the hidden layers of the neural data-based neural network during the system training phase.

2. The system according to claim 1, wherein, The one or more processors of the one or more controllers perform the system training phase in the following manner: The hidden layers of the physiological data-based neural network are projected into the latent space of the encoder-decoder by one or more feedforward layers that are part of the encoder of the encoder-decoder, so as to transform the hidden layers of the physiological data-based neural network into latent vectors.

3. The system according to claim 2, wherein, The one or more processors of the one or more controllers perform the system training phase in the following manner: The hidden layers of the neural data-based neural network are reconstructed based on the latent vectors through one or more feedforward layers of the encoder-decoder decoder.

4. The system according to claim 1, wherein, The hidden layer of the neural network based on physiological data is matched with the hidden layer of the neural network based on neural data.

5. The system according to claim 1, wherein, The neural training data is received from one or more electroencephalogram (EEG) sensors worn by the individual during the system training phase.

6. The system according to claim 1, wherein, The neural network based on physiological data and the neural network based on neural data are both feedforward neural networks.

7. The system according to claim 1, wherein, The neural network based on physiological data and the neural network based on neural data are both convolutional neural networks (CNNs).

8. The system according to claim 1, wherein, The encoder-decoder is a variational autoencoder (VAE).

9. The system according to claim 1, wherein, The one or more processors of the one or more controllers perform separate training phases for each unrelated cognitive state of the individual.

10. The system according to claim 1, wherein, The system’s one or more controllers communicate electronically with one or more controllers that are each part of one or more vehicle systems.

Images