Multi-modal driver state analysis method and system based on whale optimization algorithm

Abstract

The invention discloses a multi-modal driver state analysis method and system based on a whale optimization algorithm, belongs to the technical field of industrial intelligence, and aims to solve the technical problem of how to improve the accuracy and reliability of driver state detection. Comprising the following steps: collecting psychological state data of a plurality of drivers in different driving states to obtain a data set; extracting time-domain feature signals of the psychological state data of each driver in the data set, and combining the time-domain feature signals which pass verification and screening into a feature vector; a state analysis model is constructed based on SVM, and the state analysis model is used for outputting a driving state label of the driver by taking the feature vector as input; constructing a training set based on the feature vectors and the driving state labels corresponding to the feature vectors, and performing parameter optimization on the state analysis model through a whale optimization algorithm based on the training set; and predicting and outputting a driving state label of the driver through the optimized state analysis model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multimodal driver state analysis method and system based on a Gray Whale optimization algorithm. Background Art

[0002] When driving on a highway, if the driver is in an abnormal mental state such as tension, fear, excitement, fatigue, etc., the driver's reaction speed, judgment and concentration will often be reduced, which will often increase the risk of accidents and have a greater impact on normal traffic flow. Therefore, in order to ensure the safety of the driver and avoid potential dangers, a driver's physiological and mental state detection system can be equipped to detect the driver's physiological and mental state and issue a warning when the driving state is not good. In general, the driver's driving condition is usually analyzed by wearing a heart rate detection device or a head-mounted brain wave detection device. It detects the driver's physiological and mental changes and fatigue state by detecting the driver's heart rate, heart rate variability, or analyzing the specific frequency band of brain waves through a computational model. Although the traditional driver's physiological and mental state detection method can distinguish the driver's physiological and mental state from the fatigue state and play an important role in avoiding traffic accidents, it still has the following shortcomings:

[0003] (1) Traditional driver's physiological and psychological state detection models mostly use traditional classification methods, which have limited ability to distinguish driver's state detection and their detection accuracy is low;

[0004] (2) The detection signals used in traditional driver's bio-psychological state detection models are relatively single, usually one of the following: heart rate signal, EEG signal, skin conduction signal, eye movement signal or facial signal. The reliability of the driver's bio-psychological state detection system with only a single detection signal is low, and the detection results are difficult to be convincing.

[0005] How to improve the accuracy and reliability of driver status detection is a technical problem that needs to be solved. Summary of the invention

[0006] The technical task of the present invention is to address the above shortcomings and provide a multimodal driver state analysis method and system based on the Gray Whale optimization algorithm to solve the technical problem of how to improve the accuracy and reliability of driver state detection.

[0007] In a first aspect, the present invention provides a multimodal driver state analysis method based on the Gray Whale optimization algorithm, comprising the following steps:

[0008] Data collection: Collect the psychological state data of multiple drivers under different driving conditions to obtain a data set;

[0009] Feature extraction: extract the time domain feature signals of each driver's psychological state data in the data set, verify and screen the extracted time domain feature signals, and combine the time domain feature signals that pass the verification and screening into a feature vector;

[0010] State analysis model construction: A state analysis model is constructed based on SVM, wherein the state analysis model is used to take a feature vector as input, analyze the driver's psychological state, and output a driving state label of the driver;

[0011] State analysis model optimization: A training set is constructed based on the feature vector and the driving state label corresponding to the feature vector. Based on the training set, the parameters of the state analysis model are optimized using the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model to obtain the optimized state analysis model.

[0012] State analysis: Collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and use the optimized state analysis model to predict and output the driver's driving state label.

[0013] Preferably, the psychological state data includes the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening, and brain waves, wherein the brain waves include alpha waves, beta waves, and delta waves;

[0014] The driver's heart rate is collected through a photoplethysmography heart rate meter, the driver's blood pressure is collected through an electronic sphygmomanometer, the driver's skin conductivity is collected through a skin conductance meter, the driver's blinking frequency and eye opening are collected through an eye tracker, and the driver's brain waves are collected through an EEG device.

[0015] Preferably, the heart rate time domain characteristic signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows:

[0016]

[0017] Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time;

[0018] The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows:

[0019] SBP=y p =max|y(n)|,

[0020] SBP=y p =max|y(n)|,

[0021]

[0022] Among them, y(n) represents the nth blood pressure value;

[0023] The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows:

[0024]

[0025] SCM=k p =max|k(n)|,

[0026]

[0027] Among them, k(n) represents the value of skin conductivity at the nth time;

[0028] The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows:

[0029]

[0030] EBR=l p =max|l(n)|,

[0031]

[0032] Where l(n) represents the nth blink rate;

[0033] The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows:

[0034]

[0035] MAXEO=m p =max|m(n)|,

[0036]

[0037] REO=V pp =max|m(n)|-min|m(n)|,

[0038] m(n) represents the number of blinks in the nth unit time;

[0039] The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows:

[0040] P=z p =max|z(n)|,

[0041] A=Vpp =max|z(n)|-min|z(n)|,

[0042] Among them, z(n) represents the current wave value;

[0043] The feature vector is represented as: ,,,,,,,,

[0044] Preferably, for the extracted time domain feature signals, the validity of the time domain feature signals is verified by calculating the correlation between the time domain feature signals, and the correlation calculation formula is as follows:

[0045]

[0046] Wherein, X and Y represent two time domain characteristic signals for comparison.

[0047] Preferably, the parameters of the state analysis model are optimized based on the training set by using the Gray Whale optimization algorithm, including the following steps:

[0048] L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as:

[0049] K(x i ,x)=exp{-α‖xx i ‖ 2 / σ 2},

[0050] Among them, x i represents, x represents, α represents, σ ​​represents;

[0051] L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space;

[0052] L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows:

[0053] D=|Cx P (t)-x t |,

[0054] x t+1 =x P (t)-AD,

[0055] A=2ar1-a,

[0056] C=2r2,

[0057] Where D represents the distance between the optimal values ​​of individual distances, x P (t) indicates the location of the optimal value, x t Indicates the current individual location, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1];

[0058] L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses;

[0059] L500, calculate the fitness value of the updated individual, the calculation formula is as follows:

[0060] D q =|C l x j -x t |,

[0061]

[0062] Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration;

[0063] L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged.

[0064] L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

[0065] In a second aspect, the present invention provides a multimodal driver state analysis system based on a Gray Whale optimization algorithm, which is used to implement driver state analysis by using a multimodal driver state analysis method based on a Gray Whale optimization algorithm as described in any one of the first aspects, wherein the system includes a data acquisition module, a feature extraction module, a state analysis model construction module, a state analysis module optimization module, and a state analysis module;

[0066] The data collection module is used to perform the following: collect psychological state data of multiple drivers under different driving conditions to obtain a data set;

[0067] The feature extraction module is used to perform the following: extract the time domain feature signal of each driver's psychological state data in the data set, verify and filter the extracted time domain feature signal, and combine each time domain feature signal that passes the verification and filtering into a feature vector;

[0068] The state analysis model building module is used to perform the following: building a state analysis model based on SVM, wherein the state analysis model is used to take a feature vector as input, analyze the driver's psychological state and output a driving state label of the driver;

[0069] The state analysis model optimization module is used to perform the following: construct a training set based on the feature vector and the driving state label corresponding to the feature vector, and optimize the parameters of the state analysis model based on the training set by using the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model to obtain the optimized state analysis model;

[0070] The state analysis module is used to perform the following: collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and predict and output the driver's driving state label through the optimized state analysis model.

[0071] Preferably, the psychological state data includes the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening, and brain waves, wherein the brain waves include alpha waves, beta waves, and delta waves;

[0072] The data acquisition module is used to collect the driver's heart rate through a photoplethysmography heart rate meter, collect the driver's blood pressure through an electronic sphygmomanometer, collect the driver's skin conductivity through a skin conductance meter, collect the driver's blinking frequency and eye opening through an eye tracker, and collect the driver's brain waves through an EEG device.

[0073] Preferably, the heart rate time domain characteristic signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows:

[0074]

[0075] Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time;

[0076] The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows:

[0077] SBP=y p =max|y(n)|,

[0078] SBP=y p =max|y(n)|,

[0079]

[0080] Among them, y(n) represents the nth blood pressure value;

[0081] The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows:

[0082]

[0083] SCM=k p =max|k(n)|,

[0084]

[0085] Among them, k(n) represents the value of skin conductivity at the nth time;

[0086] The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows:

[0087]

[0088] EBR=l p =max|l(n)|,

[0089]

[0090] Where l(n) represents the nth blink rate;

[0091] The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows:

[0092]

[0093] MAXEO=m p =max|m(n)|,

[0094]

[0095] REO=V pp =max|m(n)|-min|m(n)|,

[0096] m(n) represents the number of blinks in the nth unit time;

[0097] The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows:

[0098] P=zp =max|z(n)|,

[0099] A=V pp =max|z(n)|-min|z(n)|,

[0100] Among them, z(n) represents the current wave value;

[0101] The feature vector is represented as: ,,,,,,,,

[0102] Preferably, for the extracted time domain feature signals, the feature extraction module is used to verify the validity of the time domain feature signals by calculating the correlation between the time domain feature signals, and the correlation calculation formula is as follows:

[0103]

[0104] Wherein, X and Y represent two time domain characteristic signals for comparison.

[0105] Preferably, the state analysis model optimization module is used to perform parameter optimization on the state analysis model as follows:

[0106] L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as:

[0107] K(x i ,x)=exp{-α‖xx i ‖ 2 / σ 2},

[0108] Among them, x i represents, x represents, α represents, σ ​​represents;

[0109] L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space;

[0110] L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows:

[0111] D=|Cx P (t)-x t |,

[0112] x t+1 =x P (t)-AD,

[0113] A=2ar1-a,

[0114] C=2r2,

[0115] Where D represents the distance between the individual optimal values, x P (t) indicates the location of the optimal value, x t Indicates the current location of the individual, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1];

[0116] L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses;

[0117] L500, calculate the fitness value of the updated individual, the calculation formula is as follows:

[0118] D q =|C l x j -x t |,

[0119]

[0120] Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration;

[0121] L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged.

[0122] L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

[0123] The multimodal driver state analysis method and system based on the Gray Whale optimization algorithm of the present invention have the following advantages: a state analysis model is constructed based on SVM, the constructed state analysis model is optimized by the Gray Whale optimization algorithm, and the driver's driving state is predicted by the optimized state analysis model, thereby improving the reliability and accuracy of the driver's state prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0125] The present invention is further described below in conjunction with the accompanying drawings.

[0126] Figure 1 This is a flowchart of a multimodal driver state analysis method based on the Gray Whale optimization algorithm in Example 1. DETAILED DESCRIPTION

[0127] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0128] The embodiments of the present invention provide a multimodal driver state analysis method and system based on the Gray Whale optimization algorithm, which are used to solve the technical problem of how to improve the accuracy and reliability of driver state detection.

[0129] Embodiment 1:

[0130] The present invention discloses a multimodal driver state analysis method based on the Gray Whale optimization algorithm, which includes five steps: data collection, feature extraction, state analysis model construction, state analysis module optimization and state analysis.

[0131] Step S100: Data collection: Collecting psychological state data of multiple drivers under different driving conditions to obtain a data set.

[0132] During data collection in step S100 of this implementation, the psychological state data include the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening and brain waves. The brain waves include alpha waves, beta waves and delta waves. Correspondingly, the driver's heart rate is collected by a photoplethysmography heart rate meter, the driver's blood pressure is collected by an electronic sphygmomanometer, the driver's skin conductivity is collected by a skin conductance meter, the driver's blinking frequency and eye opening are collected by an eye tracker, and the driver's brain waves are collected by an EEG device.

[0133] Among them, the driver's driving status is divided into seven status labels: tired, nervous, fearful, excited, angry, sad, and normal.

[0134] Step S200: Feature extraction: extracting the time domain feature signal of each driver's psychological state data in the data set, verifying and screening the extracted time domain feature signal, and combining each time domain feature signal that passes the verification and screening into a feature vector.

[0135] In this embodiment, step S200 extracts the time domain characteristic signal of each driver's physiological and psychological state in the data set to reveal the difference in waveform changes of each state data under different physiological and psychological states of different drivers, and then extracts the driver's state information features, verifies the validity of the time domain characteristics of each driver's physiological and psychological state information, selects the effective characteristic signal that can clearly explain the category of the driver's physiological and psychological state in each information feature, and combines each effective characteristic signal into a feature vector. The feature vector is used as subsequent data to input into the SVM state analysis model for machine learning.

[0136] When extracting the time domain feature signal, the calculation formula of each psychological state data is as follows: The heart rate time domain feature signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows:

[0137]

[0138] Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time;

[0139] The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows:

[0140] SBP=y p =max|y(n)|,

[0141] SBP=y p =max|y(n)|,

[0142]

[0143] Among them, y(n) represents the nth blood pressure value;

[0144] The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows:

[0145]

[0146] SCM=k p =max|k(n)|,

[0147]

[0148] Among them, k(n) represents the value of skin conductivity at the nth time;

[0149] The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows:

[0150]

[0151] EBR=l p =max|l(n)|,

[0152]

[0153] Where l(n) represents the nth blink rate;

[0154] The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows:

[0155]

[0156] MAXEO=m p =max|m(n)|,

[0157]

[0158] REO=V pp =max|m(n)|-min|m(n)|,

[0159] m(n) represents the number of blinks in the nth unit time;

[0160] The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows:

[0161] P=z p =max|z(n)|,

[0162] A=V pp =max|z(n)|-min|z(n)|,

[0163] Among them, z(n) represents the current wave value;

[0164] The feature vector is represented as: ,,,,,,,.

[0165] For the extracted time domain feature signals, the validity of the time domain feature signals is verified by calculating the correlation between the time domain feature signals. The correlation calculation formula is as follows:

[0166]

[0167] Wherein, X and Y represent two time domain characteristic signals for comparison.

[0168] Based on the verified time domain characteristic signal, the constructed feature vector is expressed as:

[0169]

[0170] Step S300: state analysis model construction: constructing a state analysis model based on SVM, wherein the state analysis model is used to take a feature vector as input, analyze the driver's psychological state and output a driving state label of the driver.

[0171] The state analysis model constructed in step S300 of this embodiment is constructed based on the SVM algorithm. In order to improve the accuracy of the driver's physiological state analysis, an appropriate kernel function and penalty factor are selected.

[0172] Step S400: state analysis model optimization: construct a training set based on the feature vector and the driving state label corresponding to the feature vector, and optimize the parameters of the state analysis model based on the training set and by the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model to obtain the optimized state analysis model.

[0173] As a specific implementation of step S400, the parameters of the state analysis model are optimized based on the training set by using the Gray Whale optimization algorithm, including the following steps:

[0174] L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as:

[0175] K(x i ,x)=exp{-α‖xx i ‖ 2 / σ 2},

[0176] Among them, x i represents, x represents, α represents, σ ​​represents;

[0177] L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space;

[0178] L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows:

[0179] D=|Cx P (t)-x t |,

[0180] x t+1 =x P(t)-AD,

[0181] A=2ar1-a,

[0182] C=2r2,

[0183] Where D represents the distance between the individual optimal values, x P (t) indicates the location of the optimal value, x t Indicates the current location of the individual, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1];

[0184] L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses;

[0185] L500, calculate the fitness value of the updated individual, the calculation formula is as follows:

[0186] D q =|C l x j -x t |,

[0187]

[0188]

[0189] Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration;

[0190] L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged.

[0191] L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

[0192] Step S500 state analysis: collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and predict and output the driver's driving state label through the optimized state analysis model.

[0193] In this embodiment, for the driver to be detected, his psychological state data is collected, the time domain feature signals of the psychological state data are extracted, and the time domain feature signals are combined into a feature vector. The feature vector of the driver is used as input, and the feature vector is converted from nonlinear space to linear space by the kernel function in the optimized state analysis model, and the driver's physiological psychological state is classified in the linear space. After the nonlinear space output, the driver's physiological psychological state analysis can be obtained, and the driver's driving state label can be obtained.

[0194] Embodiment 2:

[0195] The present invention discloses a multi-modal driver state analysis system based on the Gray Whale optimization algorithm, comprising a data acquisition module, a feature extraction module, a state analysis model construction module, a state analysis module optimization module and a state analysis module.

[0196] The data collection module is used to perform the following: collect psychological state data of multiple drivers under different driving conditions to obtain a data set.

[0197] In this embodiment, the psychological state data include the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening and brain waves. The brain waves include α waves, β waves and δ waves. Correspondingly, the data acquisition module is used to collect the driver's heart rate through a photoplethysmography heart rate meter, collect the driver's blood pressure through an electronic sphygmomanometer, collect the driver's skin conductivity through a skin conductance meter, collect the driver's blinking frequency and eye opening through an eye tracker, and collect the driver's brain waves through an EEG device.

[0198] Among them, the driver's driving status is divided into seven status labels: tired, nervous, fearful, excited, angry, sad, and normal.

[0199] The feature extraction module is used to perform the following: extract the time domain feature signal of each driver's psychological state data in the data set, verify and filter the extracted time domain feature signal, and combine each time domain feature signal that passes the verification and filtering into a feature vector.

[0200] This embodiment extracts the time domain characteristic signals of the physiological and psychological states of each driver in the data set through the feature extraction module to reveal the differences in the waveform changes of each state data under different physiological and psychological states of different drivers, and then extracts the driver state information features, verifies the validity of the time domain characteristics of each driver's physiological and psychological state information, and selects the effective characteristic signals that can clearly explain the category of the driver's physiological and psychological state in each information feature, and combines each effective characteristic signal into a feature vector. The feature vector is used as subsequent data to input into the SVM state analysis model for machine learning.

[0201] When extracting the time domain feature signal, the calculation formula of each psychological state data is as follows: The heart rate time domain feature signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows:

[0202]

[0203] Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time;

[0204] The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows:

[0205] SBP=y p =max|y(n)|,

[0206] SBP=y p =max|y(n)|,

[0207]

[0208] Among them, y(n) represents the nth blood pressure value;

[0209] The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows:

[0210]

[0211] SCM=k p =max|k(n)|,

[0212]

[0213] Among them, k(n) represents the value of skin conductivity at the nth time;

[0214] The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows:

[0215]

[0216] EBR=l p =max|l(n)|,

[0217]

[0218] Where l(n) represents the nth blink rate;

[0219] The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows:

[0220]

[0221] MAXEO=m p =max|m(n)|,

[0222]

[0223] REO=V pp =max|m(n)|-min|m(n)|,

[0224] m(n) represents the number of blinks in the nth unit time;

[0225] The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows:

[0226] P=z p =max|z(n)|,

[0227] A=V pp =max|z(n)|-min|z(n)|,

[0228] Among them, z(n) represents the current wave value;

[0229] The feature vector is represented as: ,,,,,,,.

[0230] For the extracted time domain feature signals, the feature extraction module is used to verify the validity of the time domain feature signals by calculating the correlation between the time domain feature signals. The correlation calculation formula is as follows:

[0231]

[0232] Wherein, X and Y represent two time domain characteristic signals for comparison.

[0233] Based on the verified time domain characteristic signal, the constructed feature vector is expressed as:

[0234]

[0235] The state analysis model building module is used to perform the following: build a state analysis model based on SVM, and the state analysis model is used to take the feature vector as input, analyze the driver's psychological state and output the driver's driving state label.

[0236] The state analysis model constructed in this embodiment is constructed based on the SVM algorithm. In order to improve the accuracy of the driver's physiological state analysis, an appropriate kernel function and penalty factor are selected.

[0237] The state analysis model optimization module is used to perform the following: construct a training set based on the feature vector and the driving state label corresponding to the feature vector, optimize the parameters of the state analysis model based on the training set and through the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model, to obtain the optimized state analysis model.

[0238] As a specific implementation of the state analysis model optimization module, the parameters of the state analysis model are optimized based on the training set using the Gray Whale optimization algorithm, including the following steps:

[0239] L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as:

[0240] K(x i ,x)=exp{-α‖xx i ‖ 2 / σ 2},

[0241] Among them, x i represents, x represents, α represents, σ ​​represents;

[0242] L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space;

[0243] L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows:

[0244] D=|Cx P (t)-x t |,

[0245] x t+1 =x P (t)-AD,

[0246] A=2ar1-a,

[0247] C=2r2,

[0248] Where D represents the distance between the individual optimal values, x P (t) indicates the location of the optimal value, x t Indicates the current location of the individual, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1];

[0249] L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses;

[0250] L500, calculate the fitness value of the updated individual, the calculation formula is as follows:

[0251] D q =|C l x j -x t |,

[0252]

[0253] Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration;

[0254] L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged.

[0255] L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

[0256] The state analysis module is used to perform the following: collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and predict and output the driver's driving state label through the optimized state analysis model.

[0257] In this embodiment, for the driver to be detected, his psychological state data is collected, the time domain feature signals of the psychological state data are extracted, and the time domain feature signals are combined into a feature vector. The feature vector of the driver is used as input, and the feature vector is converted from nonlinear space to linear space by the kernel function in the optimized state analysis model, and the driver's physiological psychological state is classified in the linear space. After the nonlinear space output, the driver's physiological psychological state analysis can be obtained, and the driver's driving state label can be obtained.

[0258] The system of this embodiment can execute the method disclosed in Example 1 to realize driver status detection.

[0259] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A multimodal driver state analysis method based on Gray Whale optimization algorithm, characterized in that: The steps include: Data collection: Collect the psychological state data of multiple drivers under different driving conditions to obtain a data set; Feature extraction: extract the time domain feature signals of each driver's psychological state data in the data set, verify and screen the extracted time domain feature signals, and combine the time domain feature signals that pass the verification and screening into a feature vector; State analysis model construction: A state analysis model is constructed based on SVM, wherein the state analysis model is used to take a feature vector as input, analyze the driver's psychological state, and output a driving state label of the driver; State analysis model optimization: A training set is constructed based on the feature vector and the driving state label corresponding to the feature vector. Based on the training set, the parameters of the state analysis model are optimized using the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model to obtain the optimized state analysis model. State analysis: Collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and use the optimized state analysis model to predict and output the driver's driving state label.

2. The multimodal driver state analysis method based on the Gray Whale optimization algorithm according to claim 1 is characterized in that: Psychological state data include the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening, and brain waves, including alpha, beta, and delta waves; The driver's heart rate is collected through a photoplethysmography heart rate meter, the driver's blood pressure is collected through an electronic sphygmomanometer, the driver's skin conductivity is collected through a skin conductance meter, the driver's blinking frequency and eye opening are collected through an eye tracker, and the driver's brain waves are collected through an EEG device.

3. The multimodal driver state analysis method based on the Gray Whale optimization algorithm according to claim 1 is characterized in that: The heart rate time domain feature signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows: Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time; The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows: SBP=y p =max|y(n)|, SBP=y p =max|y(n)|, Among them, y(n) represents the nth blood pressure value; The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows: SCM=k p =max|k(n)|, Among them, k(n) represents the value of skin conductivity at the nth time; The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows: EBR=l p =max|l(n)|, Where, l(n) represents the nth blink rate; The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows: MAXEO=m p =max|m(n)|, REO=V pp =max|m(n)|-min|m(n)|, m(n) represents the number of blinks in the nth unit time; The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows: P=z p =max|z(n)|, A=V pp =max|z(n)|-min|z(n)|, Among them, z(n) represents the current wave value; The feature vector is represented as:

4. The multimodal driver state analysis method based on the Gray Whale optimization algorithm according to claim 1 is characterized in that: For the extracted time domain feature signals, the validity of the time domain feature signals is verified by calculating the correlation between the time domain feature signals. The correlation calculation formula is as follows: Wherein, X and Y represent two time domain characteristic signals for comparison.

5. The multimodal driver state analysis method based on the Gray Whale optimization algorithm according to claim 1 is characterized in that: Based on the training set, the parameters of the state analysis model are optimized using the Gray Whale optimization algorithm, including the following steps: L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as: K(x i ,x)=exp{-α‖x-x i ‖ 2 / σ 2 }, Among them, x i and x represent two sample points in the input space, α represents the hyperparameter of the kernel function parameter, which controls the width of the kernel function and determines the decay rate of similarity as the distance increases, and σ represents the scale parameter, which is used to standardize the distance to ensure that different features have the same dimension; L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space; L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows: D=|Cx P (t)-x t |, x t+1 =x P (t)-AD, A=2ar1-a, C＝2r2, Where D represents the distance between the individual optimal values, x P (t) indicates the location of the optimal value, x t Indicates the current individual location, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1]; L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses; L500, calculate the fitness value of the updated individual, the calculation formula is as follows: D q =|C l x j -x t |, Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration; L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged. L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

6. A multimodal driver state analysis system based on Gray Whale optimization algorithm, characterized in that: Used to realize driver state analysis by a multimodal driver state analysis method based on the Gray Whale optimization algorithm as described in any one of claims 1 to 5, the system comprising a data acquisition module, a feature extraction module, a state analysis model construction module, a state analysis module optimization module and a state analysis module; The data collection module is used to perform the following: collect psychological state data of multiple drivers under different driving conditions to obtain a data set; The feature extraction module is used to perform the following: extract the time domain feature signal of each driver's psychological state data in the data set, verify and filter the extracted time domain feature signal, and combine each time domain feature signal that passes the verification and filtering into a feature vector; The state analysis model building module is used to perform the following: building a state analysis model based on SVM, wherein the state analysis model is used to take a feature vector as input, analyze the driver's psychological state and output a driving state label of the driver; The state analysis model optimization module is used to perform the following: construct a training set based on the feature vector and the driving state label corresponding to the feature vector, and optimize the parameters of the state analysis model based on the training set by using the Gray Whale optimization algorithm, including optimizing the kernel function and penalty parameters in the state analysis model to obtain the optimized state analysis model; The state analysis module is used to perform the following: collect the driver's psychological state data, extract the time domain feature signals of the psychological state data, combine the time domain feature signals into a feature vector, use the feature vector as input, and predict and output the driver's driving state label through the optimized state analysis model.

7. The multimodal driver state analysis system based on the Gray Whale optimization algorithm according to claim 6 is characterized in that: Psychological state data include the driver's heart rate, blood pressure, skin conductivity, blinking frequency, eye opening, and brain waves, including alpha, beta, and delta waves; The data acquisition module is used to collect the driver's heart rate through a photoplethysmography heart rate meter, collect the driver's blood pressure through an electronic sphygmomanometer, collect the driver's skin conductivity through a skin conductance meter, collect the driver's blinking frequency and eye opening through an eye tracker, and collect the driver's brain waves through an EEG device.

8. The multimodal driver state analysis system based on the Gray Whale optimization algorithm according to claim 6 is characterized in that: The heart rate time domain feature signal includes heart rate HR and heart rate variability SDNN, and the calculation formula is as follows: Where N represents the number of samples, and x(n) represents the number of heartbeats per unit time; The blood pressure time domain characteristic signals include systolic pressure SBP, pulse pressure PP and average pulse pressure MAP, and the calculation formula is as follows: SBP=y p =max|y(n)|, SBP=y p =max|y(n)|, Among them, y(n) represents the nth blood pressure value; The time domain characteristic signals of skin conductivity include skin conductivity level SCL, conductivity baseline drift SCBD and conductivity maximum SCM. The calculation formula is as follows: SCM=k p =max|k(n)|, Among them, k(n) represents the value of skin conductivity at the nth time; The blink frequency time domain characteristic signals include blink density BD, blink rate stability BRS and blink rate extreme value EBR, and the calculation formula is as follows: EBR=l p =max|l(n)|, Where, l(n) represents the nth blink rate; The eye opening time domain characteristic signal includes the average eye opening MEO, the standard deviation of eye opening SDEO, the maximum eye opening MAXEO and the extreme eye opening REO. The calculation formula is as follows: MAXEO=m p =max|m(n)|, REO=V pp =max|m(n)|-min|m(n)|, m(n) represents the number of blinks in the nth unit time; The time domain characteristic signals of EEG include amplitude A and peak value P, and the calculation formula is as follows: P=z p =max|z(n)|, A=V pp =max|z(n)|-min|z(n)|, Among them, z(n) represents the current wave value; The feature vector is represented as:

9. The multimodal driver state analysis system based on the Gray Whale optimization algorithm according to claim 6 is characterized in that: For the extracted time domain feature signals, the feature extraction module is used to verify the validity of the time domain feature signals by calculating the correlation between the time domain feature signals. The correlation calculation formula is as follows: Wherein, X and Y represent two time domain characteristic signals for comparison.

10. The multimodal driver state analysis system based on the Gray Whale optimization algorithm according to claim 6, characterized in that: The state analysis model optimization module is used to perform the following parameter optimization on the state analysis model: L100, define the optimization problem: determine the parameters of the state analysis model as the penalty factor C and the kernel function parameter α, the kernel function is the radial basis kernel function, expressed as: K(x i ,x)=exp{-α‖x-x i ‖ 2 / σ 2 }, Among them, x i represents, x represents, α represents, σ ​​represents; L200: Initialize the GWWO algorithm: set the population size, which is the number of gray whales, initialize the positions of gray whales, each gray whale represents a parameter combination of the state analysis model, set the number of iterations and the dimension of the search space; L300, each gray whale is regarded as an individual, and the fitness value of each individual is calculated. The individual fitness value calculation formula is as follows: D=|Cx P (t)-x t |, x t+1 =x P (t)-AD, A=2ar1-a, C＝2r2, Where D represents the distance between the individual optimal values, x P (t) indicates the location of the optimal value, x t Indicates the current location of the individual, x t+1 represents the position of the individual after iteration, a represents a constant that decreases linearly in the interval [0, 2], r1 and r2 represent two different random constants in the interval [0, 1]; L400, based on the Gray Whale algorithm, compare the fitness between individuals and determine the three optimal fitnesses; L500, calculate the fitness value of the updated individual, the calculation formula is as follows: D q =|C l x j -x t |, Among them, x q represents the fitness value of the individual optimal solution, x l represents the fitness value of the individual after iteration, b is a constant, R is a constant coefficient between [-1, 1], r3 is a random number between [0, 1], q represents the optimal solution of different regions, l represents different individuals, X t+1 Represents the individual optimal solution after iteration; L600, compare the individual fitness values ​​before and after iteration. If the fitness value after iteration is lower than the fitness value before iteration, the new position is taken as the optimal fitness value. Otherwise, the fitness value before iteration is kept unchanged. L700, the number of iterations is updated plus 1, and it is determined whether the number of iterations is greater than the maximum number of iterations. If the number of iterations has not reached the maximum value, it returns to step L300. If the number of iterations has reached the maximum value, it outputs the optimal parameter combination (C, α) to obtain the optimized state analysis model.

Images