Fatigue state detection method, system and product

Abstract

The embodiment of the invention provides a fatigue state detection method, system and product. The method comprises the following steps: converting an acquired electroencephalogram signal sequence of a user into time domain frequency spectrum data; inputting the time-domain frequency spectrum data into a qualified state recognition model for state recognition to obtain a corresponding electroencephalogram observation state sequence; and based on the electroencephalogram observation state sequence, determining the fatigue state of the user through an HSMM model. The objective of the invention is to improve the practicability and accuracy of driving fatigue detection based on EEG signals.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a fatigue state detection method, system and product. Background Art

[0002] Driver fatigue driving is an important factor causing traffic accidents. Therefore, fatigue driving detection and warning are important means to reduce traffic accidents. At present, among various physiological signals, EEG signals (Electroencephalography) have a very high temporal resolution. The information transmission is the same as that of brain neural activities, which is in the millisecond level. It can most directly reflect various activities during brain cognitive tasks and is currently considered to be the most reliable and promising signal for fatigue detection. However, it is difficult to extract accurate information that can identify the driver's fatigue state from complex EEG signals for practical application. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a fatigue state detection method, system and product, aiming to improve the practicality and accuracy of driving fatigue detection based on EEG signals.

[0004] A first aspect of an embodiment of the present application provides a fatigue state detection method, the method comprising:

[0005] Convert the collected user's EEG signal sequence into time domain spectrum data;

[0006] Inputting the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence;

[0007] Based on the EEG observation state sequence, the fatigue state of the user is determined by using the HSMM model.

[0008] Optionally, determining the fatigue state of the user through a HSMM model based on the EEG observation state sequence includes:

[0009] Obtain the model parameters of the trained HSMM model;

[0010] The EEG observation state sequence and the model parameters are input into the Viterbi model for processing to obtain the fatigue state of the user.

[0011] Optionally, train a state recognition model, including:

[0012] Segment the collected EEG signal sequence to obtain EEG signal subsequences with a target number of segments;

[0013] Annotate each EEG signal subsequence with an observation state to obtain an observation state annotation sequence corresponding to the EEG signal sequence;

[0014] Dividing a preset number of observation state annotation sequences that have been annotated with observation states into a training set and a test set;

[0015] The state recognition model is trained through the training set, and the state recognition model is tested through the test set;

[0016] When the training result of the state recognition model meets the preset conditions, the state recognition model is determined to be a qualified state recognition model.

[0017] Optionally, the method further includes:

[0018] Determine the duration of the collected EEG signal sequence;

[0019] According to the duration, a target number of segments is determined.

[0020] Optionally, train a HSMM model, including:

[0021] The training sample data in the training data set is input into the HSMM model for processing to obtain the corresponding forward probability and backward probability;

[0022] Based on the forward probability and the backward probability, the model parameters of the HSMM model are updated to maximize the likelihood function of the EEG observation state sequence;

[0023] After a preset number of model trainings are performed or the likelihood function tends to be stable, a qualified HSMM model is obtained.

[0024] Optionally, before inputting the training sample data in the training data set into the HSMM model for processing to obtain the corresponding forward probability and backward probability, the following steps are included:

[0025] Collect EEG signal sequences of a large number of users in different fatigue states;

[0026] Convert each EEG signal sequence into time-domain spectrum data;

[0027] Through a qualified state recognition model, each time-domain spectrum data is respectively recognized in state, and the EEG observation state sequence corresponding to each time-domain spectrum data is obtained;

[0028] The EEG observation state sequences corresponding to the respective time-domain spectrum data are determined as training data sets.

[0029] Optionally, the training sample data in the training data set is input into the HSMM model for processing to obtain the corresponding forward probability and backward probability, including:

[0030] Input the test sample data in the test data set into the forward algorithm of the HSMM model;

[0031] Obtain the forward probability of being in each hidden state at each moment by calculating through the forward algorithm;

[0032] Input the test sample data, the model parameters of the HSMM model and the confidence level of the target hidden state at the target time into the backward algorithm of the HSMM model;

[0033] Based on the test sample data starting from the next moment after the backward algorithm is calculated to the end moment of the test sample data, the backward probability of being in each hidden state is calculated by the backward algorithm.

[0034] Optionally, the EEG observation state sequence and the model parameters are input into a Viterbi model for processing to obtain the fatigue state of the user, including:

[0035] Inputting the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain a hidden state sequence;

[0036] Based on the hidden state sequence, a fatigue state of the user is determined.

[0037] Optionally, determining the fatigue state of the user based on the hidden state sequence includes:

[0038] Obtaining the level and duration of each fatigue state recorded in the hidden state sequence;

[0039] Based on the level and duration of each fatigue state, the fatigue state of the user is determined.

[0040] A second aspect of an embodiment of the present application provides a fatigue state detection system, the system comprising:

[0041] A data conversion module, used to convert the collected user's EEG signal sequence into time domain spectrum data;

[0042] A state recognition module, used for inputting the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence;

[0043] A fatigue state determination module is used to determine the fatigue state of the user through the HSMM model based on the EEG observation state sequence.

[0044] A third aspect of an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, the steps in a fatigue state detection method as described in the first aspect of the present application are implemented.

[0045] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in a fatigue state detection method as described in the first aspect of the present application are implemented.

[0046] The fatigue state detection method provided by this application has the following advantages:

[0047] A fatigue state detection method provided in an embodiment of the present application first converts the collected EEG signal sequence of the user into time domain spectrum data; inputs the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence; and determines the user's fatigue state through the HSMM model based on the EEG observation state sequence. Therefore, the present application introduces two models to determine the user's fatigue state in order to improve the practicality and accuracy of driving fatigue detection based on EEG signals. First, the state recognition model performs state recognition on the EEG signal sequence to obtain an observable EEG observation state sequence, and then the HSMM model (Hidden Semi-Markov Model) analyzes and processes the EEG observation state sequence obtained based on the state recognition model to obtain a hidden fatigue state sequence, and finally determines the user's fatigue state based on the hidden fatigue state sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0049] Figure 1 A flowchart of a fatigue state detection method shown in one embodiment of the present application;

[0050] Figure 2 A schematic diagram of time domain data conversion in a fatigue state detection method shown in one embodiment of the present application;

[0051] Figure 3 A structural diagram of a state recognition model in a fatigue state detection method shown in one embodiment of the present application;

[0052] Figure 4 A schematic diagram of fatigue state classification in a fatigue state detection method according to an embodiment of the present application;

[0053] Figure 5Another flow chart of a fatigue state detection method shown in one embodiment of the present application;

[0054] Figure 6 A flowchart of converting an EEG observed state sequence into a hidden state sequence in a fatigue state detection method according to an embodiment of the present application;

[0055] Figure 7 This is a schematic diagram of a fatigue status detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] refer to Figure 1 , Figure 1 This is a flow chart of a fatigue state detection method shown in one embodiment of the present application. Figure 1 As shown, the method includes:

[0058] Step S1: Convert the collected user's EEG signal sequence into time domain spectrum data.

[0059] In this embodiment, since the EEG signal (Electroencephalography) of the electroencephalogram signal has a very high time resolution and transmits information in the same millisecond level as the brain's neural activity, it can most directly reflect various types of brain activities during cognitive tasks and is considered to be the most reliable and promising signal for fatigue detection. Therefore, the present application uses EEG electroencephalogram signals to detect the user's fatigue state. First, a sequence of electroencephalogram signals of the user for a period of time is collected. The electroencephalogram signal is preferably an EEG electroencephalogram signal. Then, the electroencephalogram signal sequence is processed by discrete wavelet transform, and the one-dimensional electroencephalogram signal sequence is converted into time domain spectrum data, such as Figure 2 As shown, Figure 2 The EEG signal sequence and the time domain spectrum data after discrete wavelet transform are shown. The conversion formula for converting the one-dimensional EEG signal sequence into the time domain spectrum data is:

[0060]

[0061] Where α is the scale parameter, which defines the length change of the wavelet function; β is the translation, which defines the translation magnitude of the wavelet function; ψ(t) is the wavelet mother function; m is the frequency range index; n is the time step change index; x(t) is the original signal; W(α, β) is the wavelet transform coefficient, which represents the wavelet transform result of the original signal x(t) at scale α and position β; α0 is the initial scale parameter; β0 is the initial translation.

[0062] Step S2: Input the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence.

[0063] In this embodiment, after converting a period of EEG signal sequence into time domain spectrum data through step S1, the time domain spectrum data is input into a trained state recognition model for EEG state recognition to obtain a corresponding observable EEG observation state sequence. Among them, the state recognition model is preferably a ResNet network. The ResNet network adds a direct connection channel based on the traditional convolutional neural network. While increasing the network depth, it can effectively retain the effective features of the input information, improve the speed of the training model, and improve the accuracy. Figure 3 As shown, the model depth of the ResNet network used as the state recognition model in this application is 50 layers.

[0064] Step S3: Based on the EEG observation state sequence, determine the user's fatigue state through the HSMM model.

[0065] In this embodiment, after obtaining the observable EEG observation state sequence through step S2, the EEG observation state sequence is analyzed and processed through the HSMM model to obtain the corresponding hidden state sequence, which will record the fatigue state level of different time periods and the specific duration of the different time periods. By analyzing the hidden state sequence, the specific fatigue state of the user within the time range of the collected EEG signal sequence can be determined, thereby obtaining the user's current specific fatigue state.

[0066] A fatigue state detection method provided in an embodiment of the present application first converts the collected EEG signal sequence of the user into time domain spectrum data; inputs the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence; and determines the user's fatigue state through the HSMM model based on the EEG observation state sequence. Therefore, the present application introduces two models to determine the user's fatigue state in order to improve the practicality and accuracy of driving fatigue detection based on EEG signals. First, the state recognition model performs state recognition on the EEG signal sequence to obtain an observable EEG observation state sequence, and then the HSMM model (Hidden Semi-Markov Model) analyzes and processes the EEG observation state sequence obtained based on the state recognition model to obtain a hidden fatigue state sequence, and finally determines the user's fatigue state based on the hidden fatigue state sequence.

[0067] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, step S3 may include steps S31 to S32:

[0068] Step S31: Obtain model parameters of a qualified HSMM model.

[0069] In this embodiment, for a qualified HSMM model, the model parameters λ=(π, A, B, D) of the HSMM model are obtained, where π represents the initial state probability distribution, π={π i}, π i represents the initial probability of being in hidden state i at the initial moment; A represents the state transition probability matrix, A={a ij}, a ij represents the probability of transitioning from state i to state j; B represents the observation probability distribution, B = {b j (O t )},b j (O t ) means that state O is observed in state j t The probability of state residence time distribution; D represents the state residence time distribution, D = {d i}, d i In the i-th hidden state, the length of time that the hidden state lasts.

[0070] Step S32: inputting the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain the fatigue state of the user.

[0071] In this embodiment, the EEG observation state sequence obtained in step S2 and the model parameters obtained in step S31 are input into the Viterbi model for processing to obtain the user's fatigue state. The processing process includes an initialization process, a recursive calculation process, a termination process, and a backtracking process.

[0072] In this embodiment, the initialization process is to calculate the corresponding initial path probability δ1(i) for each possible initial hidden state. The initial path probability calculation expression is: δ1(i) = π i b i (o1). And initialize the path record to 0, the expression is: ψ1(i)=0.

[0073] In this embodiment, the recursive process is as follows: for each time step t=2, 3, ..., T and each possible hidden state s t =i, calculate from all possible previous states s t-1 = i transfers to the current state s t = j and produces the observation state O t The probability of finding the previous state i that maximizes the probability and updating δ t (j) is the maximum probability, and records the optimal previous state i to ψ t (j). The specific calculation expression is:

[0074] In this embodiment, the termination process is to find the solution that makes δ t (j) the maximum state, which makes δ t (j) The largest state is the most likely hidden state at time T. The specific calculation formula is: P * Represents the probability of the most likely hidden state.

[0075] In this embodiment, the backtracking process starts from the most likely hidden state at the final time step T, uses the ψ array to backtrack to the initial hidden state, and constructs the most likely hidden state sequence, which is the hidden state sequence corresponding to the EEG observation state sequence in the input Viterbi algorithm, and the hidden state sequence records a series of different fatigue state levels and the duration of each fatigue state level. For example, when the fatigue state levels include level 1, level 2, level 3, and level 4, a hidden state sequence can be a series of hidden state sequences recorded first, then level 1, then level 2, then level 3, then level 3, then level 4, then level 4, and then level 4. For each fatigue state level in the hidden state sequence, there is a corresponding duration. For example, a series of 7 fatigue state levels are recorded in the hidden state sequence, and the durations of the fatigue state levels from the first fatigue level to the seventh fatigue level are 30ms, 20ms, 40ms, 60ms, 20ms, 40ms, and 30ms, respectively. This is a complete hidden state sequence. Based on the obtained hidden state sequence, the user's specific fatigue state can be determined through analysis.

[0076] In this embodiment, if Figure 4 As shown, the fatigue status levels include normal state, mild fatigue, moderate fatigue and severe fatigue.

[0077] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, the method further includes a training state recognition model; the training state recognition model includes steps S01 to S05:

[0078] Step S01: Segment the collected EEG signal sequence to obtain EEG signal subsequences of a target number of segments.

[0079] In this embodiment, the collected EEG signal sequence of a user for a period of time is equally divided into EEG signal subsequences of the target number of segments. The target number of segments can be set to a fixed value according to the actual application scenario, or can be determined according to the corresponding division rules, which is not specifically limited here.

[0080] Step S02: Annotate each EEG signal subsequence with its observation state to obtain an observation state annotation sequence corresponding to the EEG signal sequence.

[0081] In this embodiment, each EEG signal subsequence obtained by division is labeled with an observation state, and then the EEG signal subsequences labeled with the observation state label belonging to the same EEG signal sequence are spliced ​​to obtain the observation state label sequence corresponding to the EEG signal sequence. Thus, a corresponding observation state label sequence will be obtained for each EEG signal sequence. Since the test set and training set of the training model are being constructed at this time, a large number of EEG signal sequences will be collected, including many EEG signal sequences in different fatigue states.

[0082] Step S03: Divide a preset number of observation state annotation sequences that have been annotated with the observation states into a training set and a test set.

[0083] In this embodiment, the ratio between the training set and the test set can be set according to the actual training scenario, and the preset number can also be set according to the actual training scenario, and the preset number can also be set according to the actual training scenario, and the preset number can also be set according to the actual training scenario. The observation state annotation sequences in the training set and the test set will also be converted into time domain spectrum data before being used as sample data for the training and test models.

[0084] Step S04: training the state recognition model using the training set, and testing the state recognition model using the test set.

[0085] In this embodiment, the constructed training set is used to train the constructed state recognition model, and the trained state recognition model is tested by the test set to evaluate the state recognition effect of the state recognition model on the EEG signal sequence.

[0086] Step S05: when the training result of the state recognition model meets the preset conditions, the state recognition model is determined to be a qualified state recognition model.

[0087] In this embodiment, when the training result of the state recognition model meets the preset conditions, the state recognition model is determined as a state recognition model that has passed the training. Determining whether the preset conditions are met may be determining whether the accuracy of the model reaches or exceeds a set value, and / or determining whether the model parameters of the state recognition model converge. If the model parameters can stably converge to a certain value during the training process, it means that the model training is qualified.

[0088] In combination with the above embodiments, in one implementation, the present application also provides a fatigue state detection method. In the fatigue state detection method, the method further includes: determining the duration of the collected EEG signal sequence; and determining the target number of segments according to the duration.

[0089] In this embodiment, in order to determine the user's fatigue state more timely and accurately, the present application determines the number of divided EEG signal subsequences based on the duration of the collected single EEG signal sequence. The longer the duration, the more divisions there are, which will more finely determine the EEG observation state sequence. Specifically: First, determine the duration of the collected EEG signal sequence, and based on the duration of the EEG signal sequence, determine the target number of segments corresponding to the duration. The specific correspondence between the duration and the target number of segments can be set according to the actual application scenario, but it is necessary to meet the requirement that after the duration increases to a certain extent, the target number of segments also needs to increase accordingly. For example, the duration of 0.5 to 1s corresponds to 10 segments for the target number of segments, 1 second to 2 seconds corresponds to 20 segments for the target number of segments, and so on.

[0090] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, the method further includes training a HSMM model; the training of the HSMM model includes steps S001 to S003:

[0091] Step S001: Input the training sample data in the training data set into the HSMM model for processing to obtain the corresponding forward probability and backward probability.

[0092] In this embodiment, a training data set for training the HSMM model is first constructed. The training sample data in the training data set is an EEG observation state sequence obtained by state recognition by the state recognition model. The training data set involves many EEG observation state sequences of different fatigue states. By inputting the training sample data in the test data set into the HSMM model for processing, the forward algorithm in the HSMM model will calculate the corresponding forward probability and the backward algorithm in the HSMM model will calculate the corresponding backward probability.

[0093] Step S002: Update the model parameters of the HSMM model based on the forward probability and the backward probability to maximize the likelihood function of the EEG observation state sequence.

[0094] In this embodiment, the model parameters of the HSMM model are updated based on the forward probability and the backward probability obtained in step S001 to maximize the likelihood function of the observable EEG observation state sequence.

[0095] Step S003: After a preset number of model trainings are performed or the likelihood function tends to be stable, a qualified HSMM model is obtained.

[0096] In this embodiment, a qualified HSMM model is obtained when a preset number of model trainings are performed or the likelihood function of the trained HSMM model tends to be stable.

[0097] In combination with the above embodiments, in one implementation, the embodiment of the present application also provides a fatigue state detection method. In the fatigue state detection method, before step S001, the method also includes: collecting a large number of EEG signal sequences of users in different fatigue states; converting each EEG signal sequence into time domain spectrum data; performing state recognition on each time domain spectrum data respectively through a qualified state recognition model to obtain the EEG observation state sequence corresponding to each time domain spectrum data; and determining the EEG observation state sequence corresponding to each time domain spectrum data obtained as a training data set.

[0098] In this embodiment, before training the HSMM model, a training data set required for the training model is first constructed. Specifically: First, a large number of EEG signal sequences of users in different fatigue states are collected. Then, each collected EEG signal sequence is converted into time domain spectrum data, and then each time domain spectrum data is state identified by training a qualified state recognition model to obtain the EEG observation state sequence corresponding to each time domain spectrum data. Finally, all EEG observation state sequences corresponding to all the obtained time domain spectrum data are determined as training sample data, and all training sample data constitute a training data set.

[0099] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, step S001 of the method may include steps S0011 to S0014:

[0100] Step S0011: Input the test sample data in the test data set into the forward algorithm of the HSMM model.

[0101] In this embodiment, the HSMM model includes a forward algorithm and a backward algorithm. The test sample data in the test data set is input into the forward algorithm of the HSMM model, and the model parameters of the HSMM model are input for processing. At this time, the model parameters of the HSMM model input are the model parameters after each update. For the first model training, the model parameters of the HSMM model input are the initial values ​​of various types of model parameters.

[0102] Step S0012: The forward probability of being in each hidden state at each moment is obtained by calculating through the forward algorithm.

[0103] In this embodiment, based on the test sample data in the test data set input into the forward algorithm and the model parameters of the HSMM model, the forward algorithm will obtain the forward probability of being in each hidden state at each moment through corresponding calculations.

[0104] Step S0013: Input the test sample data, the model parameters of the HSMM model and the confidence level of the target hidden state at the target time into the backward algorithm of the HSMM model.

[0105] In this embodiment, the test sample data in the test data set is input into the backward algorithm of the HSMM model, and the model parameters of the HSMM model are input for processing. At this time, the model parameters of the HSMM model input are the model parameters after each update. For the first model training, the model parameters of the HSMM model input are the initial values ​​of various types of model parameters.

[0106] Step S0014: Based on the test sample data between the next moment after the backward algorithm calculation moment and the end moment of the test sample data, the backward probability of being in each hidden state is obtained through the backward algorithm calculation.

[0107] In this embodiment, the backward algorithm corresponds to the forward algorithm when performing calculations. Each time the forward algorithm calculates the forward probability of being in each hidden state at a time, the backward algorithm will calculate the backward probability of being in each hidden state under this section of the EEG observation state sequence based on a section of the EEG observation state sequence in the test sample data from the next time of the time to the end time of the test sample data. For example, the forward algorithm calculates the forward probability of being in each hidden state at a time t, and the corresponding backward algorithm will calculate the backward probability of being in each hidden state under this section of the EEG observation state sequence based on the EEG observation state sequence in the test sample data from the time t+1 to the end time.

[0108] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, step S32 may include steps S321 to S322:

[0109] Step S321: input the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain a hidden state sequence.

[0110] In this embodiment, the EEG observation state sequence obtained by step S2 and the model parameters obtained by step S31 are input into the Viterbi model for processing to obtain a corresponding hidden state sequence, in which a series of different fatigue state levels and the duration of each fatigue state level are recorded. For example, when the fatigue state levels include level 1, level 2, level 3, and level 4, a hidden state sequence can record a series of hidden state sequences such as level 1, then level 2, then level 3, then level 3, then level 4, then level 4, and then level 4. For each fatigue state level in the hidden state sequence, there is a corresponding duration. For example, a series of 7 fatigue state levels are recorded in the hidden state sequence, and the durations of the fatigue state levels from the first fatigue level to the seventh fatigue level are 30ms, 20ms, 40ms, 60ms, 20ms, 40ms, and 30ms, respectively. This is a complete hidden state sequence.

[0111] Step S322: Determine the user's fatigue state based on the hidden state sequence.

[0112] In this embodiment, based on the hidden state sequence determined in step S321, the specific fatigue state of the user is determined through analysis.

[0113] In combination with the above embodiments, in one implementation, the present application embodiment further provides a fatigue state detection method. In the fatigue state detection method, step S322 may include steps S3221 to S3222:

[0114] Step S3221: Obtain the level and duration of each fatigue state recorded in the hidden state sequence.

[0115] In this embodiment, after the hidden state sequence is obtained, the levels and attributes of a series of fatigue states recorded in the hidden state sequence are analyzed and obtained, and the attributes include the duration of each fatigue state in the series of fatigue states.

[0116] Step S3222: Determine the user's fatigue state based on the level and duration of each fatigue state.

[0117] In this embodiment, after obtaining the levels of a series of fatigue states and the duration of each fatigue state in the hidden state sequence, the fatigue state of the user is determined by the level and duration of each fatigue state in the hidden state sequence. For example, the corresponding hidden state sequence is determined by observing the state sequence, and the fatigue state levels recorded in the sequence are 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4 in sequence. Since the hidden state sequence is obtained based on the current observed state sequence, the current hidden state is a fatigue state level 4. Since each state has a residence time distribution model, the probability of developing into the next fatigue state can be obtained based on the residence time of the hidden state in the current state, thereby obtaining the current fatigue state and predicting the fatigue state at the next moment.

[0118] In this embodiment, if Figure 5 As shown, firstly, a qualified state recognition model is trained ( Figure 5 ResNet in the example) and a qualified HSMM model, and then the state recognition model performs state recognition on the time domain spectrum data converted from the current EEG signal sequence to obtain the corresponding EEG observation state sequence. Then, based on the obtained EEG observation state sequence and the model parameters of the HSMM model, the Viterbi model is used to process the hidden state sequence, and the user's fatigue state is determined based on the obtained hidden state sequence. Figure 6 As shown, Figure 6 The process from the user's EEG observation state sequence to the final hidden state sequence is shown. The EEG observation state sequence is {o1, o2, o3, ..., o T}, the hidden state sequence is {s1, s2, …, s N}.

[0119] Based on the same inventive concept, an embodiment of the present application provides a fatigue state detection system, such as Figure 7 As shown, the system 700 includes:

[0120] The data conversion module 701 is used to convert the collected EEG signal sequence of the user into time domain spectrum data;

[0121] A state recognition module 702 is used to input the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence;

[0122] The fatigue state determination module 703 is used to determine the fatigue state of the user through the HSMM model based on the EEG observation state sequence.

[0123] Optionally, the fatigue state determination module 703 includes:

[0124] A model parameter acquisition module is used to obtain the model parameters of a qualified HSMM model;

[0125] The fatigue state determination submodule is used to input the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain the fatigue state of the user.

[0126] Optionally, the system 700 further includes a first model training module for training a state recognition model; the first model training module includes:

[0127] A segmentation module is used to segment the collected EEG signal sequence to obtain EEG signal subsequences with a target number of segments;

[0128] An observation state labeling sequence determination module is used to label each EEG signal subsequence with an observation state to obtain an observation state labeling sequence corresponding to the EEG signal sequence;

[0129] A data set determination module, which divides a preset number of observation state annotation sequences that have been annotated with observation states into a training set and a test set;

[0130] A first model training submodule is used to train the state recognition model through a training set and to test the state recognition model through a test set;

[0131] The first model determination module is used to determine the state recognition model as a qualified state recognition model when the training result of the state recognition model meets the preset conditions.

[0132] Optionally, the system 700 further includes:

[0133] A duration determination module, used to determine the duration of the collected EEG signal sequence;

[0134] The target segment number determination module is used to determine the target segment number according to the duration.

[0135] Optionally, the system 700 further includes a second model training module for training the HSMM model; the second model training module includes:

[0136] The probability determination module is used to input the training sample data in the training data set into the HSMM model for processing to obtain the corresponding forward probability and backward probability;

[0137] A parameter updating module, used for updating the model parameters of the HSMM model based on the forward probability and the backward probability to maximize the likelihood function of the EEG observation state sequence;

[0138] The second model determination module is used to obtain a qualified HSMM model after a preset number of model trainings or when the likelihood function tends to be stable.

[0139] Optionally, the system 700 further includes:

[0140] A data acquisition module is used to collect EEG signal sequences of a large number of users in different fatigue states;

[0141] A data conversion module, used to convert each EEG signal sequence into time domain spectrum data;

[0142] The EEG observation state sequence determination module is used to perform state recognition on each time-domain spectrum data through a qualified state recognition model to obtain the EEG observation state sequence corresponding to each time-domain spectrum data;

[0143] The training data set determination module is used to determine the EEG observation state sequences corresponding to each of the obtained time-domain spectrum data as the training data set.

[0144] Optional, probability determination module, including:

[0145] A first computing module, used for inputting the test sample data in the test data set into the forward algorithm of the HSMM model;

[0146] A forward probability determination module, used to calculate the forward probability of being in each hidden state at each moment through the forward algorithm;

[0147] A second calculation module is used to input the test sample data, the model parameters of the HSMM model and the confidence level of the target hidden state at the target time into the backward algorithm of the HSMM model;

[0148] The backward probability determination module is used to obtain the backward probability of each hidden state through the backward algorithm calculation based on the test sample data between the next moment after the backward algorithm calculation moment and the end moment of the test sample data.

[0149] Optionally, a fatigue state determination submodule includes:

[0150] A hidden state sequence determination module, used for inputting the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain a hidden state sequence;

[0151] The first fatigue state determination module is used to determine the fatigue state of the user based on the hidden state sequence.

[0152] Optionally, the first fatigue state determination module includes:

[0153] A first acquisition module, used to acquire the level and duration of each fatigue state recorded in the hidden state sequence;

[0154] The second fatigue state determination module is used to determine the fatigue state of the user based on the level and duration of each fatigue state.

[0155] Based on the same inventive concept, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and running on the processor. When the computer program is executed by the processor, the steps in a fatigue state detection method described in the first aspect of the present application are implemented.

[0156] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in a fatigue state detection method as described in the first aspect of the present application are implemented.

[0157] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0158] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0159] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0161] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0164] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0165] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0166] The above is a detailed introduction to a fatigue state detection method, system and product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A fatigue state detection method, characterized in that: The method comprises: Convert the collected user's EEG signal sequence into time domain spectrum data; Inputting the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence; Based on the EEG observation state sequence, the fatigue state of the user is determined by using the HSMM model.

2. A fatigue state detection method according to claim 1, characterized in that: Based on the EEG observation state sequence, determining the fatigue state of the user through the HSMM model includes: Obtain the model parameters of the trained HSMM model; The EEG observation state sequence and the model parameters are input into the Viterbi model for processing to obtain the fatigue state of the user.

3. A fatigue state detection method according to claim 2, characterized in that: Training state recognition model, including: Segment the collected EEG signal sequence to obtain EEG signal subsequences with a target number of segments; Annotate each EEG signal subsequence with an observation state to obtain an observation state annotation sequence corresponding to the EEG signal sequence; Dividing a preset number of observation state annotation sequences that have been annotated with observation states into a training set and a test set; The state recognition model is trained through the training set, and the state recognition model is tested through the test set; When the training result of the state recognition model meets the preset conditions, the state recognition model is determined to be a qualified state recognition model.

4. A fatigue state detection method according to claim 3, characterized in that: The method further comprises: Determine the duration of the collected EEG signal sequence; According to the duration, a target number of segments is determined.

5. A fatigue state detection method according to claim 2, characterized in that: Training the HSMM model includes: The training sample data in the training data set is input into the HSMM model for processing to obtain the corresponding forward probability and backward probability; Based on the forward probability and the backward probability, the model parameters of the HSMM model are updated to maximize the likelihood function of the EEG observation state sequence; After a preset number of model trainings are performed or the likelihood function tends to be stable, a qualified HSMM model is obtained.

6. A fatigue state detection method according to claim 5, characterized in that: Before the training sample data in the training data set is input into the HSMM model for processing and the corresponding forward probability and backward probability are obtained, the following steps are included: Collect EEG signal sequences of a large number of users in different fatigue states; Convert each EEG signal sequence into time-domain spectrum data; Through a qualified state recognition model, each time-domain spectrum data is respectively recognized in state, and the EEG observation state sequence corresponding to each time-domain spectrum data is obtained; The EEG observation state sequences corresponding to the respective time-domain spectrum data are determined as training data sets.

7. A fatigue state detection method according to claim 5, characterized in that: The training sample data in the training data set is input into the HSMM model for processing to obtain the corresponding forward probability and backward probability, including: Input the test sample data in the test data set into the forward algorithm of the HSMM model; Obtain the forward probability of being in each hidden state at each moment by calculating through the forward algorithm; Input the test sample data, the model parameters of the HSMM model and the confidence level of the target hidden state at the target time into the backward algorithm of the HSMM model; Based on the test sample data starting from the next moment after the backward algorithm is calculated to the end moment of the test sample data, the backward probability of being in each hidden state is calculated by the backward algorithm.

8. A fatigue state detection method according to claim 2, characterized in that: Inputting the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain the fatigue state of the user includes: Inputting the EEG observation state sequence and the model parameters into the Viterbi model for processing to obtain a hidden state sequence; Based on the hidden state sequence, a fatigue state of the user is determined.

9. A fatigue state detection method according to claim 2, characterized in that: Determining a fatigue state of a user based on the hidden state sequence includes: Obtaining the level and duration of each fatigue state recorded in the hidden state sequence; Based on the level and duration of each fatigue state, the fatigue state of the user is determined.

10. A fatigue state detection system, characterized in that: The system comprises: A data conversion module, used to convert the collected user's EEG signal sequence into time domain spectrum data; A state recognition module, used for inputting the time domain spectrum data into a qualified state recognition model for state recognition to obtain a corresponding EEG observation state sequence; A fatigue state determination module is used to determine the fatigue state of the user through the HSMM model based on the EEG observation state sequence.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the steps in a fatigue state detection method as claimed in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the fatigue state detection method according to any one of claims 1 to 9 are implemented.

Images