Intelligent driving fatigue recognition method based on cross-subject analysis and heterogeneous transfer learning, computer program product and terminal

Abstract

The invention discloses an intelligent driving fatigue recognition method based on cross-subject analysis and heterogeneous transfer learning, a computer program product and a terminal, and belongs to the technical field of electroencephalogram signal processing. Migrating a fatigue recognition model learned by a source domain testee to a target domain testee by adopting a heterogeneous migration learning technology, analyzing electroencephalogram signals of the target domain testee according to the migrated model, outputting an electroencephalogram fatigue recognition result, and fusing the electroencephalogram fatigue recognition result with heart rate and blood oxygen fatigue recognition results to obtain a final fatigue recognition result. Fatigue characteristic modes among different testees can be captured through cross-testee analysis, and the influence caused by individual differences is effectively overcome; through heterogeneous transfer learning, knowledge among different tasks can be used for reference, dependence of specific testees of the model is reduced, and adaptability and generalization ability of the model among different testees are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram signal processing, and in particular to an intelligent driving fatigue recognition method, a computer program product and a terminal for cross-subject analysis and heterogeneous transfer learning. Background Art

[0002] In the field of intelligent driving, the driver's fatigue state is one of the key factors for flight safety. In recent years, electroencephalogram (EEG), as a physiological signal that directly reflects brain activity, has shown great potential in the field of fatigue monitoring. EEG signals have a high temporal and spatial resolution and can capture subtle changes in the brain under different fatigue states. However, the recognition of fatigue state relies solely on a single EEG signal, and the recognition accuracy still needs to be further improved.

[0003] Furthermore, traditional fatigue monitoring methods still mainly rely on behavioral observations, physiological signal monitoring and other means. However, different drivers have different physiological and behavioral characteristics, and these differences may affect the accuracy of fatigue identification based on physiological signal monitoring and other means. In practical applications, fatigue identification needs to be able to adapt to the physiological characteristics of different subjects. How to achieve cross-subject analysis so that fatigue monitoring methods can overcome individual differences is a technical problem that urgently needs to be solved. In addition, fatigue characteristics may be different in different environments or conditions. How to improve the adaptability and generalization ability of monitoring methods among different subjects is also a technical problem that needs to be solved. Summary of the invention

[0004] The purpose of the present invention is to overcome the problems of the prior art and to provide an intelligent driving fatigue identification method, a computer program product and a terminal for cross-subject analysis and heterogeneous transfer learning.

[0005] The purpose of the present invention is to achieve the following technical solution: an intelligent driving fatigue recognition method based on cross-subject analysis and heterogeneous transfer learning, the method comprising the following steps:

[0006] Collect and pre-process EEG signals and physiological signals, including heart rate signals and blood oxygen signals;

[0007] Perform feature extraction processing on the EEG signal to obtain the EEG signal features;

[0008] Analyze the EEG signal characteristics of different subjects, identify the common fatigue feature patterns across subjects, and train a fatigue recognition model based on the common fatigue features. Then use heterogeneous transfer learning technology to transfer the fatigue recognition model learned from one or more source domain subjects to the target domain subjects.

[0009] Performing real-time analysis on the EEG signals of the target domain subjects according to the transferred fatigue recognition model, and outputting a first fatigue recognition result;

[0010] Outputting the second fatigue recognition result and the third fatigue recognition result of the current subject respectively according to the heart rate signal and the blood oxygen signal;

[0011] The first fatigue recognition result, the second fatigue recognition result, and the third fatigue recognition result are fused to obtain a final fatigue recognition result.

[0012] In one example, the EEG signal is preprocessed, including:

[0013] The EEG signal is processed by discrete wavelet transform to obtain wavelet coefficients at different scales;

[0014] The threshold value of the wavelet coefficients is calculated as follows:

[0015]

[0016] in, represents the threshold of the kth wavelet coefficient of the jth layer; W(j,k) represents the kth wavelet coefficient of the jth layer; f(W(j,k),T(j,k)) represents the nonlinear mapping function; T(j,k) represents the adaptive threshold;

[0017] The signal is reconstructed from the wavelet coefficients after threshold processing. The reconstruction expression is:

[0018]

[0019] in, represents the EEG signal after wavelet reconstruction; w j,k represents the weight factor; Represents the reconstructed wavelet basis function.

[0020] In one example, feature extraction processing is performed on the EEG signal, including:

[0021] The EMD method is used to decompose the EEG signal after wavelet reconstruction into multiple sub-band signals;

[0022] Estimate the probability density function of each subband signal;

[0023] Calculate the differential entropy of each subband signal;

[0024] The differential entropy of all sub-band signals is weighted to obtain the EEG signal features.

[0025] In one example, EEG signal features of different subjects are analyzed to identify common fatigue feature patterns across subjects, including:

[0026] Align the extracted EEG signal features of different subjects;

[0027] The aligned EEG signal features are standardized;

[0028] The recursive feature elimination algorithm was used to screen the most discriminative features of different subjects;

[0029] Correlation analysis was used to compare the characteristics between different subjects and identify common fatigue characteristic patterns across subjects.

[0030] In one example, the calculation expression for correlation analysis is:

[0031]

[0032] Among them, ρ ik is the correlation coefficient between the jth and kth features; w i is the weight of the ith subject; They represent the standardized feature values ​​of the jth and kth features of the i-th subject respectively; is the average of the j-th and k-th features.

[0033] In one example, heterogeneous transfer learning technology is used to transfer fatigue recognition models learned from one or more source domain subjects to target domain subjects, including:

[0034] By minimizing the paradigm distance, the source domain fatigue recognition model and the target domain fatigue recognition model are aligned in terms of features;

[0035] For features of different importance in the source domain fatigue recognition model, a fully connected neural network is introduced to learn the weight value of each feature in the target task;

[0036] Calculate the weighted feature matching loss based on the weight value;

[0037] The source domain fatigue recognition model is migrated to the target domain fatigue recognition model for feature matching, and the feature matching factor is introduced to obtain the final loss function, which is then minimized.

[0038] In one example, the final loss function L wfk The calculation expression is:

[0039]

[0040] Where β represents the source domain fatigue recognition model parameters; x represents the input features of the source domain fatigue recognition model; w d,k Represents the weight value of each feature in the target task; v d,k Indicates the weight value of feature matching; i and N respectively represent the upper and lower limits of the feature serial number label; represents the weighted feature matching loss; G represents the output of the source domain fatigue recognition model after parameter β transformation; k (x) represents the output of the target domain fatigue recognition model; represents a fully connected neural network; φ represents the parameters of the fully connected neural network.

[0041] In one example, the EEG signal of the target domain subject is analyzed in real time according to the transferred fatigue recognition model, and a first fatigue recognition result is output, including:

[0042] The EEG signal is input into the fatigue recognition model, and then the linear combination output matrix of the hidden layer mapping is output;

[0043] Apply an activation function to the linear combination output matrix;

[0044] Quantify the uncertainty of fatigue identification models using prior distributions;

[0045] A likelihood function is constructed to measure the matching degree between fatigue recognition model parameters and observed data;

[0046] The information of prior distribution and likelihood function is integrated by using posterior distribution;

[0047] Solve for output weights using maximum a posteriori estimation;

[0048] The trained output weights are used to predict the new feature vector and output the first fatigue recognition result.

[0049] It should be further explained that the technical features corresponding to the above examples can be combined or replaced with each other to form a new technical solution.

[0050] The present invention also includes a computer program product, including a computer program, which, when executed by a processor, implements the steps of the intelligent driving fatigue identification method formed by cross-subject analysis and heterogeneous transfer learning formed by any one of the above examples or a combination of multiple examples.

[0051] The present invention also includes a terminal, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and when the processor runs the computer instructions, the steps of the intelligent driving fatigue identification method formed by cross-subject analysis and heterogeneous transfer learning formed by any one or more of the above examples are executed.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. In one example, through cross-subject analysis, fatigue characteristic patterns between different subjects can be captured, effectively overcoming the impact of individual differences, thereby improving the accuracy of fatigue recognition between different individuals; through heterogeneous transfer learning, it is possible to draw on knowledge between different tasks, reduce the model's dependence on specific subjects, and improve the model's adaptability and generalization ability between different subjects. At the same time, this application obtains multiple fatigue recognition results based on EEG signals, heart rate signals, and blood oxygen signals and performs multimodal fusion, which can capture fatigue characteristics from different physiological dimensions, thereby improving the accuracy of fatigue recognition.

[0054] 2. In one example, threshold processing of wavelet coefficients can effectively separate signals and noise, retain important signal features, and thus improve the quality of signal processing; the wavelet coefficients after threshold processing are then used to reconstruct the signal, and the main features of the signal are retained through weight factors and reconstructed wavelet basis functions, while noise is removed, achieving high-precision recovery of the signal.

[0055] 3. In one example, by decomposing the EEG signal into multiple sub-band signals, the characteristics of each sub-band signal can be analyzed more carefully, thereby extracting more accurate features for fatigue recognition; by calculating the differential entropy of each sub-band signal and performing weighted processing, a comprehensive feature reflecting the complexity of the EEG signal can be obtained. This comprehensive feature can more comprehensively characterize the characteristics of the EEG signal, thereby improving the accuracy of fatigue recognition.

[0056] 4. In one example, feature alignment and standardization can reduce the impact of individual differences, and recursive feature elimination and correlation analysis can accurately identify fatigue feature patterns that are common across subjects, thereby improving the universality and accuracy of fatigue recognition.

[0057] 5. In one example, by minimizing the paradigm distance and introducing a fully connected neural network to learn feature weights, the model can better adapt to the data characteristics of the subjects in the target domain, thereby improving the generalization ability of the model in the new domain; by calculating the weighted feature matching loss and introducing the feature matching factor, it can more accurately identify common fatigue feature patterns across subjects. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings. The accompanying drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0059] Figure 1 A flow chart of a method provided for an example of the present invention. DETAILED DESCRIPTION

[0060] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In the description of the present invention, it should be noted that the directions or positional relationships indicated by "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. are directions or positional relationships based on the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the use of ordinal numbers (e.g., "first and second", "first to fourth", etc.) is to distinguish objects, and is not limited to this order, and cannot be understood as indicating or implying relative importance.

[0062] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0064] In one example, if Figure 1 As shown in the figure, an intelligent driving fatigue recognition method based on cross-subject analysis and heterogeneous transfer learning is described by taking the application in the field of drone driving as an example. The method includes the following steps:

[0065] S11: Collect and preprocess EEG signals.

[0066] Among them, the EEG signal data of the subject is collected through the EEG signal sensor, preferably based on a single channel for EEG signal collection, and then pre-processing operations such as denoising and filtering are performed to obtain high-quality EEG signals.

[0067] S12: Perform feature extraction processing on the EEG signal to obtain EEG signal features related to the fatigue state.

[0068] Optionally, a convolutional neural network can be used to automatically extract EEG signal features related to fatigue status, or differential entropy combined with a multi-scale analysis method can be used to extract features.

[0069] S13: Analyze the EEG signal characteristics of different subjects, identify the common fatigue feature patterns across subjects, and train a fatigue recognition model based on the common fatigue features, such as a sparse Bayesian classification model. Then use heterogeneous transfer learning technology to transfer the fatigue recognition model learned from one or more source domain subjects to the target domain subjects to enhance the generalization ability of the model and improve recognition efficiency.

[0070] The source domain refers to a fatigue-related dataset consisting of a representative or similar group of subjects that have been studied. The target domain refers to another dataset consisting of one or more subjects who may have different backgrounds or characteristics and need to identify their fatigue status.

[0071] S14: Perform real-time analysis on the EEG signal of the target domain subject according to the transferred fatigue recognition model, and output a first fatigue recognition result.

[0072] The EEG signals of the target domain subjects are preprocessed and then input into the transferred fatigue recognition model, and then the EEG signal characteristics are analyzed to output the prediction result, i.e., the first fatigue recognition result.

[0073] S21: Collect and pre-process physiological signals, where the physiological signals include heart rate signals and blood oxygen signals.

[0074] While collecting EEG signals, the heart rate and blood oxygen data are also collected through the heart rate and blood oxygen sensors to ensure that the EEG signals, heart rate signals, and blood oxygen signals are kept in the same time period to achieve synchronous data collection.

[0075] S22: Outputting the second fatigue recognition result and the third fatigue recognition result of the current subject respectively according to the heart rate signal and the blood oxygen signal.

[0076] Optionally, for the heart rate signal, a heart rate recognition model, such as a convolutional neural network, can be designed, and trained with an existing heart rate signal data set, and the trained heart rate recognition model can be used to output a second fatigue recognition result. For the blood oxygen signal, a blood oxygen recognition model, such as a convolutional neural network, can be designed, and trained with an existing blood oxygen signal data set, and the trained blood oxygen recognition model can be used to output a third fatigue recognition result.

[0077] S3: Fusing the first fatigue recognition result, the second fatigue recognition result, and the third fatigue recognition result to obtain a final fatigue recognition result.

[0078] The fusion process may be a weighted fusion process, in which different weights are assigned according to the contribution of the first fatigue recognition result, the second fatigue recognition result, and the third fatigue recognition result to the final fatigue recognition result. The fusion calculation expression is:

[0079] p total =w1p EEG +w2p HR +w3p SpO2 , w1+w2+w3=1

[0080] Among them, p total represents the final fatigue recognition result; p EEG is the fatigue prediction result of EEG signal, i.e., the first fatigue recognition result; p HR is the fatigue prediction result of heart rate, i.e., the second fatigue recognition result; p SpO2 is the fatigue prediction result of blood oxygen, that is, the third fatigue recognition result; w1, w2, and w3 represent different weight values ​​respectively.

[0081] Optionally, steps S11-S14 and steps S21-S22 may be performed simultaneously or in a reversed order.

[0082] In one example, the EEG signal is preprocessed, including:

[0083] S111: Remove baseline drift and power frequency interference from EEG signals;

[0084] S112: Perform discrete wavelet transform on the EEG signal to obtain wavelet coefficients at different scales. The calculation expression is:

[0085]

[0086] Where W(j,k) represents the kth wavelet coefficient of the jth layer; x(n) represents the discretized EEG signal; ψ j,k (n) represents discrete wavelet function.

[0087] S113: Threshold processing is performed on the wavelet coefficients to remove noise components. The calculation expression is:

[0088]

[0089] in, represents the threshold of the kth wavelet coefficient of the jth layer; f(W(j,k),T(j,k)) represents the nonlinear mapping function; T(j,k) represents the adaptive threshold;

[0090] S114: Reconstruct the signal of the wavelet coefficients after threshold processing. For discrete wavelet transform, the reconstruction expression is:

[0091]

[0092] in, represents the EEG signal after wavelet reconstruction; w j,k represents the weight factor; Represented by discrete wavelet function ψ j,k (n) The corresponding reconstructed wavelet basis function.

[0093] In one example, feature extraction is to extract features from the EEG signal after wavelet reconstruction using differential entropy combined with a multi-scale analysis method, including the following steps:

[0094] S121: Use the empirical mode decomposition method to decompose the EEG signal reconstructed by wavelet into multiple sub-band signals X i (i=1,2,3,...,N);

[0095] S122: Estimate each subband signal X i The probability density function of i (x);

[0096] S123: Calculate each subband signal X i The differential entropy h(X i ), the calculation expression is:

[0097]

[0098] Here, log represents the natural logarithm.

[0099] S124: The weighted average of the differential entropy of all sub-band signals is taken as the final feature representation H'(X i )(EEG signal characteristics), the calculation expression is:

[0100]

[0101] Among them, λ i Represents the weight factor of the differential entropy of the ith subband signal.

[0102] In one example, EEG signal features of different subjects are analyzed to identify common fatigue feature patterns across subjects, including:

[0103] S131: performing alignment processing on the extracted EEG signal features of different subjects, such as using domain adaptation technology to perform alignment processing, to ensure that the features of different subjects are compared at the same scale;

[0104] S132: Standardize the aligned EEG signal features to eliminate individual differences between different subjects. The calculation expression for the standardization process is:

[0105]

[0106] Among them, X ij is the jth eigenvalue of the ith subject; μ j is the mean of the jth feature, σ j is the standard deviation of the jth feature.

[0107] S133: Use the recursive feature elimination algorithm to select the most discriminative features of different subjects. The calculation expression is:

[0108]

[0109] Among them, x i is the feature vector of the i-th sample; i and N represent the upper and lower limits of the serial number label respectively; y j is the corresponding label; W is the weight vector; T represents transpose.

[0110] S134: Use correlation analysis to compare the characteristics between different subjects and identify the common fatigue characteristic patterns across subjects. The calculation expression is:

[0111]

[0112] Among them, ρ ik is the correlation coefficient between the jth and kth features; w i is the weight of the ith subject; They represent the standardized feature values ​​of the jth and kth features of the i-th subject respectively; is the average of the j-th and k-th features.

[0113] In one example, heterogeneous transfer learning technology is used to transfer fatigue recognition models learned from one or more source domain subjects to target domain subjects, including:

[0114] S135: By minimizing the paradigm distance, the source domain fatigue recognition model and the target domain fatigue recognition model are promoted to be aligned as much as possible in terms of features. The calculation expression for minimizing the paradigm distance is:

[0115]

[0116] Among them, β represents the source domain fatigue identification model parameter; represents the output of the source domain fatigue recognition model after parameter β transformation, G k (x) represents the output of the target domain fatigue recognition model; λ is the regularization coefficient.

[0117] S136: For features of different importance in the source domain fatigue recognition model, a layer of fully connected neural network is introduced to learn the weight value of each feature in the target task. The obtained weight value expression is:

[0118]

[0119] in, represents a fully connected neural network, and φ represents the parameters of the fully connected neural network.

[0120] S137: Calculate the weighted feature matching loss according to the weight value to remove the meaningless information to the target domain fatigue recognition model from the source domain fatigue recognition model, so as to more accurately focus on the truly valuable features of the target task. The weighted feature matching loss calculation expression is:

[0121]

[0122] in, Represents the weighted feature matching loss function.

[0123] S138: Perform feature matching on the source domain fatigue recognition model and the target domain fatigue recognition model, introduce feature matching factors, obtain the final loss function, and minimize the final loss function to optimize the target domain fatigue recognition model. wfk The calculation expression is:

[0124]

[0125] Where x represents the input feature of the source domain fatigue recognition model; w d,k Represents the weight value of each feature in the target task; v d,k Indicates the weight value of feature matching.

[0126] In one example, the EEG signal of the target domain subject is analyzed in real time according to the transferred fatigue recognition model, and a first fatigue recognition result is output, including:

[0127] S141: Input the EEG signal into the fatigue recognition model (classifier), and then output the linear combination output matrix H of the hidden layer mapping. The calculation expression is:

[0128]

[0129] Among them, X is the feature matrix; W is the input weight matrix; b is the hidden layer bias term; λ is the regularization parameter.

[0130] S142: Apply activation function to the linear combination output matrix, and the calculation expression is:

[0131] g(H)=max(H,0)

[0132] Among them, g represents the activation function.

[0133] S143: Use prior distribution to quantify the uncertainty of fatigue identification model. The calculation expression is:

[0134]

[0135] Among them, p(β') represents the prior distribution function; β' is the output weight; Λ is the diagonal matrix; λ i is the scale parameter of the i-th weight; a i 、b i Both are dynamically adjusted i Parameters; σ 2 is the noise variance; y i is the i-th label value of the fatigue state.

[0136] S144: Construct a likelihood function to measure the matching degree between fatigue recognition model parameters and observed data. The calculation expression is:

[0137] p(y|H,β')=T'(y|Hβ',σ 2 I,v)

[0138] Where p(y|H,β') represents the likelihood function; y is the label vector of fatigue state; I is the unit matrix; v is the degree of freedom parameter; and T' represents the t-distribution.

[0139] S145: To obtain the latest estimate of the parameters after observing the data, the posterior distribution is used to combine the information of the prior distribution and the likelihood function. The calculation expression of the posterior distribution is:

[0140]

[0141] Where p(β'|y,H) represents the posterior distribution function; Z represents a constant.

[0142] S146: Use the maximum a posteriori estimation to solve the output weight, and the calculation expression is:

[0143] β' (t+1) =β' (t) +η▽(logp(y|H,β')+logp(β'))

[0144] Among them, η is the learning rate and ▽ is the gradient.

[0145] S147: Use the trained output weights to predict the new feature vector and output the first fatigue recognition result. The expression is

[0146]

[0147] Among them, b is the global bias vector; R is the residual matrix; ∈ is the random error vector.

[0148] The above examples are combined to obtain a preferred example of the present invention, in which the method comprises the following steps:

[0149] S11': Collect and pre-process EEG signals, including the following sub-steps:

[0150] S111': Remove baseline drift and power frequency interference from EEG signals;

[0151] S112': Perform discrete wavelet transform on the EEG signal to obtain wavelet coefficients at different scales;

[0152] S113': threshold processing is performed on the wavelet coefficients to remove noise components;

[0153] S114': reconstruct the signal based on the wavelet coefficients after threshold processing.

[0154] S12': performing feature extraction processing on the EEG signal to obtain EEG signal features related to the fatigue state, including the following sub-steps:

[0155] S121': Decompose the EEG signal after wavelet reconstruction into multiple sub-band signals using the empirical mode decomposition method;

[0156] S122': Estimate the probability density function of each sub-band signal;

[0157] S123': Calculate the differential entropy of each subband signal;

[0158] S124': Take the weighted average of the differential entropies of all sub-band signals as the final feature representation.

[0159] S13': Analyze the EEG signal characteristics of different subjects, identify the common fatigue feature patterns across subjects, and train a fatigue recognition model based on the common fatigue features. Then, use heterogeneous transfer learning technology to transfer the fatigue recognition model learned from one or more source domain subjects to the target domain subjects. Specifically, it includes the following sub-steps:

[0160] S131': aligning the extracted EEG signal features of different subjects;

[0161] S132': standardize the aligned EEG signal features;

[0162] S133': Use recursive feature elimination algorithm to select the most discriminative features of different subjects;

[0163] S134': Use correlation analysis to compare the characteristics between different subjects and identify common fatigue characteristic patterns across subjects;

[0164] S135': Promote the source domain fatigue recognition model and the target domain fatigue recognition model to be aligned as much as possible in terms of features by minimizing the paradigm distance;

[0165] S136': For features of different importance in the source domain fatigue recognition model, a layer of fully connected neural network is introduced to learn the weight value of each feature in the target task;

[0166] S137': Calculate weighted feature matching loss according to the weight value;

[0167] S138': The source domain fatigue recognition model is migrated to the target domain fatigue recognition model for feature matching, and a feature matching factor is introduced to obtain a final loss function, and the final loss function is minimized to achieve optimization of the target domain fatigue recognition model.

[0168] S14': Performing real-time analysis on the EEG signal of the subject in the target domain according to the transferred fatigue recognition model, and outputting a first fatigue recognition result, including the following sub-steps:

[0169] S141': input the EEG signal into the fatigue recognition model, and then output the linear combination output matrix of the hidden layer mapping;

[0170] S142': Apply activation function to the linear combination output matrix;

[0171] S143': Quantifying uncertainty in fatigue identification models using prior distributions;

[0172] S144': construct a likelihood function to measure the matching degree between fatigue identification model parameters and observed data;

[0173] S145': The information of the prior distribution and the likelihood function is integrated using the posterior distribution method;

[0174] S146': Solve for output weights using maximum a posteriori estimation;

[0175] S147': Use the trained output weights to predict the new feature vector and output a first fatigue recognition result.

[0176] S21': collecting and preprocessing physiological signals, where the physiological signals include heart rate signals and blood oxygen signals.

[0177] S22': outputting the second fatigue recognition result and the third fatigue recognition result of the current subject respectively according to the heart rate signal and the blood oxygen signal.

[0178] S3': fusing the first fatigue recognition result, the second fatigue recognition result, and the third fatigue recognition result to obtain a final fatigue recognition result.

[0179] The present invention can capture more physiological and behavioral characteristics related to fatigue through the fusion of single-channel EEG acquisition and multimodal information (such as heart rate, blood oxygen, etc.), as well as cross-subject analysis and heterogeneous transfer learning, effectively overcome the impact of individual differences, and thus significantly improve the accuracy of fatigue recognition. At the same time, the method of the present invention can draw on knowledge between different tasks or fields, reduce the model's dependence on specific subjects, and improve the model's adaptability and generalization ability among different subjects. Furthermore, through heterogeneous transfer learning technology, the knowledge learned from one data source can be transferred to another data source, even if the two data sources differ in format, features, etc., effective data integration and information extraction can be achieved.

[0180] Furthermore, the present invention adopts single-channel EEG acquisition, which greatly simplifies the data acquisition process compared with traditional multi-channel EEG acquisition, and can achieve efficient fatigue recognition on a limited data set, reducing the reliance on large-scale labeled data. In addition, the method of the present invention can timely detect the fatigue state of the driver through accurate fatigue recognition, and provide timely warning and intervention for intelligent driving such as drone flight, thereby effectively avoiding flight accidents caused by driver fatigue and improving the safety of drone flight. The application of the present invention in the fields of EEG signal processing, artificial intelligence and machine learning technology not only provides a new solution for fatigue recognition of drone driving, but also provides a useful reference and reference for fatigue monitoring in other fields (such as car driving, remote operation, etc.).

[0181] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the intelligent driving fatigue identification method for cross-subject analysis and heterogeneous transfer learning formed by any one of the above examples or a combination of multiple examples. The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0182] The present invention also provides a terminal, which has the same inventive concept as any example or combination of multiple examples corresponding to the above-mentioned intelligent driving fatigue identification method for cross-subject analysis and heterogeneous transfer learning, including a memory and a processor, the memory stores computer instructions that can be run on the processor, and the processor executes the steps of the above-mentioned intelligent driving fatigue identification method for cross-subject analysis and heterogeneous transfer learning when running the computer instructions. The processor can be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0183] In one example, the terminal, i.e., the electronic device, is presented in the form of a general-purpose computing device, and the components of the electronic device may include but are not limited to: at least one processing unit (processor) mentioned above, at least one storage unit mentioned above, and a bus connecting different system components (including storage units and processing units).

[0184] The storage unit stores a program code, and the program code can be executed by the processing unit, so that the processing unit performs the steps of various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification. For example, the processing unit can perform the above-mentioned intelligent driving fatigue recognition method of cross-subject analysis and heterogeneous transfer learning.

[0185] The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit, and may further include a read-only memory unit (ROM).

[0186] The storage unit may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0187] The bus may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0188] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0189] Through the above description, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to this exemplary embodiment can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method of the exemplary embodiment of the present application.

[0190] The above specific implementation methods are detailed descriptions of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. An intelligent driving fatigue recognition method based on cross-subject analysis and heterogeneous transfer learning, characterized in that: The following steps are involved: Collect and pre-process EEG signals and physiological signals, including heart rate signals and blood oxygen signals; Perform feature extraction processing on the EEG signal to obtain the EEG signal features; Analyze the EEG signal characteristics of different subjects, identify the common fatigue feature patterns across subjects, and train a fatigue recognition model based on the common fatigue features. Then use heterogeneous transfer learning technology to transfer the fatigue recognition model learned from one or more source domain subjects to the target domain subjects. Performing real-time analysis on the EEG signals of the target domain subjects according to the transferred fatigue recognition model, and outputting a first fatigue recognition result; Outputting the second fatigue recognition result and the third fatigue recognition result of the current subject respectively according to the heart rate signal and the blood oxygen signal; The first fatigue recognition result, the second fatigue recognition result, and the third fatigue recognition result are fused to obtain a final fatigue recognition result.

2. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 1 is characterized in that: Preprocess the EEG signals, including: The EEG signal is processed by discrete wavelet transform to obtain wavelet coefficients at different scales; The threshold value of the wavelet coefficients is calculated as follows: in, represents the threshold of the kth wavelet coefficient of the jth layer; W(j,k) represents the kth wavelet coefficient of the jth layer; f(W(j,k),T(j,k)) represents the nonlinear mapping function; T(j,k) represents the adaptive threshold; The signal is reconstructed from the wavelet coefficients after threshold processing. The reconstruction expression is: in, represents the EEG signal after wavelet reconstruction; w j,k represents the weight factor; Represents the reconstructed wavelet basis function.

3. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 1 is characterized in that: Perform feature extraction on EEG signals, including: The EMD method is used to decompose the EEG signal after wavelet reconstruction into multiple sub-band signals; Estimate the probability density function of each subband signal; Calculate the differential entropy of each subband signal; The differential entropy of all sub-band signals is weighted to obtain the EEG signal features.

4. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 1 is characterized in that: Analyze the EEG signal characteristics of different subjects and identify common fatigue characteristic patterns across subjects, including: Align the extracted EEG signal features of different subjects; The aligned EEG signal features are standardized; The recursive feature elimination algorithm was used to screen the most discriminative features of different subjects; Correlation analysis was used to compare the characteristics between different subjects and identify common fatigue characteristic patterns across subjects.

5. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 4 is characterized in that: The calculation expression of correlation analysis is: Among them, ρ ik is the correlation coefficient between the jth and kth features; w i is the weight of the ith subject; They represent the standardized feature values ​​of the jth and kth features of the i-th subject respectively; is the average of the j-th and k-th features.

6. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 1 is characterized in that: Heterogeneous transfer learning technology is used to transfer the fatigue recognition model learned from one or more source domain subjects to the target domain subjects, including: By minimizing the paradigm distance, the source domain fatigue recognition model and the target domain fatigue recognition model are aligned in terms of features; For features of different importance in the source domain fatigue recognition model, a fully connected neural network is introduced to learn the weight value of each feature in the target task; Calculate the weighted feature matching loss based on the weight value; The source domain fatigue recognition model is migrated to the target domain fatigue recognition model for feature matching, and the feature matching factor is introduced to obtain the final loss function, which is then minimized.

7. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 6 is characterized in that: The final loss function L wfk The calculation expression is: Where β represents the source domain fatigue recognition model parameters; x represents the input features of the source domain fatigue recognition model; w d,k Represents the weight value of each feature in the target task; v d,k Indicates the weight value of feature matching; i and N respectively represent the upper and lower limits of the feature serial number label; represents the weighted feature matching loss; G represents the output of the source domain fatigue recognition model after parameter β transformation; k (x) represents the output of the target domain fatigue recognition model; represents a fully connected neural network; φ represents the parameters of the fully connected neural network.

8. The intelligent driving fatigue identification method based on cross-subject analysis and heterogeneous transfer learning according to claim 1 is characterized in that: The EEG signals of the subjects in the target domain are analyzed in real time according to the transferred fatigue recognition model, and the first fatigue recognition result is output, including: The EEG signal is input into the fatigue recognition model, and then the linear combination output matrix of the hidden layer mapping is output; Apply an activation function to the linear combination output matrix; Quantify the uncertainty of fatigue identification models using prior distributions; A likelihood function is constructed to measure the matching degree between fatigue recognition model parameters and observed data; The information of prior distribution and likelihood function is integrated by using posterior distribution; Solve for output weights using maximum a posteriori estimation; The trained output weights are used to predict the new feature vector and output the first fatigue recognition result.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the intelligent driving fatigue identification method of cross-subject analysis and heterogeneous transfer learning described in any one of claims 1-8 are implemented.

10. A terminal comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, characterized in that: When the processor runs the computer instructions, the steps of the intelligent driving fatigue identification method of cross-subject analysis and heterogeneous transfer learning described in any one of claims 1-8 are performed.

Images