Method for training human-centered intelligent physiological state recognition model, method and apparatus for physiological state recognition, and device

Abstract

The embodiments of the present application provide a method for training a human-centered intelligent physiological state recognition model, a method and apparatus for physiological state recognition, and a device. The method comprises: acquiring a first physiological signal, a first state label of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state label of the second physiological signal, wherein the second physiological signal is a physiological signal generated according to the first physiological signal, the first physiological signal is an acquired physiological signal, the first state label is configured for indicating a physiological state corresponding to the first physiological signal, and the second state label is configured for indicating a physiological state corresponding to the second physiological signal; and training, according to the first physiological signal, the second physiological signal, the first state label, and the second state label, the physiological state recognition model. The embodiments of the present application can achieve accurate physiological state recognition of a subject.

Description

Training method of human factor intelligent physiological state recognition model, physiological state recognition method, device and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the Chinese patent application filed with the China Patent Office on November 30, 2023, with application number 202311628678.X and invention name “Training method, physiological state recognition method, device and equipment for human intelligent physiological state recognition model”, and the Chinese patent application filed with the China Patent Office on December 26, 2023, with application number 202311810471.4 and invention name “Physiological and psychological state processing method, physiological and psychological state processing model and device”, all of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of physiological signal recognition, and in particular to a training method for a human-based intelligent physiological state recognition model, a physiological state recognition method, a device and equipment. Background Art

[0004] Physiological signals can reflect a person's mental state, emotional state, and health status (e.g., whether a disease exists). Accurately identifying a user's physiological state based on their physiological signals is a challenge. Summary of the Invention

[0005] The present application provides a training method, physiological state recognition method, device and equipment for a human factors intelligent physiological state recognition model, which can more accurately realize the recognition of a person's mental state, emotional state, or health state and other physiological states.

[0006] In a first aspect, an embodiment of the present application provides a training method for a human-based intelligent physiological state recognition model, comprising: obtaining a first physiological signal, a first state label of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state label of the second physiological signal, wherein the second physiological signal is a physiological signal generated based on the first physiological signal; the first physiological signal is a collected physiological signal; the first state label is used to indicate the physiological state corresponding to the first physiological signal; the second state label is used to indicate the physiological state corresponding to the second physiological signal; and training a physiological state recognition model based on the first physiological signal, the second physiological signal, the first state label, and the second state label. In addition to using the collected real physiological signal, i.e., the first physiological signal, to train the physiological state recognition model, the method also uses the second physiological signal generated based on the first physiological signal to train the physiological state recognition model, thereby suppressing the individual differences of the physiological signals to a certain extent, thereby being able to use relatively fewer real physiological signals to train a physiological state recognition model with relatively higher accuracy. When the physiological state recognition model is used to identify a person's mental state, emotional state, or health state, etc., the recognition result has relatively higher accuracy.

[0007] In a second aspect, embodiments of the present application provide a physiological state recognition method, comprising: obtaining a physiological signal from a user; inputting the physiological signal into a preset physiological state recognition model, the physiological state recognition model being used to recognize the physiological state based on the physiological signal; training the preset physiological state recognition model using any of the methods described in the first aspect; and obtaining a physiological state recognition result output by the physiological state recognition model as the physiological state recognition result of the user. Because the above method trains a physiological state recognition model with relatively higher accuracy, the method achieves relatively higher accuracy in recognizing a person's mental state, emotional state, or health state, or other physiological state using the physiological state recognition model.

[0008] In a third aspect, an embodiment of the present application provides a training device for a human factor intelligent physiological state recognition model, comprising:

[0009] an acquisition module, configured to acquire a first physiological signal, a first state tag of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state tag of the second physiological signal, wherein the second physiological signal is a physiological signal generated based on the first physiological signal; the first physiological signal is a collected physiological signal; the first state tag is used to indicate the physiological state corresponding to the first physiological signal; and the second state tag is used to indicate the physiological state corresponding to the second physiological signal;

[0010] The training module is used to train a physiological state recognition model according to the first physiological signal, the second physiological signal, the first state label and the second state label.

[0011] In a fourth aspect, an embodiment of the present application provides a physiological state recognition device, comprising:

[0012] An acquisition module, used to acquire the user's physiological signals;

[0013] An identification module is used to input physiological signals into a preset physiological state recognition model, and the physiological state recognition model is used to identify the physiological state based on the physiological signals; the preset physiological state recognition model is trained by any method of the first aspect; and the physiological state recognition result output by the physiological state recognition model is obtained as the user's physiological state.

[0014] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory; wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the processor, enable the electronic device to execute any one of the methods of the first aspect or the second aspect.

[0015] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method of any one of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0017] FIG2 is a flow chart of a method for training a human factor intelligent physiological state recognition model according to an embodiment of the present application;

[0018] FIG3 is a schematic diagram of the structure of a GAN network provided in an embodiment of the present application;

[0019] FIG4 is a flow chart of a training method for a generator provided in an embodiment of the present application;

[0020] FIG5A is another schematic diagram of a flow chart of a generator training method according to an embodiment of the present application;

[0021] FIG5B is another flowchart of a training method for a physiological state recognition model provided in an embodiment of the present application;

[0022] FIG5C is a schematic diagram of an electrocardiogram signal generated by a trained generator according to an embodiment of the present application;

[0023] FIG6 is a flow chart of a physiological state recognition method provided in an embodiment of the present application;

[0024] FIG7 is a schematic structural diagram of a training device for a human factor intelligent physiological state recognition model provided by an embodiment of the present application;

[0025] FIG8 is a schematic structural diagram of a physiological status recognition device provided in an embodiment of the present application;

[0026] FIG9 is a schematic diagram of data decomposition provided in an embodiment of the present application;

[0027] FIG10 is a schematic diagram of a physiological state recognition model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The preferred embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the scope of protection of the present application.

[0029] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] It should be noted that, in the description of this application, the terms "first" and "second" are merely for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore should not be understood as limiting this application. In addition, the term "and / or" in this application is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally indicates that the related objects before and after are in an "or" relationship.

[0031] Example 1

[0032] Human factors intelligence combines human factors engineering with artificial intelligence (AI). Human factors engineering applies human psychology and physiology to the engineering and design of products, processes, and systems. In other words, it is the technology for designing and improving human-machine-environment systems based on human characteristics. Combining this technology with artificial intelligence can enable machines to interact with humans more intelligently. For example, in some scenarios, a person's physiological state can be identified based on their physiological signals.

[0033] Physiological signals such as electrocardiogram (ECG) signals can reflect a person's mental state, emotional state, and health status (e.g., whether a disease exists). Accurately identifying a user's physiological state based on their physiological signals is a challenge.

[0034] After analysis, it is found that physiological signals have individual differences. Suppressing the individual differences of physiological signals is a way to improve the accuracy of identifying the user's physiological state.

[0035] To this end, an embodiment of the present application provides a training method for a human-based intelligent physiological state recognition model. The trained physiological state recognition model can suppress the individual differences in physiological signals to a certain extent, so that when the physiological state recognition model is used to identify the physiological state of a person based on the physiological signal of the person, the recognition result is relatively more accurate.

[0036] Furthermore, the embodiment of the present application also provides a physiological state recognition method, which can use the physiological state recognition model to identify the physiological state of a person based on the physiological signals of the person. Since the physiological state recognition model can suppress the individual differences of physiological signals to a certain extent, the recognition results of the physiological state recognition method of the embodiment of the present application can be made relatively more accurate.

[0037] As shown in Figure 1, which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device 100 includes: a processor 110, a memory 120, etc.

[0038] Optionally, in order to improve the functions of the electronic device, the electronic device may also include: a display screen, a camera, a speaker, an antenna, a mobile communication module, a wireless communication module, an audio module, a receiver, a microphone, a headphone interface, a charging management module, a power management module, a battery, etc. One or more devices, the embodiment of the present application is not limited.

[0039] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an ISP, a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0040] The electrical signal transmitted by the camera can be converted into a digital image signal by the ISP. The ISP can output the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or other format.

[0041] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0042] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0043] The memory 120 can be used to store computer executable program codes, which include instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data (such as audio data, etc.) created during the use of the electronic device 100, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the memory 120, and / or instructions stored in a memory provided in the processor.

[0044] It should be noted that, in the embodiment shown in FIG1 , the memory 120 is set in the electronic device 100 as an example. In other embodiments provided in the embodiments of the present application, the above-mentioned memory 120 may not be set in the electronic device 100. In this case, the memory 120 can be connected to the electronic device 100 through the interface provided by the electronic device 100, and then can be connected to the processor 110 in the electronic device 100.

[0045] The following will describe in detail the training method of the physiological state recognition model and the physiological state recognition method of the embodiment of the present application in conjunction with the structure of the above-mentioned electronic device. It is understood that the electronic device in the embodiment of the present application can also be used to execute the vital sign detection method based on human intelligence shown in Example 2 and / or the human information processing method based on target object recognition shown in Example 3.

[0046] The training method for the human-factor intelligent physiological state recognition model provided in the embodiment of the present application can be performed by an electronic device having the above-mentioned structure. The physiological state recognition method provided in the embodiment of the present application can also be performed by an electronic device having the above-mentioned structure. It is understandable that the electronic device that performs the training method for the human-factor intelligent physiological state recognition model and the electronic device that performs the physiological state recognition method can be the same electronic device or different electronic devices.

[0047] The physiological signals in the embodiments of the present application may include electrocardiogram signals or electroencephalogram signals.

[0048] The physiological state in the embodiments of the present application may include: mental state, emotional state, or health state, etc. For example, the mental state may specifically include concentration, lack of concentration, etc., the emotional state may specifically include happiness, calmness, depression, etc., and the health state may specifically include health, unhealthy, etc.

[0049] FIG2 is a flow chart of a method for training a human factor intelligent physiological state recognition model according to an embodiment of the present application. As shown in FIG2 , the method may include:

[0050] Step 201: Acquire a first physiological signal, a first state label of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state label of the second physiological signal under each physiological state.

[0051] The first physiological signal is a physiological signal collected by a physiological detection device such as an electrocardiograph or an electroencephalogram, in other words, a real physiological signal collected from a person. The first state label is used to indicate the physiological state corresponding to the first physiological signal.

[0052] Optionally, the timing and subject of collecting the first physiological signal may vary depending on the application domain of the physiological state recognition model. For example, if the physiological state recognition model is applied in the field of autonomous driving, such as an autonomous driving assistance system, the first physiological signal may be obtained by using a portable biometric information collection device to collect physiological signals of the driver while driving.

[0053] The second physiological signal is a physiological signal generated based on the first physiological signal. The second state tag is used to indicate the physiological state corresponding to the second physiological signal.

[0054] Optionally, the first state label of the first physiological signal is the same as the second state label of the second physiological signal corresponding to the first physiological signal. For example, if the physiological state is a mental state, if the first state label is mental concentration, the second state label is also mental concentration.

[0055] Optionally, different numerical values ​​may be used to identify different physiological states. For example, assuming that the physiological state includes three types, 1, 2, and 3 may be used to represent a physiological state respectively.

[0056] It can be understood that there is a corresponding relationship between the above-mentioned first physiological signal, the first state label, the second physiological signal and the second state label. For the sake of convenience, they are referred to as a set of signal data (first physiological signal, second physiological signal, first state label and second state label) in the embodiment of the present application. For example, for a first physiological signal a1, it has a corresponding second physiological signal b1, a first state label t1, and the second physiological signal b1 has a corresponding second state label p1. Accordingly, it constitutes a set of signal data (a1, b1, t1, p1). Accordingly, in this step, multiple sets of signal data under each physiological state are to be obtained. For example, assuming that the physiological state is a mental state, which is divided into two categories: mental concentration and mental inattention, then in this step, multiple sets of signal data under the mental state of mental concentration can be obtained, and multiple sets of signal data under the mental state of mental inattention can be obtained.

[0057] In one embodiment, a generator for generating physiological signals can be pre-trained. The specific training method for the generator is described in detail in the embodiment shown in FIG. 4 and is not further described here. Therefore, the second physiological signal in this step can be a physiological signal generated by the generator. Specifically, the second physiological signal can be generated in advance using the generator before step 201, or it can be generated in real time based on the first physiological signal during the execution of step 201.

[0058] Optionally, when the second physiological signal is generated in real time according to the first physiological signal during step 201, obtaining the second physiological signal corresponding to the first physiological signal in this step may include:

[0059] generating a feature vector of a second physiological signal based on the first physiological signal and a first state label of the first physiological signal;

[0060] The feature vector is input into a preset generator, and the physiological signal generated by the generator according to the feature vector is used as the second physiological signal corresponding to the first physiological signal.

[0061] Optionally, the generator can be composed of a combination of a convolutional (Conv) layer and a linear (Linear) layer, and the specific structure is not limited in the embodiment of this application.

[0062] Optionally, generating a feature vector of the second physiological signal according to the first physiological signal and the first state label of the first physiological signal may specifically include:

[0063] extracting a preset feature of the first physiological signal;

[0064] generating a feature vector of the first physiological signal according to a preset feature of the first physiological signal and a first state label;

[0065] A feature vector of the second physiological signal is generated according to the feature vector of the first physiological signal.

[0066] Optionally, generating the feature vector of the second physiological signal according to the feature vector of the first physiological signal may specifically include: performing normalization processing on the feature vector of the first physiological signal to obtain the feature vector of the second physiological signal.

[0067] For example, if the first physiological signal is an electrocardiogram (ECG) signal, the time intervals between adjacent R waves in the ECG signal vary, i.e., heart rate variability (HRV). In other words, HRV refers to the variation in the difference between successive heartbeat cycles. Accordingly, the preset feature in the above step can be the HRV feature of the ECG signal. Specifically, extracting the preset feature of the first physiological signal can be extracting the HRV feature of the first ECG signal.

[0068] Optionally, the HRV feature may specifically include an HRV time domain feature of the first electrocardiogram signal. In some embodiments, the HRV feature may further include: an HRV frequency domain feature and / or an HRV nonlinear feature.

[0069] Among them, HRV time domain features may include: RR interval mean value mean_nni, RR interval standard deviation sdnn, standard deviation sdsd of differences between adjacent RR intervals, number nni_50 of consecutive RR intervals with interval differences greater than 50ms, expressed as nni_50 / number of RR intervals pnni_50, number nni_20 of consecutive RR intervals with interval differences greater than 20ms, expressed as nni_20 / number of RR intervals pnni_20, square root of the mean of the sum of squares of differences of adjacent RR intervals rmssd, absolute value of the median of RR interval differences median_nni, difference between maximum and minimum RR interval values ​​range_nni, coefficient of variation cvcd of consecutive differences, coefficient of variation cvnni, mean heart rate mean_hr, maximum heart rate max_hr, minimum heart rate min_hr, and / or heart rate standard deviation std_hr, etc.

[0070] The above-mentioned HRV frequency domain features may include: the variance lf of HRV at low frequency (0.04~0.15hz), the variance hf of HRV at high frequency (0.15~40hz), the ratio lf / hf of the variance of HRV at low frequency (0.04~0.15hz) to the variance of HRV at high frequency (0.15~40hz), normalized low-frequency power lf_nu, normalized high-frequency power hf_nu, total power density spectrum total_power, variance vlf of HRV at very low frequency (0.003~0.04hz), and / or HRV triangular index measurement value triangular_index.

[0071] The HRV nonlinear characteristics may include: measuring the baseline width tinn of the distribution with the triangle as the base, the ellipse minor axis sd1, the ellipse major axis sd2, the ratio of the ellipse major axis to the ellipse minor axis sd2 / sd1, the sympathetic nerve index csi, the vagus nerve index cvi, and / or the modified CSImodified_csi, etc.

[0072] Optionally, extracting the HRV feature of the first electrocardiogram signal may specifically include:

[0073] Extracting RR interval information of the first electrocardiogram signal;

[0074] The HRV feature of the first electrocardiogram signal is extracted according to the RR interval information of the first electrocardiogram signal.

[0075] The RR interval refers to the distance (time interval) between two adjacent R waves in an electrocardiogram signal. In the embodiment of the present application, the time interval between two adjacent R wave peaks is referred to as the RR interval. The R wave refers to the peak value in the electrocardiogram signal.

[0076] Optionally, in this step, the QRS waveform in the first ECG signal can be detected, the position of the R wave peak in each QRS waveform can be extracted, the distance between each two adjacent R wave peaks can be calculated, and the distance between each two adjacent R wave peaks can be divided by the sampling rate to obtain the RR interval list of the first ECG signal, the RR interval list including each RR interval RRI in the first ECG signal, wherein the i-th RR interval is recorded as RRI i , represents the time interval between the (i+1)th R wave peak and the i-th R wave peak in the first ECG signal.

[0077] Optionally, detecting the QRS waveform in the first electrocardiographic signal may be implemented using a Pan-Tompkins algorithm.

[0078] Taking the first physiological signal being an EEG signal as an example, the preset features for extracting the first physiological signal may specifically be extracting time domain features, frequency domain features, time-frequency features and / or nonlinear features of the EEG signal.

[0079] The time domain characteristics of the EEG signal may include: amplitude, variance, mean, root mean square, skewness, peak value, etc. of the EEG signal.

[0080] The frequency domain characteristics of EEG signals can include the power, power spectral density, energy, etc. of EEG signals in each frequency band: delta band (0-4 Hz), theta band (4-8 Hz), alpha band (8-13 Hz), beta band (13-25 Hz), and gamma band (25-50 Hz).

[0081] The time-frequency characteristics of the EEG signal may include: the frequency domain characteristics of the EEG signal of each time window in the EEG signal of multiple time windows obtained by dividing the EEG signal by the time-frequency analysis method. The frequency domain characteristics of the EEG signal of each time window can refer to the examples of the frequency domain characteristics of the EEG signal mentioned above, which will not be repeated here. The above-mentioned time-frequency analysis methods may include: short-time Fourier transform (STFT), wavelet transform (WT), Hilbert-Huang transform (HHT), etc.

[0082] The nonlinear characteristics of EEG signals may include: various types of entropy of EEG signals, correlation dimension, fractal dimension, etc. Various types of entropy of EEG signals may include: approximate entropy, wavelet entropy feature, asymmetric entropy feature, etc.

[0083] Optionally, generating a feature vector of the first physiological signal according to a preset feature of the first physiological signal and the first state label may specifically include:

[0084] Normalizing the first state label of the first physiological signal, and using the normalized result as the label component of the first physiological signal;

[0085] The preset features and the label component of the first physiological signal are respectively used as components in a feature vector to generate a feature vector of the first physiological signal.

[0086] Still taking the first physiological signal as the first ECG signal as an example, assuming that the HRV features extracted from the first ECG signal include all the above 31 HRV features, the above 31 features are respectively recorded as F1, F2, F3, ..., F 31 , the label component can be recorded as F 32 , then the feature vector F of the first ECG signal may include 32 components in total, including HRV features and label components.

[0087] The above-mentioned extraction of the HRV feature of the first electrocardiogram signal based on the RR interval information of the first electrocardiogram signal can be achieved using relevant technologies, which will not be described in detail in the embodiment of the present application.

[0088] Optionally, if the physiological state includes n types, and the first state label is represented by 1, 2, ..., n to represent different physiological states, then the normalization processing of the first state label can be the first state label / n. For example, the physiological state is a mental state, which includes 3 types, represented by 1, 2, and 3 respectively. Then, when the first state label is 1, the label component obtained after normalization is 1 / 3, when the first state label is 2, the label component obtained after normalization is 2 / 3, and when the first state label is 3, the label component obtained after normalization is 1.

[0089] Step 202: Train a physiological state recognition model according to the first physiological signal, the second physiological signal, the first state label, and the second state label.

[0090] Optionally, for each set of signal data (first physiological signal, second physiological signal, first state label and second state label), a training sample (first physiological signal, first state label) can be generated based on the first physiological signal and the first state label, and another training sample (second physiological signal, second state label) can be generated based on the second physiological signal and the second state label, and the above two training samples can be used to train the physiological state recognition model respectively.

[0091] In the above implementation method, for each group of signal data (first physiological signal, second physiological signal, first state label and second state label), two training samples (first physiological signal, first state label) and (second physiological signal, second state label) can be generated. Optionally, in order to improve the accuracy of the trained physiological state recognition model, for the two training samples corresponding to different groups of signal data, the order in which the above training samples are input into the physiological state recognition model can be disrupted, so that the order in which the two training samples of the same group of signal data are input into the physiological state recognition model is not adjacent. For example, assuming there are two training samples (first physiological signal 1, first state label) and (second physiological signal 1, second state label) generated by one set of signal data, and two training samples (first physiological signal 2, first state label) and (second physiological signal 2, second state label) generated by another set of signal data, by disrupting the order in which the above samples are input into the physiological state recognition model, for example, the order in which the above four samples are input into the physiological state recognition model can be made to be: training sample (first physiological signal 1, first state label), training sample (second physiological signal 2, second state label), training sample (second physiological signal 1, second state label), training sample (first physiological signal 2, first state label). In this way, the order in which the training sample (first physiological signal 1, first state label) and the training sample (second physiological signal 1, second state label) are input into the physiological state recognition model is not adjacent, and the order in which the training sample (first physiological signal 2, first state label) and the training sample (second physiological signal 2, second state label) are input into the physiological state recognition model is not adjacent. It can be understood that the order in which the above four training samples are input into the physiological state recognition model is only an example. In actual applications, the number of groups of signal data is greater and more training samples are generated. As long as the order in which the two training samples corresponding to the same group of signal data are input into the physiological state recognition model is not adjacent, the specific order in which multiple training samples are input into the physiological state recognition model is not limited in this embodiment of the application.

[0092] Optionally, the physiological state recognition model in the embodiment of the present application can be implemented by a Bayesian neural network combined with a translation encoder. Specifically, the physiological state recognition model in the embodiment of the present application can increase the parameter dimension calculated by each neuron in the translation encoder on the basis of the translation encoder. For example, the mean and variance of each neuron are fitted, thereby expanding the calculated parameters from 1 dimension to 2 dimensions, and obtaining the above-mentioned Bayesian neural network combined with the translation encoder. Based on the Bayesian neural network based on the convolutional or linear layer, the Bayesian neural network combined with the translation encoder is implemented, which can realize the conversion of the original translation encoder's point estimate of the parameter to the new translation encoder's parameter distribution estimate, effectively reducing the sensitivity of the physiological state recognition model to individual differences, and for physiological signals with a relatively long duration, it can more effectively capture long-term temporal dependencies, effectively combining the characteristics of physiological signals such as electrocardiogram signals with long-term temporal dependencies, that is, it can combine the "context" information of the physiological signal to give a more accurate recognition result, thereby improving the recognition accuracy of the physiological state recognition model.

[0093] Optionally, in this step, one or more training samples may be obtained each time, input into a preset physiological state recognition model, and the physiological state recognition model may be trained. The forward propagation and back propagation processes in the training process are not limited in this embodiment of the application and may be implemented with reference to related technologies.

[0094] In the physiological state recognition method of the embodiment of the present application, not only the collected real physiological signals are used to train the physiological state recognition model, but also the physiological signals generated based on the real physiological signals are used to train the physiological state recognition model. Therefore, relatively more physiological signals can be used to train the physiological state recognition model, and a relatively higher accuracy physiological state recognition model can be obtained by training with relatively fewer real physiological signals.

[0095] The generator for generating physiological signals in the embodiment of the present application can be trained by generative adversarial methods. Specifically, a generative adversarial network (GAN) can be pre-established in the embodiment of the present application. As shown in FIG3 , the GAN network may include a generator and a discriminator, and the physiological signal generated by the generator can be input into the discriminator for discrimination. The above-mentioned generator and discriminator can be implemented by neural networks respectively. The generator and discriminator initially established are untrained networks. The generator obtained after training by the method shown in FIG4 can be used as the generator for generating the second physiological signal in the embodiment of the present application shown in FIG2 .

[0096] As shown in Figure 4, the training method of the generator may include:

[0097] Step 401: Acquire a plurality of fourth physiological signals and fourth state labels of the fourth physiological signals in each physiological state.

[0098] The fourth physiological signal may be a physiological signal collected by a physiological detection device, in other words, a real physiological signal collected from a person. The fourth state tag is used to indicate the physiological state of the fourth physiological signal.

[0099] The number of fourth physiological signals in each physiological state is not limited in this embodiment of the application. It can be understood that the greater the number of fourth physiological signals, the better the performance of the generator obtained by training.

[0100] The implementation of the fourth state tag of the fourth physiological signal can refer to the above description of the first state tag and the second state tag, which will not be repeated here.

[0101] Optionally, the fourth physiological signal obtained in this step may be the same physiological signal as the first physiological signal in step 201 , or may be a different physiological signal, as long as it is a real physiological signal.

[0102] Step 402 : For each fourth physiological signal, generate a feature vector of a fifth physiological signal according to the fourth physiological signal and a fourth state label of the fourth physiological signal.

[0103] The implementation of this step can refer to the aforementioned description of generating the characteristic vector of the second physiological signal based on the first physiological signal and the first state label of the first physiological signal. The main difference is that the first physiological signal is replaced by the fourth physiological signal, the first state label is replaced by the fourth state label, and the second physiological signal is replaced by the fifth physiological signal. Details are not repeated here.

[0104] Step 403: For each feature vector of the fifth physiological signal, input the feature vector of the fifth physiological signal into a preset generator, and use the physiological signal output by the generator as the fifth physiological signal.

[0105] Through the processing of this step, the fifth physiological signal corresponding to each fourth physiological signal can be obtained.

[0106] Optionally, in this step, one characteristic vector of the fifth physiological signal can be obtained each time and input into a preset generator to generate a corresponding fifth physiological signal; or, in this step, multiple characteristic vectors of the fifth physiological signal can be obtained each time and input into a preset generator to generate multiple fifth physiological signals accordingly.

[0107] The generator in the embodiment of the present application generates a physiological signal based on the characteristic vector of the physiological signal, so that the physiological signal generated by the generator is relatively more in line with expectations, thereby improving the availability of the generated data.

[0108] Step 404: Train a preset discriminator according to the fourth physiological signal and the fifth physiological signal.

[0109] Optionally, this step may include:

[0110] Setting a first source tag for the fourth physiological signal, the first source tag being used to indicate that the fourth physiological signal is a real physiological signal; setting a second source tag for the fifth physiological signal, the second source tag being used to indicate that the fifth physiological signal is a physiological signal generated by a generator;

[0111] The discriminator is trained according to the fourth physiological signal, the first source label, the fifth physiological signal and the second source label.

[0112] Optionally, the first source tag and the second source tag may be implemented by 1 and 0 respectively, that is, 1 is used to indicate that the physiological signal is a real physiological signal, and 0 is used to indicate that the physiological signal is a generated physiological signal.

[0113] The discriminator is used to determine whether the received physiological signal is a real physiological signal or a generated physiological signal.

[0114] The training of the discriminator according to the fourth physiological signal, the first source label, the fifth physiological signal, and the second source label may specifically include:

[0115] generating a training sample for the discriminator based on the fourth physiological signal and the first source label of the fourth physiological signal, and generating a training sample for the discriminator based on the fifth physiological signal and the second source label of the fifth physiological signal;

[0116] The discriminator is trained using the above training samples.

[0117] Based on the above processing steps, for each fourth physiological signal, two training samples can be generated, which are assumed to be (fourth physiological signal, first source label) and (fifth physiological signal, second source label). Optionally, in order to improve the processing accuracy of the discriminator, when using the training samples to train the discriminator, the order in which the two training samples corresponding to the same fourth physiological signal are input into the discriminator for discriminator training is not adjacent. For example, assuming that the fourth physiological signal 1 corresponds to two training samples (fourth physiological signal 1, first source label) and (fifth physiological signal 1, second source label), and the fourth physiological signal 2 corresponds to two training samples (fourth physiological signal 2, first source label) and (fifth physiological signal 2, second source label), the order in which the above four training samples are input into the discriminator after being shuffled is: training sample (fourth physiological signal 1, first source label), training sample (fifth physiological signal 2, second source label), training sample (fifth physiological signal 1, second source label), training sample (fourth physiological signal 2, first source label). This allows the order in which the training samples (fourth physiological signal 1, first source label) and (fifth physiological signal 1, second source label) are input into the discriminator to be non-adjacent, and the order in which the training samples (fourth physiological signal 2, first source label) and (fifth physiological signal 2, second source label) are input into the discriminator to be non-adjacent. It is understood that the order in which the above four training samples are input into the discriminator is merely an example. In actual applications, the number of fourth physiological signals is greater, and more training samples are generated. As long as the order in which the two training samples corresponding to the same fourth physiological signal are input into the discriminator is non-adjacent, the specific order in which the multiple training samples are input into the discriminator is not limited in this embodiment of the application.

[0118] Among them, the specific implementation of using training samples to train the discriminator can be achieved using relevant training methods of the discriminator, which will not be described in detail in this application.

[0119] Step 405: Acquire multiple sixth physiological signals in each physiological state and a sixth state label of each sixth physiological signal.

[0120] The sixth physiological signal may be a physiological signal collected by a physiological detection device, in other words, a real physiological signal collected from a person. The sixth state tag is used to indicate the physiological state of the sixth physiological signal.

[0121] The number of sixth physiological signals in each physiological state is not limited in this embodiment of the application. It can be understood that the greater the number of sixth physiological signals, the better the performance of the generator obtained by training.

[0122] The implementation of the sixth state label of the sixth physiological signal can refer to the above description of the first state label and the second state label, which will not be repeated here.

[0123] Optionally, the sixth physiological signal obtained in this step may be the same physiological signal as the fourth physiological signal in step 401 , or may be a different physiological signal, as long as it is a real physiological signal.

[0124] Step 406: Based on the sixth physiological signal and the sixth state label of the sixth physiological signal, perform adversarial training on the generator and the discriminator to obtain a trained generator.

[0125] Optionally, this step may include:

[0126] For each sixth physiological signal, generating a feature vector of a seventh physiological signal according to the sixth physiological signal and a sixth state label of the sixth physiological signal;

[0127] Inputting the feature vector of each seventh physiological signal into the generator, and using the physiological signal outputted by the generator as the seventh physiological signal;

[0128] A first source tag is set for the sixth physiological signal, and a second source tag is set for the seventh physiological signal; the first source tag is used to indicate that the sixth physiological signal is a real physiological signal; the second source tag is used to indicate that the seventh physiological signal is a physiological signal generated by a generator;

[0129] generating a training sample for the discriminator based on the sixth physiological signal and the first source label of the sixth physiological signal, and generating a training sample for the discriminator based on the seventh physiological signal and the second source label of the seventh physiological signal;

[0130] The training samples of the discriminator are input into the discriminator, and the generator and discriminator are back-propagated according to the discrimination results output by the discriminator, that is, the parameters of the generator and discriminator are adjusted.

[0131] The implementation of the first source tag and the second source tag can refer to the above corresponding description and will not be repeated here.

[0132] Optionally, if the sixth physiological signal acquired in step 405 is the same physiological signal as the fourth physiological signal in step 401 , the step of generating a feature vector of the seventh physiological signal in the above steps may be omitted.

[0133] Among them, the implementation of generating the characteristic vector of the seventh physiological signal based on the sixth physiological signal and the sixth state label of the sixth physiological signal can refer to the relevant instructions for generating the characteristic vector of the second physiological signal based on the first physiological signal and the first state label of the first physiological signal. The main difference is that the first physiological signal is replaced by the sixth physiological signal, the first state label is replaced by the sixth state label, and the second physiological signal is replaced by the seventh physiological signal, which will not be repeated here.

[0134] Among them, the specific implementation of the above-mentioned inputting the characteristic vector of the seventh physiological signal into the generator and using the physiological signal output by the generator as the seventh physiological signal can refer to the description in step 403. The main difference is that the fifth physiological signal is replaced by the seventh physiological signal.

[0135] Among them, the above-mentioned back propagation of the generator and the discriminator based on the discrimination result output by the discriminator can be implemented using relevant technologies, which will not be elaborated in this application.

[0136] In this step, the generator and discriminator compete and collaborate with each other through adversarial training. The generator aims to generate physiological signals that are increasingly close to real physiological signals, making it difficult for the discriminator to accurately identify the source of the physiological signals. The discriminator aims to accurately distinguish whether the input physiological signal is a generator-generated signal or a real physiological signal. By adjusting the parameters of the generator and discriminator during adversarial training, the trained generator can ultimately generate physiological signals that are very close to real physiological signals, while the discriminator cannot accurately distinguish between the generated and real physiological signals.

[0137] In an embodiment of the present application, a generator is obtained by training through generative adversarial training, so that the generator can be used to generate physiological signals that are very close to the real ones for training the physiological state recognition model, thereby increasing the amount of sample data for training the physiological state recognition model and improving the diversity of the sample data. Therefore, when training the physiological state recognition model, the sensitivity of the physiological state recognition model to individual differences in physiological signals can be effectively reduced.

[0138] Below, the training method of the generator is illustrated by taking the physiological signal as the electrocardiogram signal, the physiological state as the mental state, and the mental state being divided into three types as an example.

[0139] As shown in FIG5A , the method may include:

[0140] Step 501: Acquire multiple real ECG signals X and state labels Y corresponding to the real ECG signals X in each mental state.

[0141] The following steps 502 to 504 are executed for each real ECG signal X.

[0142] Step 502: For each real ECG signal X, extract the RR interval information of the real ECG signal X.

[0143] Step 503: Generate a feature vector F of the real ECG signal X.

[0144] In this step, the HRV features of the real ECG signal X can be extracted based on the RR interval information of the real ECG signal X as the component of the feature vector F of the real ECG signal X, and the mental state label Y of the real ECG signal X can be normalized as the label component F in the feature vector F of the real ECG signal X. 32 .

[0145] Assuming that the HRV feature components include all 31 HRV features in the above example, the feature vector F of the real ECG signal X can include 31 of the above 31 HRV feature components, and the above 31 HRV feature components are respectively recorded as F1, F2, F3, ..., F 31 .

[0146] For example, assuming that there are three types of mental states based on ECG signals, the mental state label Y of the first type of mental state can be recorded as 1, the mental state label Y of the second type of mental state can be recorded as 2, and the mental state label Y of the third type of mental state can be recorded as 3. Then the normalized mental state label Y of the first type of mental state is scaled-1 It can be recorded as 1 / 3, the normalized mental state label Y of the second type of mental state scaled-2 It can be recorded as 2 / 3, the normalized mental state label Y of the third type of mental state scaled-3 It can be recorded as 1.

[0147] Through the processing of this step, corresponding to each real ECG signal X, its feature vector F may include F1~F 32 A total of 32 servings.

[0148] Step 504: normalize the eigenvector F of the real ECG signal X to obtain the eigenvector FG of the simulated ECG signal XG corresponding to the real ECG signal X.

[0149] The simulated ECG signal refers to an ECG signal generated based on a real ECG signal.

[0150] For example, the feature vector F of the real ECG signal X includes F1~F 32 Taking a total of 32 components as an example, the feature vector FG of the ECG signal to be generated also includes 32 components, which are assumed to be: FG1, FG2, FG3, ..., FG 32 , where the characteristic component FG i is the component F in the first eigenvector F i The processing result obtained by normalization processing, i ranges from 1 to 32.

[0151] By performing the above steps 502 to 505 on each real electrocardiogram signal X, the feature vector FG of the generated electrocardiogram signal XG corresponding to the real electrocardiogram signal X can be obtained. In other words, assuming there are N real electrocardiogram signals X, N feature vectors FG can be obtained.

[0152] Step 505: Randomly extract M feature vectors FG, and input the M extracted feature vectors FG into the generator to generate the simulated electrocardiogram signal XG corresponding to each feature vector FG.

[0153] Here, M is a natural number, and M < N. The specific value of M is not limited in the embodiments of this application.

[0154] It should be noted that each feature vector FG can be extracted only once, that is, input into the generator only once, as long as the simulated electrocardiogram signal XG corresponding to each feature vector FG is generated.

[0155] Through the processing of this step, N simulated electrocardiogram signals XG corresponding to N feature vectors FG can be obtained.

[0156] Step 506: The preliminary training step of the discriminator.

[0157] In this step, the first source label L1 can be set for each real electrocardiogram signal X to obtain the real electrocardiogram sample YR(X, L1), and the second source label L2 can be set for each simulated electrocardiogram signal XG to obtain the simulated electrocardiogram sample YG(XG, L2). Randomly input the real electrocardiogram sample YR and the simulated electrocardiogram sample YG into the discriminator respectively for the training of the discriminator to obtain the trained discriminator. Optionally, when inputting the real electrocardiogram sample YR and the simulated electrocardiogram sample YG into the discriminator, the input order of the real electrocardiogram sample YR and the simulated electrocardiogram sample YG corresponding to the same real electrocardiogram signal X is not adjacent.

[0158] The first source label L1 is used to indicate that the electrocardiogram signal is real data, and the second source label L2 is used to indicate that the electrocardiogram signal is generated data. For example, the first source label L1 can be 1, and the second source label L2 can be 0. Then the above real electrocardiogram sample YR can be (X, 1), and the above simulated electrocardiogram sample YG can be (XG, 0).

[0159] Step 507: The adversarial training step of the generator and the discriminator.

[0160] In the first example, if the multiple real ECG signals X obtained in step 501 are still used for adversarial training in this step, the feature vector FG can be input into the generator to obtain the simulated ECG signal XG generated by the generator, a first source label L1 is set for the real ECG signal X to obtain an adversarial training sample XF(X, L1), a second source label L2 is set for the simulated ECG signal XG to obtain an adversarial training sample YF(XG, L2), the adversarial training samples XF(X, L1) and YF(XG, L1) are respectively input into the discriminator to obtain the corresponding discrimination results output by the discriminator, and the generator and the discriminator are back-propagated according to the discrimination results, and the above training process is repeated until the ECG signal XG generated by the generator is sufficiently close to the real ECG signal and meets the preset training stop condition.

[0161] In the second example, real ECG signals other than step 501 can be used for adversarial training in this step. Steps 502 to 504 can be executed for each real ECG signal to obtain the feature vector of each real ECG signal. Afterwards, steps such as inputting the feature vector into the generator are executed in a similar manner to that in the first example to implement adversarial training of the generator and the discriminator.

[0162] In combination with the above embodiments, referring to the flow chart shown in FIG5B , in one embodiment provided by the present application, a feature vector of a simulated physiological signal can be first generated based on a real physiological signal and input into a generator. The generator and the discriminator are subjected to generative adversarial training based on the simulated physiological signal and the real physiological signal generated by the generator to obtain a trained generator. Afterwards, a feature vector of a simulated physiological signal is generated based on the real physiological signal and input into the trained generator. The generator outputs a simulated physiological signal that is close to the real one, and the physiological state recognition model is trained based on the real physiological signal and the corresponding simulated physiological signal. Referring to FIG5C , a schematic diagram of a 12-lead ECG signal generated by the trained generator in an embodiment of the present application is shown.

[0163] FIG6 is a flow chart of a physiological status recognition method provided in an embodiment of the present application. The method can be executed by an electronic device, such as an in-vehicle device for recognizing a driver's physiological status, or a medical device for recognizing a patient's health status. As shown in FIG6 , the method may include:

[0164] Step 601: Acquire the user's physiological signals.

[0165] The above-mentioned users may be, for example, drivers, medical personnel, etc., and different implementations may be made based on different applicable scenarios of the physiological state recognition method.

[0166] Step 602: Input the physiological signal into a preset physiological state recognition model, which is used to recognize the physiological state according to the physiological signal.

[0167] Optionally, the physiological state recognition model used in this step is a physiological state recognition model trained by the aforementioned method.

[0168] Step 603: Obtain the physiological state recognition result output by the physiological state recognition model as the physiological state recognition result of the user.

[0169] Since the physiological state recognition model of the embodiment of the present application can suppress the individual differences of physiological signals and has a relatively more accurate physiological state recognition result, the physiological state recognition method of the embodiment of the present application can also more accurately realize the recognition of the user's physiological state.

[0170] FIG7 is a schematic diagram of the structure of a training device for a human factor intelligent physiological state recognition model provided by an embodiment of the present application. As shown in FIG7 , the device 700 may include:

[0171] An acquisition module 710 is configured to acquire a first physiological signal, a first state tag of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state tag of the second physiological signal, wherein the second physiological signal is a physiological signal generated based on the first physiological signal; the first physiological signal is a collected physiological signal; the first state tag is used to indicate the physiological state corresponding to the first physiological signal; and the second state tag is used to indicate the physiological state corresponding to the second physiological signal.

[0172] The training module 720 is configured to train a physiological state recognition model according to the first physiological signal, the second physiological signal, the first state label, and the second state label.

[0173] FIG8 is a schematic diagram of the structure of a physiological state recognition device provided in an embodiment of the present application. As shown in FIG8 , the device 800 may include:

[0174] An acquisition module 810 is used to acquire a user's physiological signals;

[0175] The recognition module 820 is used to input the physiological signal into a preset physiological state recognition model, and the physiological state recognition model is used to identify the physiological state based on the physiological signal; the preset physiological state recognition model is trained by the method provided in the above embodiment; and the physiological state recognition result output by the physiological state recognition model is obtained as the user's physiological state.

[0176] The apparatus provided in the embodiments shown in FIG7 and FIG8 can be used to implement the technical solution of the method embodiment of the present application. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0177] It should be understood that the division of the various modules of the devices shown in Figures 7 and 8 above is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called through processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called through processing elements, and some modules can be implemented in the form of hardware. For example, the identification module can be a separately established processing element, or it can be integrated into a chip of an electronic device. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. During the implementation process, each step of the above method or each module above can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0178] Example 2

[0179] Physiological data is generated by the human body during activity. It can, to a certain extent, characterize a person's physiological state. Examples of physiological data include EEG data, galvano-dermal data, and electrocardiogram data. Examples of physiological states include fatigue, emotional state, and workload.

[0180] Illustratively, an embodiment of the present application provides a physiological state recognition method, for example, including steps one to five.

[0181] Step 1: Acquire multiple physiological time series data.

[0182] Exemplarily, physiological data include, for example, EEG data, skin electricity data, ECG data, and the like. Taking EEG data as an example, EEG data can be collected through EEG electrodes. Since EEG data is usually a kind of fluctuating data that changes with time, EEG data can be used as a kind of time series data (which can be called timing data). Physiological timing data is obtained by collecting physiological data. Each physiological timing data is collected through the corresponding data acquisition channel. For ease of understanding, the data acquisition channels of this application are taken as 16 as an example. Each data acquisition channel corresponds to an electrode. The 16 electrodes are respectively attached to different positions of the human head for data collection. The EEG data are collected through the 16 acquisition channels to obtain 16 physiological timing data (EEG data).

[0183] Step 2: perform feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data.

[0184] For example, each physiological time series data contains implicit features that conform to certain change rules, so each physiological time series data needs to be feature decomposed. Each physiological time series data can be decomposed into multiple subsequence data, and each subsequence data represents a change rule, for example.

[0185] Step three: extract features from the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0186] For each physiological time series data corresponding to the plurality of sub-sequence data, feature extraction is performed on the plurality of sub-sequence data to obtain initial feature data. The plurality of physiological time series data corresponds one to one to the plurality of initial feature data.

[0187] Step 4: Fusing the multiple initial feature data corresponding one-to-one to the multiple physiological time series data to obtain fused feature data.

[0188] The physiological time series data collected by each data acquisition channel is different. For example, the physiological time series data collected by some channels has more information or can better reflect the user's physiological state. The initial feature data corresponding to multiple channels are fused so that the fused feature data has more information.

[0189] Step 5: Predict the physiological state based on the fused feature data.

[0190] For example, physiological states include fatigue, emotional state, load bearing state, anxiety, relaxation, etc. In the embodiments of the present application, by decomposing physiological time series data, subsequence data representing the changing patterns of physiological time series data is obtained, and then feature extraction is further performed on the subsequence data to obtain richer initial feature data. Taking into account the differences in data collected by different channels, the initial feature data of different channels are fused before predicting the physiological state, thereby improving the accuracy of physiological state prediction.

[0191] It can be understood that the physiological time series data obtained in Example 2 of the present application belongs to the physiological signal in the above-mentioned Example 1, that is, the physiological signal in Example 1 may include multiple physiological time series data, so that the physiological signal can be predicted and other physiological state recognition processing can be performed according to the technical solution of the present application.

[0192] In another example, feature decomposition may be performed on each physiological time series data to obtain subsequence data for each physiological time series data.

[0193] For example, each physiological time series data item can be feature decomposed to obtain trend subsequence data and periodic subsequence data. Trend subsequence data represents the trend of physiological time series data over time, while periodic subsequence data represents the periodic variation characteristics of physiological time series data. Periodic subsequence data can also be called seasonal subsequence data. For example, if the physiological time series data is electrocardiogram (ECG) data, ECG data can represent the user's heartbeat to a certain extent, while periodic subsequence data can reflect the heartbeat cycle.

[0194] Then, at least one of the trend subsequence data and the periodic subsequence data is used as the subsequence data for each physiological time series data. For example, the trend subsequence data and the periodic subsequence data can be used as the subsequence data.

[0195] When performing feature decomposition, either additive or multiplicative decomposition can be used. Using the additive decomposition method, the trend subsequence data and the periodic subsequence data are added together to obtain physiological time series data. Using the multiplicative decomposition method, the trend subsequence data and the periodic subsequence data are multiplied together to obtain physiological time series data.

[0196] FIG9 is a schematic diagram of data decomposition provided in an embodiment of the present application.

[0197] As shown in Figure 9, taking 16-channel EEG data as an example, feature decomposition is performed on each of the 16 physiological time series data. Figure 9 shows that the feature decomposition of the EEG data of the third channel is performed to obtain trend subsequence data, periodic subsequence data (seasonal subsequence data), and error data, where the error data is, for example, white noise. The error data can be removed, and the trend subsequence data and periodic subsequence data can be retained as subsequence data for each physiological time series data.

[0198] When performing eigendecomposition, either additive or multiplicative decomposition can be used. Additive decomposition is used to obtain trend subsequence data, periodic subsequence data, and error data. Adding these data together yields physiological time series data. Multiplicative decomposition is used to obtain trend subsequence data, periodic subsequence data, and error data. Multiplying these data together yields physiological time series data.

[0199] It can be understood that the present application can choose the additive decomposition method or the multiplicative decomposition method according to the actual situation requirements. By decomposing the physiological time series data, subsequence data representing the change law of the physiological time series data can be obtained, and then the subsequence data can be further feature extracted to obtain richer initial feature data. Compared with the method of directly extracting features from the physiological time series data, the present application extracts features from the decomposed subsequence data, and can extract more important features at a deeper level. It can be seen that by extracting features after data decomposition, the effect of feature extraction is improved, thereby improving the accuracy of physiological state processing.

[0200] In another example, a neural network can be used to extract features of subsequence data. For example, the subsequence data corresponding to each physiological time series data is input into the neural network corresponding to each physiological time series data for feature extraction to obtain initial feature data.

[0201] Neural networks include, for example, long short-term memory (LSTM) networks, bidirectional long short-term memory (BiLSTM) networks, etc., and may also be other networks capable of extracting features from time series data.

[0202] As a deep learning method, neural networks possess powerful signal processing and recognition capabilities and can be applied to the analysis of physiological signals. When using neural networks to detect physiological signals, it is first necessary to select an appropriate network structure and set appropriate parameters for learning and training the physiological data. Compared to traditional machine learning algorithms that require manual feature extraction and screening, neural networks can automatically select and extract features from input physiological data, reducing labor costs and improving feature extraction efficiency.

[0203] In another example, when fusing multiple initial feature data, the fusion can be performed based on the weights of the neural network. For example, for multiple neural networks corresponding one-to-one to multiple physiological time series data, multiple weights corresponding one-to-one to the multiple neural networks are determined, where the multiple weights form a weight vector. Then, the multiple initial feature data corresponding one-to-one to the multiple physiological time series data are concatenated to obtain a concatenated vector. Finally, the concatenated vector and the weight vector are multiplied to obtain the fused feature data.

[0204] It can be understood that, taking 16 channels of physiological time series data as an example, the 16 channels of physiological time series data correspond to 16 neural networks, and each of the 16 neural networks corresponds to a weight. The weights of different neural networks may be different. For example, the physiological time series data collected by some channels have richer information. When predicting the physiological state, the weight corresponding to the physiological time series data collected by this channel is greater, thereby improving the prediction accuracy. For example, when collecting EEG data, a local area of ​​the brain is more active. The channel corresponding to this area can collect more and richer EEG data through electrodes, so the weight of the neural network corresponding to this channel should be larger.

[0205] FIG10 is a schematic diagram of a physiological state recognition model provided in an embodiment of the present application.

[0206] As shown in Figure 10, the physiological state recognition model includes, for example, an input layer, a hidden layer, a neural network layer, a fusion layer, a fully connected layer, and an output layer. The physiological state recognition model includes a multivariate temporal decomposition memory network architecture.

[0207] The input layer is used to input multiple collected physiological time series data. Each physiological time series data is collected through a corresponding data collection channel. Physiological time series data, for example, includes EEG data. Physiological time series data is a type of multivariate time series data.

[0208] The hidden layer is used to perform feature decomposition on each physiological time series data in the multiple physiological time series data to obtain subsequence data for each physiological time series data.

[0209] The neural network layer is used to extract features from the subsequence data corresponding to each physiological time series data to obtain initial feature data. For example, when physiological time series data is collected through n = 16 channels, the physiological time series data includes n = 16 pieces of data, each of which corresponds to a set of subsequence data (a set of subsequence data, for example, includes trend subsequence data and periodic subsequence data), thereby obtaining n = 16 sets of subsequence data. The neural network layer includes 16 neural networks in one-to-one correspondence, such as BiLSTM_1 to BiLSTM_n. Each neural network is used to extract features from the corresponding subsequence data, obtaining initial feature data 1 to initial data feature n that correspond to the 16 neural networks in one-to-one correspondence.

[0210] The fusion layer is used to fuse multiple initial feature data corresponding one-to-one to multiple physiological time series data to obtain fused feature data. When fusing multiple initial feature data, the fusion can be performed based on the weights of the neural network. For example, for multiple neural networks corresponding one-to-one to multiple physiological time series data, multiple weights corresponding one-to-one to the multiple neural networks are determined, where the multiple weights form a weight vector. The multiple initial feature data corresponding one-to-one to the multiple physiological time series data are then spliced ​​to obtain a splicing vector. Finally, the splicing vector and the weight vector are multiplied to obtain the fused feature data.

[0211] The fully connected layer predicts the physiological state based on the fused feature data and obtains a prediction result. Alternatively, the physiological state recognition model can be used to classify the physiological state. The fully connected layer obtains the classification result based on the fused feature data. The classification categories include, for example, fatigue state (fatigue, non-fatigue, fatigue level), emotional state (happy, sad), and load state (high load, low load, load level).

[0212] The output layer is used to output the prediction results or classification results of the physiological state.

[0213] Illustratively, an embodiment of the present application provides a method for training a physiological state recognition model, for example, including steps 1 to 5. The physiological state recognition model includes a hidden layer, a neural network layer, a fusion layer, and a fully connected layer.

[0214] Step 1: Using a hidden layer, perform feature decomposition on each physiological time series data in a plurality of physiological time series data to obtain subsequence data for each physiological time series data.

[0215] Exemplarily, each physiological time series data is acquired through a corresponding data acquisition channel. Multiple physiological time series data are, for example, sample data used to train a model. For example, with 16 data acquisition channels, the 16 physiological time series data acquired by the 16 data acquisition channels are considered as a sample data set. The sample data is, for example, labeled with a sample data set, and the sample label data represents, for example, the physiological state of the sample data.

[0216] Step 2: Using the neural network corresponding to each physiological time series data, feature extraction is performed on the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0217] Step three: using the fusion layer, fuse the multiple initial feature data corresponding to the multiple physiological time series data to obtain fused feature data.

[0218] Step 4: Use the fully connected layer to process the physiological state based on the fused feature data to obtain the prediction result.

[0219] Step 5: Based on the error between the prediction result and the sample label data, the model parameters of the physiological state recognition model are adjusted to train the physiological state recognition model.

[0220] For example, based on the error between the prediction result and the sample label data, the model parameters of the physiological state recognition model are reversely adjusted. The model parameters of the physiological state recognition model include, for example, parameters of the hidden layer, neural network layer, fusion layer, and fully connected layer.

[0221] In one example, the model parameters of the physiological state recognition model include weights of a neural network corresponding to each physiological time series sample data.

[0222] Exemplarily, a neural network corresponding to each physiological time series sample data is used to perform feature extraction on the subsequence data corresponding to each physiological time series sample data to obtain initial feature data, including: for multiple neural networks corresponding one-to-one to multiple physiological time series data, determining multiple weights corresponding one-to-one to the multiple neural networks, wherein the multiple weights constitute a weight vector; splicing the multiple initial feature data corresponding one-to-one to the multiple physiological time series sample data to obtain a splicing vector; multiplying the splicing vector and the weight vector to obtain fused feature data.

[0223] Before model training, weights can be initially set. During multiple rounds of training, model parameters, including weights, are adjusted inversely based on the error between the predicted results and the sample label data. After model training is complete, weights are determined. When the trained physiological state recognition model is subsequently used to predict physiological states, the determined weights are used to predict the physiological state.

[0224] Exemplarily, an embodiment of the present application provides a physiological state recognition device, which includes, for example: an acquisition module, a first decomposition module, a first extraction module, a first fusion module and a first prediction module.

[0225] Exemplarily, the acquisition module is used to acquire a plurality of physiological time series data, wherein each physiological time series data is acquired through a corresponding data acquisition channel.

[0226] Exemplarily, the first decomposition module is used to perform feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data.

[0227] Exemplarily, the first extraction module is used to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0228] Exemplarily, the first fusion module is used to fuse a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data to obtain fused feature data.

[0229] Exemplarily, the first prediction module is used to predict the physiological state based on the fused feature data.

[0230] Exemplarily, the first decomposition module is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; at least one of the trend subsequence data and the periodic subsequence data is used as subsequence data for each physiological time series data.

[0231] Exemplarily, the trend subsequence data and the periodic subsequence data are added together to obtain the physiological time series data, or the trend subsequence data and the periodic subsequence data are multiplied together to obtain the physiological time series data.

[0232] Exemplarily, the first decomposition module is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; remove the error data, and use the trend subsequence data and periodic subsequence data as subsequence data for each physiological time series data.

[0233] Exemplarily, the first extraction module is further configured to input the subsequence data corresponding to each physiological time series data into the neural network corresponding to each physiological time series data to perform feature extraction and obtain initial feature data.

[0234] Exemplarily, the first fusion module is also used to determine a plurality of weights corresponding one-to-one to a plurality of neural networks corresponding one-to-one to a plurality of physiological time series data, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain fused feature data.

[0235] It can be understood that the specific implementation process of the physiological state recognition device can refer to the implementation process of the physiological state recognition method above, and will not be repeated here.

[0236] Exemplarily, an embodiment of the present application provides a training device for a physiological state recognition model, comprising: a second decomposition module, a second extraction module, a second fusion module, a second prediction module and an adjustment module.

[0237] Exemplarily, the second decomposition module is used to perform feature decomposition on each physiological time series data in a plurality of physiological time series data using a hidden layer to obtain subsequence data for each physiological time series data; wherein each physiological time series data is collected through a corresponding data acquisition channel.

[0238] Exemplarily, the second extraction module is used to use the neural network corresponding to each physiological time series data to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0239] Exemplarily, the second fusion module is used to fuse a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data using a fusion layer to obtain fused feature data.

[0240] Exemplarily, the second prediction module is used to utilize a fully connected layer to perform physiological state processing based on fused feature data to obtain a prediction result.

[0241] Exemplarily, the adjustment module is used to adjust the model parameters of the physiological state recognition model based on the error between the prediction result and the sample label data, so as to train the physiological state recognition model.

[0242] Exemplarily, the model parameters of the physiological state recognition model include the weights of the neural network corresponding to each physiological time series sample data.

[0243] Exemplarily, the second fusion module is also used to determine a plurality of weights corresponding one-to-one to a plurality of neural networks corresponding one-to-one to a plurality of physiological time series data, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series sample data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain fused feature data.

[0244] Exemplarily, the second decomposition module is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; at least one of the trend subsequence data and the periodic subsequence data is used as subsequence data for each physiological time series data.

[0245] Exemplarily, the trend subsequence data and the periodic subsequence data are added together to obtain the physiological time series data, or the trend subsequence data and the periodic subsequence data are multiplied together to obtain the physiological time series data.

[0246] Exemplarily, the second decomposition module is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; remove the error data, and use the trend subsequence data and periodic subsequence data as subsequence data for each physiological time series data.

[0247] Exemplarily, the second extraction module is further configured to input the subsequence data corresponding to each physiological time series data into the neural network corresponding to each physiological time series data to perform feature extraction and obtain initial feature data.

[0248] It can be understood that the specific implementation process of the training device for the physiological state recognition model can refer to the implementation process of the training method for the physiological state recognition model mentioned above, and will not be repeated here.

[0249] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0250] One embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method of any of the above embodiments.

[0251] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0252] Thus far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present application is clearly not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present application.

Claims

1. A training method for a human factor intelligent physiological state recognition model, characterized in that: include: Acquire a first physiological signal, a first state label of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state label of the second physiological signal, wherein the second physiological signal is a physiological signal generated according to the first physiological signal; the first physiological signal is a physiological signal obtained by collection; the first state label is used to indicate the physiological state corresponding to the first physiological signal; and the second state label is used to indicate the physiological state corresponding to the second physiological signal; A physiological state recognition model is trained according to the first physiological signal, the second physiological signal, the first state label, and the second state label.

2. The method according to claim 1, characterized in that The step of training a physiological state recognition model according to the first physiological signal, the second physiological signal, the first state label, and the second state label includes: Generate a training sample of the physiological state recognition model according to the first physiological signal and the first state label of the first physiological signal; generate another training sample of the physiological state recognition model according to the second physiological signal and the second state label of the second physiological signal; The physiological state recognition model is trained using the training samples.

3. The method according to claim 2, characterized in that Acquiring a second physiological signal corresponding to the first physiological signal includes: generating a feature vector of a second physiological signal according to the first physiological signal and a first state label of the first physiological signal; The feature vector is input into a preset generator to obtain a second physiological signal output by the generator, and the generator is used to generate a physiological signal according to the feature vector.

4. The method according to claim 3, characterized in that The step of generating a feature vector of a second physiological signal according to the first physiological signal and a first state label of the first physiological signal includes: extracting a preset feature of the first physiological signal; generating a feature vector of the first physiological signal according to a preset feature of the first physiological signal and a first state label; A feature vector of the second physiological signal is generated according to the feature vector of the first physiological signal.

5. The method according to claim 4, characterized in that The physiological signal is an electrocardiogram signal, and the preset feature is an HRV feature.

6. The method according to any one of claims 3 to 5, characterized in that: The training method of the generator includes: Acquire a fourth physiological signal and a fourth state label of the fourth physiological signal; the fourth physiological signal is a collected physiological signal; generating a feature vector of a fifth physiological signal according to the fourth physiological signal and a fourth state label of the fourth physiological signal; Inputting the feature vector of the fifth physiological signal into a preset generator to obtain a fifth physiological signal output by the generator; Training a preset discriminator according to the fourth physiological signal and the fifth physiological signal; Acquire a sixth physiological signal and a sixth state label of the sixth physiological signal; the sixth physiological signal is a collected physiological signal; According to the sixth physiological signal and the sixth state label of the sixth physiological signal, adversarial training is performed on the generator and the discriminator to obtain a trained generator.

7. The method according to claim 6, characterized in that The preset discriminator is trained according to the fourth physiological signal and the fifth physiological signal; Setting a first source tag for the fourth physiological signal, wherein the first source tag is used to indicate that the fourth physiological signal is a collected physiological signal; setting a second source tag for the fifth physiological signal, wherein the second source tag is used to indicate that the fifth physiological signal is a generated physiological signal; A discriminator is trained according to the fourth physiological signal, the first source label, the fifth physiological signal, and the second source label.

8. The method according to claim 6, characterized in that The method of performing adversarial training on the generator and the discriminator according to the sixth physiological signal and the sixth state label of the sixth physiological signal to obtain a trained generator includes: generating a feature vector of a seventh physiological signal according to the sixth physiological signal and a sixth state label of the sixth physiological signal; Inputting the feature vector of the seventh physiological signal into the generator to obtain the seventh physiological signal output by the generator; A first source tag is set for the sixth physiological signal, and a second source tag is set for the seventh physiological signal; the first source tag is used to indicate that the sixth physiological signal is a collected physiological signal, and the second source tag is used to indicate that the seventh physiological signal is a generated physiological signal; generating a training sample for the discriminator according to the sixth physiological signal and the first source label, and generating another training sample for the discriminator according to the seventh physiological signal and the second source label; Inputting the training samples of the discriminator into the discriminator; The parameters of the generator and the discriminator are adjusted according to the discrimination result output by the discriminator.

9. The method according to any one of claims 1 to 5, characterized in that: The physiological state includes: mental state, emotional state or health state.

10. The method according to any one of claims 1 to 5, characterized in that: The physiological state recognition model is implemented by combining a Bayesian neural network with a translation encoder.

11. A physiological state recognition method, characterized in that: include: Obtaining the user's physiological signals; Inputting the physiological signal into a physiological state recognition model, wherein the physiological state recognition model is used to recognize the physiological state according to the physiological signal; The physiological state recognition model is obtained by training the method according to any one of claims 1 to 10; A physiological state recognition result output by the physiological state recognition model is obtained as the physiological state recognition result of the user.

12. The method according to claim 11, characterized in that The physiological signal includes a plurality of physiological time series data, and the identifying the physiological state according to the physiological signal includes: Acquire multiple physiological time series data, wherein each physiological time series data is acquired through a corresponding data acquisition channel; Performing feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data; Performing feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; Fusing a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain fused feature data; and Based on the fused feature data, the physiological state is predicted.

13. The method according to claim 12, characterized in that The feature decomposition of each physiological time series data to obtain subsequence data for each physiological time series data includes: Performing feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; At least one of the trend sub-sequence data and the periodic sub-sequence data is used as sub-sequence data for each physiological time series data.

14. The method according to claim 13, characterized in that The trend sub-series data and the periodic sub-series data are added to obtain the physiological time series data, or the trend sub-series data and the periodic sub-series data are multiplied to obtain the physiological time series data.

15. The method according to any one of claims 12 to 14, characterized in that: The step of extracting features from the subsequence data corresponding to each physiological time series data to obtain initial feature data includes: The subsequence data corresponding to each physiological time series data is input into the neural network corresponding to each physiological time series data to perform feature extraction to obtain the initial feature data.

16. The method according to claim 15, characterized in that The step of fusing the multiple initial feature data corresponding to the multiple physiological time series data to obtain fused feature data includes: For a plurality of neural networks corresponding one-to-one to the plurality of physiological time series data, determining a plurality of weights corresponding one-to-one to the plurality of neural networks, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain a splicing vector; and The concatenation vector and the weight vector are multiplied to obtain the fused feature data.

17. The method according to claim 11, characterized in that The physiological state recognition model comprises: The hidden layer is used to perform feature decomposition on each physiological time series data in the multiple physiological time series data to obtain subsequence data for each physiological time series data; wherein each physiological time series data is collected through a corresponding data collection channel; A neural network layer is used to extract features from the subsequence data corresponding to each physiological time series data to obtain initial feature data; A fusion layer, used for fusing a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain fused feature data; and The fully connected layer predicts the physiological state based on the fused feature data.

18. A training device for a physiological state recognition model, characterized in that: include: an acquisition module, used to acquire a first physiological signal, a first state tag of the first physiological signal, a second physiological signal corresponding to the first physiological signal, and a second state tag of the second physiological signal, wherein the second physiological signal is a physiological signal generated according to the first physiological signal; the first physiological signal is a physiological signal obtained by collection; and the first state tag is used to indicate a physiological state corresponding to the first physiological signal; The second state label is used to indicate the physiological state corresponding to the second physiological signal; A training module is used to train the physiological state recognition model according to the first physiological signal, the second physiological signal, the first state label and the second state label.

19. A physiological state recognition device, characterized in that: include: An acquisition module, used for acquiring physiological signals of a target object; An identification module, used for inputting the physiological signal into a preset physiological state identification model, wherein the physiological state identification model is used for identifying the physiological state according to the physiological signal; The preset physiological state recognition model is obtained by training the method according to any one of claims 1 to 9; A physiological state recognition result output by the physiological state recognition model is obtained as the physiological state of the target object.

20. An electronic device, characterized in that: include: Processor, memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the processor, enable the electronic device to perform the method according to any one of claims 1 to 17.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 17.

Images