Physiological and psychological state processing method, physiological and psychological state processing model and device

Abstract

The present invention discloses a physiological and psychological state processing method, a physiological and psychological state processing model and a device. The physiological and psychological state processing method comprises: obtaining a plurality of physiological time series data, wherein each physiological time series data is collected through a corresponding data collection channel; performing feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data; performing feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; fusing a plurality of initial feature data corresponding to the plurality of physiological time series data one by one to obtain fused feature data; and predicting the physiological and psychological state based on the fused feature data. The present invention improves the accuracy of physiological and psychological state processing.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence learning, and in particular to a physiological and psychological state processing method, a physiological and psychological state processing model training method and a device. Background Art

[0002] Physiological data is data generated by the human body during activities. Physiological data can represent a person's physiological state to a certain extent. Physiological data includes, for example, EEG data, galvanic skin data, ECG data, etc. Physiological states include, for example, fatigue state, emotional state, load state, etc. Related technologies can predict a person's physiological state through physiological data, but the accuracy of physiological and psychological state processing in related technologies is not high, and the prediction effect is not good. Summary of the invention

[0003] The embodiments of the present application aim to solve at least one of the technical problems in the related art to a certain extent. To this end, the purpose of the embodiments of the present application is to propose a physiological and psychological state processing method, a training method for a physiological and psychological state processing model, a physiological and psychological state processing model, a device, an electronic device, a storage medium and a program product.

[0004] An embodiment of the present application provides a method for processing physiological and psychological states, the method comprising: acquiring multiple physiological time series data, wherein each physiological time series data is acquired through a corresponding data acquisition channel; performing feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data; performing feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; fusing multiple initial feature data corresponding one-to-one to the multiple physiological time series data to obtain fused feature data; and predicting the physiological and psychological state based on the fused feature data.

[0005] Exemplarily, the feature decomposition of each physiological time series data to obtain subsequence data for each physiological time series data includes: feature decomposition of each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; at least one of the trend subsequence data and the periodic subsequence data is used as the subsequence data for each physiological time series data.

[0006] Exemplarily, the trend sub-sequence data and the periodic sub-sequence data are added to obtain the physiological time series data, or the trend sub-sequence data and the periodic sub-sequence data are multiplied to obtain the physiological time series data.

[0007] Exemplarily, the feature decomposition of each physiological time series data to obtain subsequence data for each physiological time series data includes: feature decomposition of each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; removing the error data, and using the trend subsequence data and the periodic subsequence data as subsequence data for each physiological time series data.

[0008] Exemplarily, the extracting features from the subsequence data corresponding to each physiological time series data to obtain initial feature data includes: inputting the subsequence data corresponding to each physiological time series data into a neural network corresponding to each physiological time series data to perform feature extraction to obtain the initial feature data.

[0009] Exemplarily, the fusing of multiple initial feature data corresponding one-to-one to the multiple physiological time series data to obtain fused feature data includes: determining multiple weights corresponding one-to-one to the multiple neural networks corresponding one-to-one to the multiple physiological time series data, wherein the multiple weights constitute a weight vector; splicing the multiple initial feature data corresponding one-to-one to the multiple physiological time series data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain the fused feature data.

[0010] Another embodiment of the present application provides a training method for a physiological and psychological state processing model, wherein the physiological and psychological state processing model includes a hidden layer, a neural network layer, a fusion layer and a fully connected layer, and the training method includes: using the hidden layer to perform feature decomposition on each physiological time series data in a plurality of physiological time series data to obtain subsequence data for each physiological time series data; wherein each physiological time series data is acquired through a corresponding data acquisition channel; using a neural network corresponding to each physiological time series data, performing feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; using the fusion layer to fuse a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain fused feature data; using the fully connected layer to perform physiological and psychological state processing based on the fused feature data to obtain a prediction result; based on the error between the prediction result and the sample label data, adjusting the model parameters of the physiological and psychological state processing model to train the physiological and psychological state processing model.

[0011] Exemplarily, the model parameters of the physiological and psychological state processing model include the weights of the neural network corresponding to each physiological time series sample data.

[0012] Exemplarily, the method utilizes a neural network corresponding to each physiological time series sample data to perform feature extraction on the subsequence data corresponding to each physiological time series sample data to obtain initial feature data, including: for a plurality of neural networks corresponding one-to-one to the plurality of physiological time series data, determining a plurality of weights corresponding one-to-one to the plurality of neural networks, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series sample data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain the fused feature data.

[0013] Another embodiment of the present application provides a physiological and psychological state processing model, which includes: a hidden layer, which is used to perform feature decomposition on each physiological time series data in a plurality of physiological time series data to obtain subsequence data for each physiological time series data; wherein each physiological time series data is acquired through a corresponding data acquisition channel; a neural network layer, which is used to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; a fusion layer, which is used to fuse a plurality of initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain fused feature data; and a fully connected layer, which predicts the physiological and psychological state based on the fused feature data.

[0014] Another embodiment of the present application provides a physiological and psychological state processing device, which includes: an acquisition module, used to acquire multiple physiological time series data, wherein each physiological time series data is acquired through a corresponding data acquisition channel; a first decomposition module, used to perform feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data; a first extraction module, used to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; a first fusion module, used to fuse multiple initial feature data corresponding one-to-one to the multiple physiological time series data to obtain fused feature data; a first prediction module, used to predict the physiological and psychological state based on the fused feature data.

[0015] Another embodiment of the present application provides a training device for a physiological and psychological state processing model, wherein the physiological and psychological state processing model includes a hidden layer, a neural network layer, a fusion layer and a fully connected layer, and the training device includes: a second decomposition module, which is used to use the hidden layer to perform feature decomposition on each physiological time series data in a plurality of physiological time series data to obtain subsequence data for each physiological time series data; wherein each physiological time series data is acquired through a corresponding data acquisition channel; a second extraction module, which is used to use a neural network corresponding to each physiological time series data to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data; a second fusion module, which is used to use the fusion layer to fuse a plurality of initial feature data corresponding one by one to the plurality of physiological time series data to obtain fused feature data; a second prediction module, which is used to use the fully connected layer to perform physiological and psychological state processing based on the fused feature data to obtain a prediction result; and an adjustment module, which is used to adjust the model parameters of the physiological and psychological state processing model based on the error between the prediction result and the sample label data to train the physiological and psychological state processing model.

[0016] Another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the above embodiments are implemented.

[0017] Another embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0018] Another embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can perform the steps of the method described in any of the above embodiments.

[0019] In the above implementation, multiple physiological time series data are obtained; each physiological time series data is feature decomposed to obtain subsequence data for each physiological time series data; the subsequence data corresponding to each physiological time series data is feature extracted to obtain initial feature data; multiple initial feature data corresponding to the multiple physiological time series data are fused to obtain fused feature data; based on the fused feature data, the physiological and psychological state is predicted. The present invention improves the accuracy of physiological and psychological state processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a process for processing physiological and psychological states provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of data decomposition provided for an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a physiological and psychological state processing model provided for an embodiment of the present application;

[0023] Figure 4 A flowchart of a training method for a physiological and psychological state processing model provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of a physiological and psychological state processing device provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of a training device for a physiological and psychological state processing model provided in an embodiment of the present application;

[0026] Figure 7 A block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0028] Physiological data is data generated by the human body during activities. Physiological data can represent a person's physiological state to a certain extent. Physiological data includes, for example, EEG data, galvanic skin data, ECG data, etc. Physiological states include, for example, fatigue state, emotional state, load state, etc. Related technologies can predict a person's physiological state through physiological data, but the accuracy of physiological and psychological state processing in related technologies is not high, and the prediction effect is not good.

[0029] In view of this, the embodiments of the present application provide an optimized physiological and psychological state processing method based on human factor intelligence, a training method of a physiological and psychological state processing model, and a physiological and psychological state processing model. The present application proposes a technology that combines human factor engineering with artificial intelligence, thereby realizing the application of artificial intelligence in human factor engineering.

[0030] Figure 1 A flowchart of a method for processing physiological and psychological states provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the physiological and psychological state processing method 100 provided in the embodiment of the present application includes, for example, steps S110-S150.

[0032] Step S110, acquiring a plurality of physiological time series data.

[0033] Exemplarily, physiological data include, for example, EEG data, skin electricity data, ECG data, and the like. Taking EEG data as an example, EEG data can be collected through EEG electrodes. Since EEG data is usually a kind of fluctuating data that changes with time, EEG data can be used as a kind of time series data (which can be called timing data). Physiological timing data is obtained by collecting physiological data. Each physiological timing data is collected through a corresponding data acquisition channel. For ease of understanding, the data acquisition channels of this application are taken as 16 as an example. Each data acquisition channel corresponds to an electrode. The 16 electrodes are respectively attached to different positions of the human head for data collection. The EEG data are collected through the 16 acquisition channels to obtain 16 physiological timing data (EEG data).

[0034] Step S120, performing feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data.

[0035] Exemplarily, each physiological time series data contains implicit features that conform to a certain variation law, so each physiological time series data needs to be feature decomposed. Each physiological time series data can be decomposed into multiple subsequence data, and each subsequence data, for example, represents a variation law.

[0036] Step S130, extracting features from the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0037] For each of the multiple subsequence data corresponding to the physiological time series data, feature extraction is performed on the multiple subsequence data to obtain initial feature data. The multiple physiological time series data correspond one-to-one to the multiple initial feature data.

[0038] Step S140, fusing a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data to obtain fused feature data.

[0039] The physiological time series data collected by each data acquisition channel is different. For example, the physiological time series data collected by some channels has more information or can better reflect the physiological and psychological state of the user. The initial feature data corresponding to multiple channels are fused so that the fused feature data has more information.

[0040] Step S150: predicting the physiological and psychological state based on the fused feature data.

[0041] Exemplarily, the physiological and psychological state includes at least one of a physiological state and a psychological state. The physiological state of the user is predicted based on the fused feature data. The physiological state includes, for example, a fatigue state, an emotional state, a load state, etc., and the psychological state includes anxiety, relaxation, and the like.

[0042] In the embodiments of the present application, the physiological time series data is decomposed to obtain sub-sequence data representing the changing rules of the physiological time series data, and then the features of the sub-sequence data are further extracted to obtain richer initial feature data. Taking into account the different data collected by different channels, the initial feature data of different channels are fused to predict the physiological and psychological state, thereby improving the prediction accuracy of the physiological and psychological state.

[0043] In another example, feature decomposition may be performed on each physiological time series data to obtain subsequence data for each physiological time series data.

[0044] For example, each physiological time series data can be feature decomposed to obtain trend subsequence data and periodic subsequence data. Trend subsequence data represents the trend of physiological time series data changing over time, and periodic subsequence data represents the periodic change characteristics of physiological time series data. Periodic subsequence data can also be called seasonal subsequence data. For example, if the physiological time series data is electrocardiogram data, the electrocardiogram data represents the user's heartbeat to a certain extent, and the periodic subsequence data can reflect the heartbeat cycle.

[0045] Then, at least one of the trend subsequence data and the periodic subsequence data is used as the subsequence data for each physiological time series data. For example, the trend subsequence data and the periodic subsequence data can be used as the subsequence data.

[0046] When performing feature decomposition, an additive decomposition method or a multiplicative decomposition method can be used. The trend subsequence data and the periodic subsequence data obtained by the additive decomposition method are added to obtain physiological time series data. The trend subsequence data and the periodic subsequence data obtained by the multiplicative decomposition method are multiplied to obtain physiological time series data.

[0047] Figure 2 A schematic diagram of data decomposition provided for an embodiment of the present application.

[0048] like Figure 2 As shown in FIG. 1 , taking the 16-channel EEG data as an example, each of the 16 physiological time series data is subjected to feature decomposition. Figure 2 It is shown that the EEG data of the third channel is subjected to feature decomposition to obtain trend subsequence data, periodic subsequence data (seasonal subsequence data) and error data, where the error data is, for example, white noise. The error data can be removed, and the trend subsequence data and the periodic subsequence data can be retained as subsequence data for each physiological time series data.

[0049] When performing feature decomposition, an additive decomposition method or a multiplicative decomposition method can be used. The additive decomposition method is used to obtain trend subsequence data, periodic subsequence data and error data, and the trend subsequence data, periodic subsequence data and error data are added to obtain physiological time series data. The multiplicative decomposition method is used to obtain trend subsequence data, periodic subsequence data and error data, and the trend subsequence data, periodic subsequence data and error data are multiplied to obtain physiological time series data.

[0050] It can be understood that the present application can select the additive decomposition method or the multiplicative decomposition method according to the actual situation requirements. By decomposing the physiological time series data, subsequence data representing the changing law of the physiological time series data can be obtained, and then the subsequence data is further subjected to feature extraction to obtain richer initial feature data. Compared with the method of directly extracting features from the physiological time series data, the present application extracts features from the decomposed subsequence data, and can extract more important features at a deeper level. It can be seen that by extracting features after data decomposition, the effect of feature extraction is improved, thereby improving the accuracy of physiological and psychological state processing.

[0051] In another example, a neural network may be used to extract features of subsequence data. For example, the subsequence data corresponding to each physiological time series data is input into a neural network corresponding to each physiological time series data to extract features and obtain initial feature data.

[0052] Neural networks include, for example, long short-term memory networks (LSTM), bidirectional long short-term memory networks BiLSTM, etc., and may also be other networks that can extract features from time series data.

[0053] As a deep learning method, neural networks have powerful signal processing and recognition capabilities. Neural networks can be applied to the analysis of physiological signals. When using neural networks to detect physiological signals, you first need to select a suitable network structure and set corresponding parameters to learn and train physiological data. Compared with traditional machine learning algorithms that require manual feature extraction and screening, neural networks can automatically select and extract features from input physiological data, reducing labor costs and improving feature extraction effects.

[0054] In another example, when fusing multiple initial feature data, the fusion can be performed based on the weight of the neural network. For example, for multiple neural networks corresponding to multiple physiological time series data, multiple weights corresponding to the multiple neural networks are determined, wherein the multiple weights form a weight vector. Then, the multiple initial feature data corresponding to the multiple physiological time series data are spliced ​​to obtain a splicing vector, and finally the splicing vector and the weight vector are multiplied to obtain the fused feature data.

[0055] It can be understood that, taking 16 channels of physiological time series data as an example, the 16 channels of physiological time series data correspond to 16 neural networks, and each of the 16 neural networks corresponds to a weight. The weights of different neural networks may be different. For example, the physiological time series data collected by some channels have richer information. When predicting physiological and psychological states, the weight corresponding to the physiological time series data collected by this channel is larger, thereby improving the prediction accuracy. For example, when collecting EEG data, a local area of ​​the brain is more active, and the channel corresponding to this area can collect more and richer EEG data through electrodes, so the weight of the neural network corresponding to this channel should be larger.

[0056] Figure 3 A schematic diagram of a physiological and psychological state processing model provided for an embodiment of the present application.

[0057] like Figure 3 As shown, the physiological and psychological state processing model includes, for example, an input layer, a hidden layer, a neural network layer, a fusion layer, a fully connected layer, and an output layer. The physiological and psychological state processing model includes a multivariate temporal decomposition memory network architecture.

[0058] The input layer is used to input a plurality of collected physiological time series data, each of which is collected through a corresponding data collection channel. The physiological time series data includes, for example, electroencephalogram data, which is a multivariate time series data.

[0059] The hidden layer is used to perform feature decomposition on each physiological time series data in the multiple physiological time series data to obtain subsequence data for each physiological time series data.

[0060] The neural network layer is used to extract features from the subsequence data corresponding to each physiological time series data to obtain initial feature data. For example, when the physiological time series data is collected through n=16 channels, the physiological time series data includes n=16 pieces, and each physiological time series data corresponds to a set of subsequence data (a set of subsequence data, for example, includes trend subsequence data and periodic subsequence data), thereby obtaining n=16 sets of subsequence data. The neural network layer includes 16 neural networks in one-to-one correspondence, for example, including BiLSTM_1~BiLSTM_n, each neural network is used to extract features from the corresponding subsequence data, and obtain initial feature data 1~initial data feature n corresponding to the 16 neural networks.

[0061] The fusion layer is used to fuse multiple initial feature data corresponding to multiple physiological time series data to obtain fused feature data. When fusing multiple initial feature data, the fusion can be based on the weight of the neural network. For example, for multiple neural networks corresponding to multiple physiological time series data, multiple weights corresponding to the multiple neural networks are determined, wherein the multiple weights constitute a weight vector. Then, the multiple initial feature data corresponding to the multiple physiological time series data are spliced ​​to obtain a splicing vector, and finally the splicing vector and the weight vector are multiplied to obtain the fused feature data.

[0062] The fully connected layer predicts the physiological and psychological state based on the fused feature data to obtain the prediction result. Alternatively, the physiological and psychological state processing model can be used to classify the physiological and psychological state. The fully connected layer obtains the classification result based on the fused feature data. The classification categories include, for example, the fatigue state of the person (fatigue, not fatigued, fatigue level), emotional state (happy, sad), and load state (high load, low load, load level).

[0063] The output layer is used to output the prediction or classification results of physiological and psychological states.

[0064] Figure 4 A flowchart of a training method for a physiological and psychological state processing model provided in an embodiment of the present application.

[0065] like Figure 4 As shown, the training method 400 of the physiological and psychological state processing model provided in the embodiment of the present application includes steps S410 to S450. The physiological and psychological state processing model includes a hidden layer, a neural network layer, a fusion layer and a fully connected layer.

[0066] Step S410: using the hidden layer, perform feature decomposition on each physiological time series data in the plurality of physiological time series data to obtain subsequence data for each physiological time series data.

[0067] Exemplarily, each physiological time series data is collected through a corresponding data acquisition channel. Multiple physiological time series data are, for example, sample data for training a model. Taking 16 data acquisition channels as an example, 16 physiological time series data collected by the 16 data acquisition channels are used as a sample data. The sample data, for example, corresponds to a sample label data tag, and the sample label data, for example, represents the physiological and psychological state of the sample data.

[0068] Step S420, using the neural network corresponding to each physiological time series data, extracting features from the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0069] Step S430, using a fusion layer, a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data are fused to obtain fused feature data.

[0070] Step S440, using the fully connected layer, the physiological and psychological state is processed based on the fused feature data to obtain a prediction result.

[0071] Step S450, based on the error between the prediction result and the sample label data, the model parameters of the physiological and psychological state processing model are adjusted to train the physiological and psychological state processing model.

[0072] For example, based on the error between the prediction result and the sample label data, the model parameters of the physiological and psychological state processing model are reversely adjusted. The model parameters of the physiological and psychological state processing model include, for example, parameters of the hidden layer, the neural network layer, the fusion layer, and the fully connected layer.

[0073] In one example, the model parameters of the physiological and psychological state processing model include the weights of the neural network corresponding to each physiological time series sample data.

[0074] Exemplarily, a neural network corresponding to each physiological time series sample data is used to perform feature extraction on subsequence data corresponding to each physiological time series sample data to obtain initial feature data, including: for multiple neural networks corresponding one-to-one to multiple physiological time series data, determining multiple weights corresponding one-to-one to the multiple neural networks, wherein the multiple weights constitute a weight vector; splicing multiple initial feature data corresponding one-to-one to multiple physiological time series sample data to obtain a splicing vector; multiplying the splicing vector and the weight vector to obtain fused feature data.

[0075] Before model training, the weights can be initially set, and in multiple rounds of training, the model parameters are reversely adjusted based on the error between the prediction results and the sample label data, including the weights. After the model training is completed, the weights are determined, and when the trained physiological and psychological state processing model is used to predict the physiological and psychological state, the physiological and psychological state is predicted based on the determined weights.

[0076] Figure 5 A schematic diagram of a physiological and psychological state processing device provided in an embodiment of the present application.

[0077] like Figure 5 As shown, the physiological and psychological state processing device 500 provided in the embodiment of the present application includes: an acquisition module 510, a first decomposition module 520, a first extraction module 530, a first fusion module 540 and a first prediction module 550.

[0078] Exemplarily, the acquisition module 510 is used to acquire a plurality of physiological time series data, wherein each physiological time series data is acquired through a corresponding data acquisition channel.

[0079] Exemplarily, the first decomposition module 520 is used to perform feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data.

[0080] Exemplarily, the first extraction module 530 is used to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0081] Exemplarily, the first fusion module 540 is used to fuse a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data to obtain fused feature data.

[0082] Exemplarily, the first prediction module 550 is used to predict the physiological and psychological state based on the fused feature data.

[0083] Exemplarily, the first decomposition module 520 is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; at least one of the trend subsequence data and the periodic subsequence data is used as subsequence data for each physiological time series data.

[0084] Exemplarily, the trend subsequence data and the periodic subsequence data are added to obtain the physiological time series data, or the trend subsequence data and the periodic subsequence data are multiplied to obtain the physiological time series data.

[0085] Exemplarily, the first decomposition module 520 is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; remove the error data, and use the trend subsequence data and periodic subsequence data as subsequence data for each physiological time series data.

[0086] Exemplarily, the first extraction module 530 is further used to input the subsequence data corresponding to each physiological time series data into the neural network corresponding to each physiological time series data to perform feature extraction to obtain initial feature data.

[0087] Exemplarily, the first fusion module 540 is also used to determine a plurality of weights corresponding one-to-one to a plurality of neural networks corresponding one-to-one to a plurality of physiological time series data, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain fused feature data.

[0088] It can be understood that the specific implementation process of the physiological and psychological state processing device 500 can refer to the implementation process of the physiological and psychological state processing method above, and will not be repeated here.

[0089] Figure 6 A schematic diagram of a training device for a physiological and psychological state processing model provided in an embodiment of the present application.

[0090] like Figure 6 As shown, the training device 600 for the physiological and psychological state processing model provided in the embodiment of the present application includes: a second decomposition module 610, a second extraction module 620, a second fusion module 630, a second prediction module 640 and an adjustment module 650.

[0091] Exemplarily, the second decomposition module 610 is used to perform feature decomposition on each physiological time series data in a plurality of physiological time series data using a hidden layer to obtain subsequence data for each physiological time series data; wherein each physiological time series data is collected through a corresponding data collection channel.

[0092] Exemplarily, the second extraction module 620 is used to use the neural network corresponding to each physiological time series data to perform feature extraction on the subsequence data corresponding to each physiological time series data to obtain initial feature data.

[0093] Exemplarily, the second fusion module 630 is used to fuse a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series data using a fusion layer to obtain fused feature data.

[0094] Exemplarily, the second prediction module 640 is used to utilize a fully connected layer to perform physiological and psychological state processing based on fused feature data to obtain a prediction result.

[0095] Exemplarily, the adjustment module 650 is used to adjust the model parameters of the physiological and psychological state processing model based on the error between the prediction result and the sample label data, so as to train the physiological and psychological state processing model.

[0096] Exemplarily, the model parameters of the physiological and psychological state processing model include the weights of the neural network corresponding to each physiological time series sample data.

[0097] Exemplarily, the second fusion module 630 is also used to determine a plurality of weights corresponding one-to-one to a plurality of neural networks corresponding one-to-one to a plurality of physiological time series data, wherein the plurality of weights constitute a weight vector; splicing a plurality of initial feature data corresponding one-to-one to a plurality of physiological time series sample data to obtain a splicing vector; and multiplying the splicing vector and the weight vector to obtain fused feature data.

[0098] Exemplarily, the second decomposition module 610 is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; at least one of the trend subsequence data and the periodic subsequence data is used as subsequence data for each physiological time series data.

[0099] Exemplarily, the trend subsequence data and the periodic subsequence data are added to obtain the physiological time series data, or the trend subsequence data and the periodic subsequence data are multiplied to obtain the physiological time series data.

[0100] Exemplarily, the second decomposition module 610 is also used to perform feature decomposition on each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; remove the error data, and use the trend subsequence data and periodic subsequence data as subsequence data for each physiological time series data.

[0101] Exemplarily, the second extraction module 620 is further used to input the subsequence data corresponding to each physiological time series data into the neural network corresponding to each physiological time series data to perform feature extraction to obtain initial feature data.

[0102] It can be understood that the specific implementation process of the training device 600 for the physiological and psychological state processing model can refer to the implementation process of the training method for the physiological and psychological state processing model mentioned above, which will not be repeated here.

[0103] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0104] One embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can perform the steps of the method of any of the above embodiments.

[0105] Figure 7 A block diagram of an electronic device provided for an embodiment of the present application.

[0106] The embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned physiological and psychological state processing method and the training method of the physiological and psychological state processing model when executing the computer program. Of course, the physiological and psychological state processing method and the training method of the physiological and psychological state processing model can be executed by different electronic devices.

[0107] like Figure 7As shown, for ease of understanding, the embodiment of the present application shows a specific electronic device 700.

[0108] The electronic device 700 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0110] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0111] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods described above. For example, in some embodiments, any one or more of the various methods described above may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, any one or more of the various methods described above may be executed. One or more steps. Alternatively, in other embodiments, the computing unit 701 may be configured to perform any one or more of the various methods described above in any other appropriate manner (e.g., by means of firmware).

[0112] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this application, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0113] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0114] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0115] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0116] In addition, the terms "first", "second", etc. used in the embodiments of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present application can explicitly or implicitly indicate that at least one of the features is included in the embodiment. In the description of the present application, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0117] In this application, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected" and "fixed" etc. appearing in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can be understood that it can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements, or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific implementation situation.

[0118] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0119] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for processing physiological and psychological states, characterized in that: The method comprises: For the same physiological type, multiple areas of the user's body are collected through multiple data collection channels to obtain multiple physiological time series data, wherein the multiple data collection channels correspond to the multiple physiological time series data one by one, each physiological time series data is collected through the corresponding data collection channel, and different data collection channels are used to collect different areas; Before performing feature extraction based on the multiple physiological time series data, feature decomposition is performed on each physiological time series data to obtain subsequence data for each physiological time series data, including: feature decomposition is performed on each physiological time series data to obtain trend subsequence data and periodic subsequence data, and at least one of the trend subsequence data and the periodic subsequence data is used as the subsequence data for each physiological time series data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; Performing feature extraction on the subsequence data corresponding to each physiological time series data to obtain in-depth initial feature data; Based on the weights corresponding to the multiple data acquisition channels, weighted fusion is performed on the multiple deep-level initial feature data corresponding one-to-one to the multiple physiological time series data to obtain fused feature data, wherein the weight corresponding to each data acquisition channel is associated with the physiological activity level of the area corresponding to the data acquisition channel; and Based on the fused feature data, the physiological and psychological state is predicted.

2. The method according to claim 1, characterized in that The trend sub-series data and the periodic sub-series data are added to obtain the physiological time series data, or the trend sub-series data and the periodic sub-series data are multiplied to obtain the physiological time series data.

3. The method according to claim 1, characterized in that The feature decomposition of each physiological time series data to obtain subsequence data for each physiological time series data includes: Perform feature decomposition on each physiological time series data to obtain trend subsequence data, periodic subsequence data and error data; The error data is removed, and the trend sub-sequence data and the periodic sub-sequence data are used as sub-sequence data for each physiological time series data.

4. The method according to any one of claims 1 to 3, characterized in that: The feature extraction of the subsequence data corresponding to each physiological time series data to obtain deep initial feature data includes: The subsequence data corresponding to each physiological time series data is input into the neural network corresponding to each physiological time series data to perform feature extraction, so as to obtain the deep-level initial feature data.

5. The method according to claim 4, characterized in that The step of weighting and fusing a plurality of initial feature data at a deep level corresponding to the plurality of physiological time series data based on the weights corresponding to the plurality of data acquisition channels to obtain fused feature data includes: For a plurality of neural networks corresponding one-to-one to the plurality of physiological time series data, a plurality of weights corresponding one-to-one to the plurality of neural networks are determined to obtain a plurality of weights corresponding one-to-one to the plurality of data acquisition channels, wherein the plurality of weights constitute a weight vector; splicing a plurality of deep-level initial feature data corresponding one-to-one to the plurality of physiological time series data to obtain a splicing vector; and The concatenation vector and the weight vector are multiplied to obtain the fused feature data.

6. A training method for a physiological and psychological state processing model, characterized in that: The physiological and psychological state processing model includes a hidden layer, a neural network layer, a fusion layer and a fully connected layer, and the training method includes: Using the hidden layer, before extracting features based on multiple physiological time series data, feature decomposition is performed on each physiological time series data in the multiple physiological time series data to obtain subsequence data for each physiological time series data, including: feature decomposition is performed on each physiological time series data to obtain trend subsequence data and periodic subsequence data, and at least one of the trend subsequence data and the periodic subsequence data is used as the subsequence data for each physiological time series data; wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data, multiple physiological time series data are for the same physiological type, and multiple physiological time series data are obtained by collecting data from multiple areas of the user's body through multiple data acquisition channels, and the multiple data acquisition channels correspond one-to-one to the multiple physiological time series data, and each physiological time series data is collected through the corresponding data acquisition channel, and different data acquisition channels are used to collect data from different areas; Using a neural network corresponding to each physiological time series data, feature extraction is performed on the subsequence data corresponding to each physiological time series data to obtain deep initial feature data; Using the fusion layer, based on the weights corresponding to the multiple data acquisition channels, multiple deep-level initial feature data corresponding one to one to the multiple physiological time series data are weightedly fused to obtain fused feature data, wherein the weight corresponding to each data acquisition channel is associated with the physiological activity level of the area corresponding to the data acquisition channel; Using the fully connected layer, processing the physiological and psychological state based on the fused feature data to obtain a prediction result; and Based on the error between the prediction result and the sample label data, the model parameters of the physiological and psychological state processing model are adjusted to train the physiological and psychological state processing model.

7. The method according to claim 6, characterized in that The model parameters of the physiological and psychological state processing model include the weights of the neural network corresponding to each physiological time series sample data.

8. The method according to claim 7, characterized in that The step of weighting and fusing a plurality of initial feature data at a deep level corresponding to the plurality of physiological time series data based on the weights corresponding to the plurality of data acquisition channels to obtain fused feature data includes: For a plurality of neural networks corresponding one-to-one to the plurality of physiological time series data, a plurality of weights corresponding one-to-one to the plurality of neural networks are determined to obtain a plurality of weights corresponding one-to-one to the plurality of data acquisition channels, wherein the plurality of weights constitute a weight vector; splicing a plurality of deep-level initial feature data corresponding one-to-one to the plurality of physiological time series sample data to obtain a splicing vector; and The concatenation vector and the weight vector are multiplied to obtain the fused feature data.

9. A method for processing physiological and psychological states, characterized in that: The physiological and psychological state processing method is applied to a neural network physiological and psychological state processing model, and the physiological and psychological state processing model includes: A hidden layer is used for performing feature decomposition on each physiological time series data in the multiple physiological time series data before feature extraction based on the multiple physiological time series data to obtain subsequence data for each physiological time series data, including: performing feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, and using at least one of the trend subsequence data and the periodic subsequence data as the subsequence data for each physiological time series data; wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; the multiple physiological time series data are for the same physiological type, and the multiple physiological time series data are obtained by collecting data from multiple areas of the user's body through multiple data acquisition channels, the multiple data acquisition channels correspond to the multiple physiological time series data one by one, and each physiological time series data is collected through the corresponding data acquisition channel, and different data acquisition channels are used to collect data from different areas; A neural network layer is used to extract features from the subsequence data corresponding to each physiological time series data to obtain deep initial feature data; A fusion layer, for weightedly fusing a plurality of deep-level initial feature data corresponding one-to-one to the plurality of physiological time series data based on the weights corresponding to the plurality of data acquisition channels, to obtain fused feature data, wherein the weight corresponding to each data acquisition channel is associated with the physiological activity level of the region corresponding to the data acquisition channel; and The fully connected layer predicts the physiological and psychological state based on the fused feature data.

10. A physiological and psychological state processing device, characterized in that: The device comprises: An acquisition module, for collecting data from multiple areas of a user's body through multiple data acquisition channels for the same physiological type, so as to obtain multiple physiological time series data, wherein the multiple data acquisition channels correspond to the multiple physiological time series data one by one, each physiological time series data is collected through the corresponding data acquisition channel, and different data acquisition channels are used to collect data from different areas; A first decomposition module is used for performing feature decomposition on each physiological time series data to obtain subsequence data for each physiological time series data before performing feature extraction based on the multiple physiological time series data, including: performing feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, and using at least one of the trend subsequence data and the periodic subsequence data as the subsequence data for each physiological time series data, wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; A first extraction module is used to extract features from the subsequence data corresponding to each physiological time series data to obtain in-depth initial feature data; a first fusion module, configured to perform weighted fusion on a plurality of deep-level initial feature data corresponding one-to-one to the plurality of physiological time series data based on weights corresponding to the plurality of data acquisition channels, to obtain fused feature data, wherein the weight corresponding to each data acquisition channel is associated with a physiological activity level of a region corresponding to the data acquisition channel; and The first prediction module is used to predict the physiological and psychological state based on the fused feature data.

11. A training device for a physiological and psychological state processing model, characterized in that: The physiological and psychological state processing model includes a hidden layer, a neural network layer, a fusion layer and a fully connected layer, and the training device includes: A second decomposition module is used to utilize the hidden layer to perform feature decomposition on each physiological time series data in the multiple physiological time series data before performing feature extraction based on the multiple physiological time series data to obtain subsequence data for each physiological time series data, including: performing feature decomposition on each physiological time series data to obtain trend subsequence data and periodic subsequence data, and using at least one of the trend subsequence data and the periodic subsequence data as the subsequence data for each physiological time series data; wherein the trend subsequence data represents the trend of the physiological time series data changing over time, and the periodic subsequence data represents the periodic change characteristics of the physiological time series data; the multiple physiological time series data are for the same physiological type, and the multiple physiological time series data are obtained by performing data collection on multiple areas of the user's body through multiple data collection channels, the multiple data collection channels correspond one-to-one to the multiple physiological time series data, and each physiological time series data is collected through the corresponding data collection channel, and different data collection channels are used to collect different areas; A second extraction module is used to extract features of the subsequence data corresponding to each physiological time series data by using a neural network corresponding to each physiological time series data to obtain deep initial feature data; A second fusion module is used to use the fusion layer to perform weighted fusion on a plurality of deep-level initial feature data corresponding to the plurality of physiological time series data based on the weights corresponding to the plurality of data acquisition channels, so as to obtain fused feature data, wherein the weight corresponding to each data acquisition channel is associated with the physiological activity level of the area corresponding to the data acquisition channel; A second prediction module is used to use the fully connected layer to process the physiological and psychological state based on the fused feature data to obtain a prediction result; and An adjustment module is used to adjust the model parameters of the physiological and psychological state processing model based on the error between the prediction result and the sample label data, so as to train the physiological and psychological state processing model.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images