Sleep state recognition model training method, sleep state recognition method and device

Abstract

The embodiment of the present application discloses a training method and device for a sleep state recognition model, a sleep state recognition method, a computer and a computer-readable storage medium, the method comprising: obtaining physiological data collected by a wearable device for the same target object in the same time period, and obtaining EEG data collected by an EEG monitoring device; using the first sleep state recognition result of the EEG data to label the physiological data with a sleep state category, and obtaining the labeled physiological data; using the labeled physiological data, or the first physiological parameter feature extracted from the physiological data and the labeled physiological data to train a sleep state recognition model to be trained, and obtaining a trained sleep state recognition model. The embodiment of the present application uses the sleep state recognition result of the EEG data to label the physiological data that is not the EEG data with a sleep state category, so that the sleep state category label of the physiological data is more accurate, and the sleep state recognition accuracy of the physiological data of the wearable device is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of sleep monitoring, and in particular, relates to a method and device for training a sleep state recognition model, a sleep state recognition method, a computer, and a computer-readable storage medium. Background Art

[0002] Currently, users can monitor their sleep status through wearable devices such as smart bracelets and smart watches, or through EEG monitoring devices.

[0003] Among them, wearable devices such as smart bracelets and smart hands recognize sleep status based on the user's physiological data and movements (such as the number of times they turn over), and physiological data include heartbeat, electrocardiogram, and skin electricity, etc. EEG monitoring equipment recognizes sleep status based on the user's EEG data.

[0004] Sleep is controlled by the center at the tail end of the brainstem, so monitoring sleep with brain waves is more accurate than monitoring sleep with physiological data. In other words, the current method of monitoring sleep status through physiological data collected by wearable devices has low accuracy. Summary of the invention

[0005] The embodiments of the present application provide a method and device for training a sleep state recognition model, a sleep state recognition method, a computer, and a computer-readable storage medium, which can solve the problem of low accuracy of existing sleep state monitoring using physiological data collected by wearable devices.

[0006] In a first aspect, an embodiment of the present application provides a method for training a sleep state recognition model, comprising:

[0007] Obtain physiological data collected by wearable devices. Physiological data is non-EEG data.

[0008] Obtaining EEG data collected by an EEG monitoring device, where the EEG data and physiological data are data collected from the same target object in the same time period;

[0009] Performing sleep state recognition on the EEG data to obtain a first sleep state recognition result;

[0010] Using the first sleep state recognition result to label the physiological data into a sleep state category, to obtain labeled physiological data, wherein the labeled physiological data includes a sleep state category label;

[0011] The sleep state recognition model to be trained is trained using the annotated physiological data, or the first physiological parameter feature and the annotated physiological data to obtain a trained sleep state recognition model, wherein the first physiological parameter feature is a feature extracted from the physiological data.

[0012] As can be seen from the above, the embodiment of the present application uses the sleep state recognition results based on EEG data to label the sleep state category of physiological data that is not EEG data, so that the sleep state category labels of the physiological data are more accurate, and then the sleep state recognition model trained based on the labeled physiological data has higher sleep state recognition accuracy for the physiological data collected by the wearable device.

[0013] In some possible implementations of the first aspect, using the labeled physiological data, or the first physiological parameter feature and the labeled physiological data, to train a sleep state recognition model to be trained, and obtaining a trained sleep state recognition model includes:

[0014] Inputting the second physiological parameter feature extracted from the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, into the sleep state recognition model to be trained, and obtaining a second sleep state recognition result output by the sleep state recognition model to be trained;

[0015] According to the second sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained;

[0016] After multiple iterations of training, a trained sleep state recognition model is obtained.

[0017] In some possible implementations of the first aspect, using the labeled physiological data, or the physiological parameter features and the labeled physiological data, to train the sleep state recognition model to be trained, and obtaining the trained sleep state recognition model includes:

[0018] Selecting a first target feature from the first physiological parameter feature, or selecting a second target feature from the second physiological parameter feature, where the second physiological parameter feature is a feature extracted from the annotated physiological data;

[0019] Inputting the second target feature, or the first target feature and the annotated physiological data into the sleep state recognition model to be trained, to obtain a third sleep state recognition result output by the sleep state recognition model to be trained;

[0020] According to the third sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained, and obtaining the target sleep state recognition model after multiple iterations of training;

[0021] Use the test dataset to test the sleep state classification accuracy of the target sleep state recognition model;

[0022] If the sleep state classification accuracy meets the preset indicator requirements, the target sleep state recognition model is determined as the trained sleep state recognition model;

[0023] If the sleep state classification accuracy does not meet the preset index requirement, return to the step of selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, until the sleep state classification accuracy meets the preset index requirement.

[0024] In this implementation, feature selection and parameter optimization are performed based on the sleep state classification accuracy. While maintaining the model classification accuracy, the computational overhead is reduced by reducing the number of features.

[0025] In some possible implementations of the first aspect, selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, includes:

[0026] A first target feature is selected from the first physiological parameter feature using a multi-objective genetic algorithm, or a second target feature is randomly selected from the second physiological parameter feature using a multi-objective genetic algorithm.

[0027] In some possible implementations of the first aspect, performing sleep state recognition on EEG data to obtain a first sleep state recognition result includes:

[0028] Extract features from EEG data to obtain EEG features;

[0029] According to the EEG characteristics, determine the dominant EEG wave type in the EEG data;

[0030] A first sleep state identification result is obtained based on the dominant brain wave type.

[0031] In this implementation, the sleep state recognition result is obtained by occupying the dominant brain wave type, and the sleep state recognition is more accurate, thereby further improving the label accuracy of the physiological data.

[0032] In a second aspect, an embodiment of the present application provides a sleep state recognition method, comprising:

[0033] Obtain physiological data collected by wearable devices, which are non-EEG data;

[0034] Extract target features from physiological data;

[0035] Inputting the target feature into the trained sleep state recognition model to obtain a sleep state recognition result output by the sleep state recognition model;

[0036] The sleep state recognition model is a model obtained by training using any one of the training methods in the first aspect mentioned above.

[0037] In a third aspect, an embodiment of the present application provides a training device for a sleep state recognition model, comprising:

[0038] The physiological data acquisition module is used to acquire the physiological data collected by the wearable device. The physiological data is non-EEG data;

[0039] The EEG data acquisition module is used to acquire the EEG data collected by the EEG monitoring device. The EEG data and physiological data are data collected from the same target object in the same time period;

[0040] An identification module, used to identify the sleep state of the EEG data and obtain a first sleep state identification result;

[0041] a labeling module, configured to label the physiological data into a sleep state category using the first sleep state recognition result to obtain labeled physiological data, wherein the labeled physiological data includes a sleep state category label;

[0042] The training module is used to train the sleep state recognition model to be trained using the labeled physiological data, or the first physiological parameter feature and the labeled physiological data, to obtain a trained sleep state recognition model, wherein the first physiological parameter feature is a feature extracted from the physiological data.

[0043] In some possible implementations of the third aspect, the training module is specifically used to:

[0044] Selecting a first target feature from the first physiological parameter feature, or selecting a second target feature from the second physiological parameter feature, where the second physiological parameter feature is a feature extracted from the annotated physiological data;

[0045] Inputting the second target feature, or the first target feature and the annotated physiological data into the sleep state recognition model to be trained, to obtain a third sleep state recognition result output by the sleep state recognition model to be trained;

[0046] According to the third sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained, and obtaining the target sleep state recognition model after multiple iterations of training;

[0047] Use the test dataset to test the sleep state classification accuracy of the target sleep state recognition model;

[0048] If the sleep state classification accuracy meets the preset indicator requirements, the target sleep state recognition model is determined as the trained sleep state recognition model;

[0049] If the sleep state classification accuracy does not meet the preset index requirement, return to the step of selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, until the sleep state classification accuracy meets the preset index requirement.

[0050] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method of any one of the first or second aspects described above is implemented.

[0051] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method of any one of the first aspect or the second aspect described above is implemented.

[0052] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods in the first or second aspect above.

[0053] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A schematic flow chart of a method for training a sleep state recognition model provided in an embodiment of the present application;

[0056] Figure 2 A schematic block diagram of a sleep state recognition model training process provided in an embodiment of the present application;

[0057] Figure 3 Another schematic block diagram of the sleep state recognition model training process provided in an embodiment of the present application;

[0058] Figure 4 Another schematic block diagram of the sleep state recognition model training process provided in the embodiment of the present application;

[0059] Figure 5 Another schematic block diagram of the sleep state recognition model training process provided in the embodiment of the present application;

[0060] Figure 6 A schematic block diagram of a process of a sleep state recognition method provided in an embodiment of the present application;

[0061] Figure 7A structural block diagram of a sleep state recognition model training device provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0064] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0065] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0066] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0067] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0068] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0069] Before introducing the technical solutions provided by the embodiments of the present application, the following describes the relevant contents that may be involved in the embodiments of the present application.

[0070] Brain waves refer to the electrical oscillations produced by the activity of nerve cells in the human brain. Because these oscillations appear like waves on scientific instruments, they are called brain waves.

[0071] Brain waves can be divided into four categories according to their frequency: beta waves (conscious), alpha waves (bridge consciousness), theta waves (subconscious) and delta waves (unconscious). The combination of these consciousnesses forms a person's internal and external behavior, emotions and learning performance.

[0072] Normally, when a person is in a state of tension, the brain mainly produces beta waves; when a person's body is relatively relaxed, the brain mainly produces alpha waves; when a person feels sleepy, the brain mainly produces theta waves; and when a person enters deep sleep, the brain mainly produces delta waves.

[0073] EEG data is obtained by collecting the weak bioelectricity generated by the human brain at the scalp through EEG acquisition equipment, amplifying and recording the signal data.

[0074] The technical solution provided in the embodiments of the present application will be described in detail below.

[0075] At present, wearable devices such as smart bracelets, smart hands and smart glasses can collect physiological data in addition to EEG data, and monitor sleep status based on these physiological data to obtain sleep status monitoring results.

[0076] However, wearable devices such as smart bracelets and smart wrists judge whether a person is asleep by whether they turn over or whether their heart rate and blood pressure are stable, which is not very accurate. Because a person's heart rate and blood pressure are stable or they do not turn over frequently, it does not necessarily mean that they are in deep sleep. In addition, these external indicators cannot finely classify the sleeping state into light sleep and deep sleep.

[0077] In other words, the existing non-EEG physiological data collected by wearable devices has low accuracy in monitoring sleep status.

[0078] In the embodiment of the present application, the sleep state classification of the non-EEG physiological data collected by the wearable device is labeled by using the sleep state recognition result of the EEG data, and the labeled physiological data is used to train the sleep state model. In this way, since the sleep state recognition result of the EEG data is used to label the sleep state classification of the physiological data, the sleep state classification label of the physiological data is more accurate, thereby making the recognition accuracy of the trained sleep state recognition model higher.

[0079] It is understandable that human sleep is actually controlled by the central nervous system, and the sleep center is located at the tail end of the brainstem, so brain waves can accurately represent the human sleep state. That is, compared with the existing method of judging the sleep state by external indicators such as whether to turn over, whether the heart rate and blood pressure are stable, the sleep state recognition result based on the EEG data in the embodiment of the present application marks the sleep state category of the physiological data, which can make the model more accurately identify the sleep state.

[0080] See also Figure 1 , is a schematic flow chart of a method for training a sleep state recognition model provided in an embodiment of the present application. The method can be applied to electronic devices such as computers, workstations, and servers, and is not limited here. The method comprises the following steps:

[0081] Step S101: Acquire physiological data collected by a wearable device, where the physiological data is non-EEG data.

[0082] It should be noted that the above-mentioned wearable devices refer to wearable smart devices, which may include but are not limited to: smart watches, smart bracelets, smart glasses and smart headphones.

[0083] Wearable devices can collect the above physiological data through sensors. The physiological data refers to physiological data other than EEG, which may include but are not limited to: heart rate, electrocardiogram, skin electricity, and various biomarkers in human epidermal tissue fluid.

[0084] Among them, heart rate refers to the number of heart beats per minute in a normal person in a quiet state, also called resting heart rate, which is generally 60-100 beats / minute; electrocardiogram (ECG) refers to the combination of action potentials generated by myocardial cells when the human heart pulses; skin electricity is an emotional physiological indicator that represents the changes in skin electrical conduction when the body is stimulated.

[0085] Various biomarkers in human epidermal tissue fluid may include but are not limited to: glucose, lactic acid, pH, cholesterol, etc.

[0086] Optionally, three types of physiological data, namely, heart rate, electrocardiogram and skin charge, of the user may be obtained. Of course, other types of physiological data may also be obtained.

[0087] Step S102: acquiring EEG data collected by an EEG monitoring device, wherein the EEG data and physiological data are data collected from the same target object in the same time period.

[0088] In a specific application, after the user enters the sleep state, the wearable device and the EEG monitoring device are used to monitor the same user at the same time to collect the above-mentioned EEG data and non-EEG physiological data. For example, the user is asked to wear a headband (i.e., EEG monitoring device) and a smart bracelet to sleep for a period of time (e.g., one hour) to collect EEG data and non-EEG physiological data during this period.

[0089] The EEG monitoring device refers to a device used to monitor the user's EEG, which can be a wearable device, for example, a smart headband or headband used to collect EEG data; of course, the EEG monitoring device can also be a medical device for monitoring EEG. When the EEG monitoring device is a wearable device, the wearable device in the above step S101 refers to a wearable device used to collect non-EEG physiological data, and the EEG monitoring device refers to a wearable device used to collect EEG data.

[0090] Step S103: performing sleep state recognition on the EEG data to obtain a first sleep state recognition result.

[0091] In a specific application, after obtaining the EEG data, the EEG data can be preprocessed and feature extracted to extract EEG features from the EEG data, and the first sleep state recognition result can be obtained based on the EEG features.

[0092] The EEG preprocessing may include but is not limited to the following steps: removing power frequency (e.g., 50 Hz or 60 Hz) interference, removing electrooculographic interference through independent component analysis (ICA), bandpass filtering, extreme value processing, and wavelet noise filtering.

[0093] After the EEG data is preprocessed, EEG features are extracted from the preprocessed EEG data. For example, for EEG data, features are calculated every 8 seconds, and five band signals of the EEG data in the past 8 seconds are extracted: alpha, beta, gamma, theta, and sigma; the maximum value, minimum value, mean, variance, rate of change, first-order difference average, second-order difference average, normalized first-order difference average, normalized second-order difference average, range of change, and sum of squares of sequence differences are calculated for each band signal. Among them, for the above-mentioned rate of change, the rate of change is positive when it rises and negative when it falls.

[0094] In some embodiments, the dominant brain wave type can be determined based on the extracted EEG features, and the first sleep state recognition result can be obtained based on the dominant brain wave type. The first sleep state recognition result is used to characterize the sleep state, which can be expressed as a descriptive state classification as shown in Table 1 below, or as a number used to describe the degree of sleep, which is not limited here.

[0095] Table 1

[0096] Dominant brain wave type First sleep state recognition result Beta waves wide awake Alpha waves Light sleep Theta Waves Mid-sleep Delta Wave Deep sleep

[0097] The definition of the dominant brain wave type may refer to the brain wave with the highest power statistical average value or the highest normalized power statistical average value in a certain period of time (e.g., 2 minutes). Of course, the definition of the dominant brain wave type can be determined based on actual experience or specific circumstances, and is not limited here.

[0098] In other embodiments, the sleep state recognition result of the EEG data of the EEG monitoring device may also be obtained through existing sleep monitoring methods or sleep state monitoring models, which are not limited here.

[0099] Step S104: label the physiological data with a sleep state category using the first sleep state recognition result to obtain labeled physiological data, where the labeled physiological data includes a sleep state category label.

[0100] According to the sleep state recognition result of the EEG data, the non-EEG physiological data of the same user in the same time period is labeled with a sleep state category label, and the sleep state category label is used to characterize the sleep state. For example, in time period A, through the first sleep state recognition result of the EEG data, it is known that the user is in a light sleep state, and the physiological data of the user in time period A is labeled with a light sleep state label; in time period B, through the first sleep state recognition result of the EEG data, it is known that the user is in a deep sleep state, and the physiological data of the user in time period B is labeled with a deep sleep state label. After the above labeling process, physiological data including sleep state category labels can be obtained.

[0101] In specific applications, the labeling process can be automatic labeling, that is, automatically labeling the corresponding physiological data with a sleep state category label based on the EEG characteristics; or it can be manual labeling, that is, manually labeling the corresponding physiological data with a sleep state category label based on the above-mentioned first sleep state recognition result.

[0102] Step S105: Use the annotated physiological data, or the first physiological parameter feature and the annotated physiological data to train the sleep state recognition model to obtain a trained sleep state recognition model, where the first physiological parameter feature is a feature extracted from the physiological data.

[0103] In a specific application, before extracting features from physiological data or annotated physiological data, a preprocessing step may be performed before feature extraction.

[0104] The preprocessing step may include, but is not limited to, signal amplification and noise removal. Relatively pure physiological data can be obtained through the preprocessing step.

[0105] The categories of physiological data are different, and the physiological parameter features extracted from the physiological data or the annotated physiological data may also be different accordingly.

[0106] For example, for ECG data, the physiological parameter features may include physiological parameter features at four levels: linear, nonlinear, time domain, and frequency domain. The physiological parameter features may include the following feature quantities:

[0107] SDNN: standard deviation of all sinus beat RR intervals;

[0108] NN50: the number of adjacent NNs with a difference of >50ms;

[0109] PNN50: the percentage of the number of adjacent NNs with a difference of >50ms in the total number of sinus beats;

[0110] SDSD: standard deviation of the difference between adjacent RR intervals;

[0111] RR_MEAN: the average value of RR gap;

[0112] ECG: Min, Max, Mean, Var after analyzing baseline drift;

[0113] Wavelets: Use db6 wavelet, 3-layer decomposition processing, and count the maximum, minimum, median, and standard deviation of 3 layers of high-frequency details and 1 layer of low-frequency approximation;

[0114] VLF (very low frequency), LF (low frequency), and HF (high frequency) energy.

[0115] For heart rate data, electrodermal data, and biomarkers, feature extraction is mainly based on statistical features. At this time, physiological parameter features may include maximum value, minimum value, mean, variance, rate of change (if the rate of change is positive when rising, the rate of change is negative when falling), first-order difference average, second-order difference average, normalized first-order difference average, normalized second-order difference average, range of change, and sum of squares of sequence differences.

[0116] In the specific feature extraction process, within the same period of time as the EEG data extraction, the time of the physiological data is cut into smaller units (for example, 2 minutes each), and all averaging processes are performed in the smaller time unit.

[0117] It should be noted that the above sleep state recognition model is a machine learning model, which can be but not limited to: k-nearest neighbor model, perceptron, naive Bayes model, decision tree, logistic regression model, support vector machine, adaBoost, Bayesian network or neural network model.

[0118] As can be seen from the above, the embodiment of the present application uses the sleep state recognition results based on EEG data to label the sleep state category of physiological data that is not EEG data, so that the sleep state category labels of the physiological data are more accurate, and then the sleep state recognition model trained based on the labeled physiological data has higher sleep state recognition accuracy for the physiological data collected by the wearable device.

[0119] Based on the above embodiment, during the model training process, the second physiological parameter feature extracted from the annotated physiological data can be used as the model input, or the annotated physiological data and the first physiological parameter feature extracted from the unannotated physiological data can be used as the model input. The following will introduce and explain these two situations respectively.

[0120] In some embodiments, after extracting the second physiological parameter feature from the annotated physiological data, the second physiological parameter feature is input into the sleep state recognition model to be trained to obtain the second sleep state recognition result output by the sleep state recognition model to be trained; then, based on the second sleep state recognition result and the sleep state category label, a loss value is calculated, and back propagation is performed based on the loss value to adjust the network parameters of the sleep state recognition model to be trained; then, the second physiological parameter feature is input into the sleep state recognition model to be trained after the network parameters are adjusted, and the sleep state recognition result output by the model is obtained, and back propagation is performed after the loss value is calculated; after iterative training for multiple times until the loss value tends to be stable, a trained sleep state recognition model can be obtained.

[0121] For example, see Figure 2 A schematic diagram of the sleep state recognition model training process is shown in FIG. Figure 2As shown, the physiological data collected by the wearable device is obtained, and the EEG data collected by the EEG monitoring device in the same time period is obtained; the sleep state recognition result is obtained based on the EEG data, and the sleep state recognition result of the EEG data is used to mark the sleep state category label of the physiological data to obtain the labeled physiological data; the feature extraction of the labeled physiological data is performed, and the extracted physiological parameter features are applied to the training of the machine learning model; when the training is completed, the trained model and the output features are obtained.

[0122] The machine learning model refers to a sleep state recognition model. In this process, all the extracted physiological parameter features are used for model training, and after the training is completed, all the physiological parameter features will be output.

[0123] In other embodiments, a first physiological parameter feature is extracted from unlabeled physiological data; the first physiological parameter feature and the labeled physiological data are input into the sleep state recognition model to be trained to obtain a second sleep state recognition result output by the sleep state recognition model to be trained; then, based on the second sleep state recognition result and the sleep state category label, a loss value is calculated, and back propagation is performed based on the loss value to adjust the network parameters of the sleep state recognition model to be trained; after multiple iterative trainings until the loss value becomes stable, a trained sleep state recognition model can be obtained.

[0124] For example, see Figure 3 Another schematic diagram of the sleep state recognition model training process is shown in FIG. Figure 3 As shown, the physiological data collected by the wearable device is obtained, and the EEG data collected by the EEG monitoring device in the same time period is obtained; the sleep state recognition result is obtained according to the EEG data, and the sleep state recognition result of the EEG data is used to label the sleep state category label of the physiological data to obtain the labeled physiological data; the unlabeled physiological data is subjected to feature extraction, and the extracted physiological parameter features and the labeled physiological data are applied to the training of the machine learning model; when the training is completed, the trained model and the output features are obtained.

[0125] exist Figure 2 and Figure 3 In the corresponding embodiment, all the extracted physiological parameter features are input into the sleep state recognition model and act on the model training process. However, if all the extracted physiological parameter features are used for model training, the number of features is large and the calculation cost is large.

[0126] In order to reduce the number of features to reduce computing overhead while maintaining model accuracy, the embodiments of the present application can perform feature selection and parameter optimization based on the sleep state classification accuracy to improve the sleep state classification accuracy and reduce the number of features.

[0127] In some embodiments, if the second physiological parameter feature extracted from the labeled physiological data is used as the model input, a second target feature is selected from the second physiological parameter feature, and the second target feature refers to a partial feature set selected from the second physiological parameter feature. The partial feature set is used as the input of the model, rather than using all the extracted second physiological parameter features as the model input.

[0128] In specific applications, the feature selection method can be random selection, or the second generation non-dominated sorting genetic algorithm (NSGA-II) can be used for feature selection. The steps of NSGA-II may include: first, find the Pareto solution for M physiological parameter features; then obtain the Pareto set of F1, F2...Fk, etc.; put all the physiological parameter features of F1 into N, if N is not full, continue to put F2, until there is Fk that cannot be fully put into N (space) that has been put into F1, F2,..., F(k-1), then solve Fk; for the Pareto solution in Fk, find the crowding distance Lk[i] (crowding distance) of each Pareto in Fk, sort in Fk in descending order according to Lk[i], put them into N, until N is full, and finally obtain the physiological parameter feature set. The physiological parameter feature set is the selected feature parameter.

[0129] After selecting the second target feature, the second target feature is input into the sleep state recognition model to be trained to obtain the third sleep state recognition result output by the sleep state recognition model to be trained, and the loss value is calculated according to the third sleep state recognition result and the sleep state category label, and back propagation is performed according to the loss value to adjust the network parameters of the sleep state recognition model to be trained; the training is iterated multiple times until the loss value tends to be stable, and the target sleep state recognition model is obtained.

[0130] After obtaining the target sleep state recognition model, the sleep state classification accuracy of the target sleep state recognition model is tested using a test data set. The test data set is pre-constructed and includes physiological data and corresponding sleep state category labels.

[0131] If the sleep state classification accuracy meets the preset indicator requirement, the target sleep state recognition model is determined as the trained sleep state recognition model, and the selected second target feature is output. The preset indicator requirement can represent a threshold, that is, if the sleep state classification accuracy is greater than a certain threshold, it is considered to meet the preset indicator requirement, otherwise, if it is less than the threshold, it is considered not to meet the preset indicator requirement.

[0132] If the accuracy of sleep state classification does not meet the preset index requirement, the process returns to the step of selecting the second target feature from the second physiological parameter feature, that is, returns to the step of feature selection, continues to select a partial feature set, and then inputs the selected partial feature set into the sleep state recognition model. This cycle is repeated until the accuracy of sleep state classification meets the preset index requirement.

[0133] For example, see Figure 4 Another schematic block diagram of the sleep state recognition model training process is shown in FIG. Figure 4 As shown, the physiological data collected by the wearable device is obtained, and the EEG data collected by the EEG monitoring device in the same time period is obtained; the sleep state recognition result is obtained according to the EEG data, and the sleep state recognition result of the EEG data is used to label the sleep state category label of the physiological data to obtain the labeled physiological data; the labeled physiological data is subjected to feature extraction, and the extracted physiological parameter features are used for the training of the machine learning model; it is determined whether the sleep state classification accuracy meets the index requirements, and if so, the model and features are output, that is, the current sleep state recognition model is used as the trained model, and the physiological parameter feature set input by the current model is used as the input of the model; if not, the step of feature selection is returned to continue to randomly select or use a multi-objective genetic algorithm for feature selection, and the selected physiological parameter feature set is used for model training. This cycle is repeated until the sleep state classification accuracy meets the index requirements.

[0134] In other embodiments, if the annotated physiological data and the first physiological parameter feature extracted from the unannotated physiological data are used as model inputs, a first target feature is selected from the first physiological parameter feature. The first target feature is a partial feature set of the first physiological parameter feature.

[0135] The feature selection method can be random selection or the use of a multi-objective genetic algorithm for feature selection, which will not be described in detail here.

[0136] After selecting the first target feature, the first target feature and the annotated physiological data are input into the sleep state recognition model to be trained, and the third sleep state recognition result output by the sleep state recognition model to be trained is obtained. The loss value is calculated based on the third sleep state recognition result and the sleep state category label, and back propagation is performed based on the loss value to adjust the network parameters of the sleep state recognition model to be trained. The training is iterated multiple times, and when the loss value tends to be stable, the target sleep state recognition model is obtained;

[0137] After obtaining the target sleep state recognition model, the sleep state classification accuracy of the target sleep state recognition model is tested using a test dataset.

[0138] If the sleep state classification accuracy meets the preset indicator requirement, the target sleep state recognition model is determined as the trained sleep state recognition model, and the selected first target feature is output.

[0139] If the accuracy of sleep state classification does not meet the preset index requirement, the process returns to the step of selecting the first target feature from the first physiological parameter feature, that is, returns to the step of feature selection, continues to select a partial feature set, and then inputs the selected partial feature set into the sleep state recognition model. This cycle is repeated until the accuracy of sleep state classification meets the preset index requirement.

[0140] For example, see Figure 5 Another schematic block diagram of the sleep state recognition model training process is shown in FIG. Figure 5 As shown, the physiological data collected by the wearable device is obtained, and the EEG data collected by the EEG monitoring device in the same time period is obtained; the sleep state recognition result is obtained according to the EEG data, and the sleep state recognition result of the EEG data is used to label the sleep state category label of the physiological data to obtain the labeled physiological data; the unlabeled physiological data is subjected to feature extraction, and the extracted physiological parameter features and the labeled physiological data are used for the training of the machine learning model; it is determined whether the sleep state classification accuracy meets the index requirements, and if so, the model and features are output, that is, the current sleep state recognition model is used as the trained model, and the physiological parameter feature set input by the current model is used as the input of the model; if not, the step of feature selection is returned to continue random selection or use of a multi-objective genetic algorithm for feature selection, and the selected physiological parameter feature set and the labeled physiological data are used for model training. This cycle is repeated until the sleep state classification accuracy meets the index requirements.

[0141] It should be noted that if the feature selection method is random selection, various feature parameter combinations can be tested and a feature parameter combination with the highest accuracy or exceeding a threshold can be selected as the final model input.

[0142] After obtaining the trained sleep state recognition model through the model training method mentioned above, the sleep state recognition model can be used to perform sleep state recognition.

[0143] See also Figure 6 , is a schematic flow chart of a sleep state recognition method provided in an embodiment of the present application. The method can be applied to wearable devices, such as smart bracelets and smart watches; it can also be applied to non-wearable devices, such as mobile phones, computers and other terminals, which are not limited here. The method may include the following steps:

[0144] Step S601: Acquire physiological data collected by the wearable device, where the physiological data is non-EEG data.

[0145] It is understandable that when users wear wearable devices to sleep, the wearable devices collect physiological data in addition to EEG.

[0146] Step S602: extract target features from physiological data.

[0147] It should be noted that the above-mentioned target features may refer to all the features extracted from the physiological data mentioned above, or may refer to some features selected from all the features extracted through the feature selection step mentioned above. In other words, the input of the model can be determined according to the model training stage. If the model input in the model training stage is all the features extracted from the physiological data, then the target feature refers to all the features. If the model input in the model training stage is a partial feature set selected through the feature selection step, then the target feature is a partial feature set.

[0148] Of course, the target feature may also refer to all the physiological parameter features mentioned in the above training stage. After all the physiological parameter features are extracted, some physiological parameter features are selected as the target feature.

[0149] Step S603: input the target feature into the trained sleep state recognition model to obtain the sleep state recognition result output by the sleep state recognition model.

[0150] Among them, the sleep state recognition model is a model trained using any of the training methods mentioned above, which will not be repeated here.

[0151] It can be seen that the embodiment of the present application uses the sleep state recognition results of EEG data to label non-EEG physiological data, so that the sleep state recognition accuracy of the trained model is higher.

[0152] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0153] Corresponding to the training method of the sleep state recognition model described in the above embodiment, Figure 7 A structural block diagram of a training device for a sleep state recognition model provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0154] Reference Figure 7 , the device comprises:

[0155] A physiological data acquisition module 71 is used to acquire physiological data collected by a wearable device, where the physiological data is non-EEG data;

[0156] The EEG data acquisition module 72 is used to acquire EEG data collected by an EEG monitoring device. The EEG data and physiological data are data collected from the same target object in the same time period;

[0157] The recognition module 73 is used to perform sleep state recognition on the EEG data to obtain a first sleep state recognition result;

[0158] A labeling module 74, configured to label the physiological data according to the sleep state category using the first sleep state recognition result, and obtain labeled physiological data, wherein the labeled physiological data includes a sleep state category label;

[0159] The training module 75 is used to train the sleep state recognition model to be trained using the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, to obtain a trained sleep state recognition model, wherein the first physiological parameter feature is a feature extracted from the physiological data.

[0160] In some possible implementations, the training module 75 is specifically used to:

[0161] Selecting a first target feature from the first physiological parameter feature, or selecting a second target feature from the second physiological parameter feature, where the second physiological parameter feature is a feature extracted from the annotated physiological data;

[0162] Inputting the second target feature, or the first target feature and the annotated physiological data into the sleep state recognition model to be trained, to obtain a third sleep state recognition result output by the sleep state recognition model to be trained;

[0163] According to the third sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained, and obtaining the target sleep state recognition model after multiple iterations of training;

[0164] Use the test dataset to test the sleep state classification accuracy of the target sleep state recognition model;

[0165] If the sleep state classification accuracy meets the preset indicator requirements, the target sleep state recognition model is determined as the trained sleep state recognition model;

[0166] If the sleep state classification accuracy does not meet the preset index requirement, return to the step of selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, until the sleep state classification accuracy meets the preset index requirement.

[0167] In some possible implementations, the training module 75 is specifically used to:

[0168] Inputting the second physiological parameter feature extracted from the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, into the sleep state recognition model to be trained, and obtaining a second sleep state recognition result output by the sleep state recognition model to be trained;

[0169] According to the second sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained;

[0170] After multiple iterations of training, a trained sleep state recognition model is obtained.

[0171] In some possible implementations, the training module 75 is specifically used to:

[0172] A first target feature is selected from the first physiological parameter feature using a multi-objective genetic algorithm, or a second target feature is randomly selected from the second physiological parameter feature using a multi-objective genetic algorithm.

[0173] In some possible implementations, the identification module 73 is specifically used to:

[0174] Extract features from EEG data to obtain EEG features;

[0175] According to the EEG characteristics, determine the dominant EEG wave type in the EEG data;

[0176] A first sleep state identification result is obtained based on the dominant brain wave type.

[0177] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0178] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8 Only one is shown), a memory 81, the memory 81 stores a computer program 82 that can be executed on at least one processor 80, and when the processor 80 executes the computer program 82, the steps in any of the above-mentioned method embodiments are implemented.

[0179] The electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will appreciate that Figure 8 It is only an example of the electronic device 8 and does not constitute a limitation on the electronic device 8. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0180] In one embodiment, the electronic device may be integrated with a gesture recognition module, and the gesture recognition module may be an infrared gesture recognition module. In one embodiment, the electronic device is integrated with two gesture recognition modules, and both gesture recognition modules are communicatively connected to the processor 80. It is understandable that the principles of waving gesture recognition and infrared gesture recognition are well known to those skilled in the art, and will not be elaborated here.

[0181] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0182] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 8. In one embodiment, the memory 81 may also include both an internal storage unit of the electronic device 8 and an external storage device. The memory 81 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0183] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0184] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0185] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0186] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0188] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0189] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0190] In the embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for training a sleep state recognition model, characterized in that: include: Acquire physiological data collected by a wearable device, wherein the physiological data is non-EEG data; Acquire EEG data collected by an EEG monitoring device, wherein the EEG data and the physiological data are data collected from the same target object in the same time period; Performing sleep state recognition on the EEG data to obtain a first sleep state recognition result; Using the first sleep state recognition result to label the physiological data into a sleep state category, to obtain labeled physiological data, wherein the labeled physiological data includes a sleep state category label; The sleep state recognition model to be trained is trained using the annotated physiological data, or the first physiological parameter feature and the annotated physiological data to obtain a trained sleep state recognition model, wherein the first physiological parameter feature is a feature extracted from the physiological data.

2. The method according to claim 1, characterized in that Using the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, to train the sleep state recognition model to be trained, and obtaining a trained sleep state recognition model, including: Inputting a second physiological parameter feature extracted from the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, into the sleep state recognition model to be trained, to obtain a second sleep state recognition result output by the sleep state recognition model to be trained; According to the second sleep state recognition result and the sleep state category label, adjusting network parameters of the sleep state recognition model to be trained; After multiple iterations of training, the trained sleep state recognition model is obtained.

3. The method according to claim 1, characterized in that Using the annotated physiological data, or the physiological parameter characteristics and the annotated physiological data, to train the sleep state recognition model to be trained, and obtaining a trained sleep state recognition model, including: Selecting a first target feature from the first physiological parameter features, or selecting a second target feature from the second physiological parameter features, where the second physiological parameter feature is a feature extracted from the annotated physiological data; inputting the second target feature, or the first target feature and the annotated physiological data into the sleep state recognition model to be trained, to obtain a third sleep state recognition result output by the sleep state recognition model to be trained; According to the third sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained, and obtaining a target sleep state recognition model after multiple iterations of training; Using a test data set to test the sleep state classification accuracy of the target sleep state recognition model; If the sleep state classification accuracy meets the preset index requirement, the target sleep state recognition model is determined as the trained sleep state recognition model; If the sleep state classification accuracy rate does not meet the preset index requirement, return to the step of selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, until the sleep state classification accuracy rate meets the preset index requirement.

4. The method according to claim 3, characterized in that Selecting a first target feature from the first physiological parameter feature, or selecting a second target feature from the second physiological parameter feature, comprises: The first target feature is selected from the first physiological parameter features using a multi-objective genetic algorithm, or the second target feature is randomly selected from the second physiological parameter features using a multi-objective genetic algorithm.

5. The method according to claim 1, characterized in that Performing sleep state recognition on the EEG data to obtain a first sleep state recognition result includes: Extracting features from the EEG data to obtain EEG features; Determining the dominant brain wave type in the brain wave data according to the brain wave characteristics; The first sleep state recognition result is obtained according to the dominant brain wave type.

6. A method for identifying a sleeping state, characterized in that: include: Acquire physiological data collected by a wearable device, wherein the physiological data is non-EEG data; extracting target features from the physiological data; Inputting the target feature into a trained sleep state recognition model to obtain a sleep state recognition result output by the sleep state recognition model; The sleep state recognition model is a model obtained by training using the training method described in any one of claims 1 to 5.

7. A training device for a sleep state recognition model, characterized in that: include: A physiological data acquisition module is used to acquire physiological data collected by a wearable device, wherein the physiological data is non-EEG data; An EEG data acquisition module is used to acquire EEG data collected by an EEG monitoring device, wherein the EEG data and the physiological data are data collected from the same target object in the same time period; an identification module, configured to perform sleep state identification on the EEG data to obtain a first sleep state identification result; a labeling module, configured to label the physiological data according to the sleep state category using the first sleep state recognition result to obtain labeled physiological data, wherein the labeled physiological data includes a sleep state category label; A training module is used to train the sleep state recognition model to be trained using the annotated physiological data, or the first physiological parameter feature and the annotated physiological data, to obtain a trained sleep state recognition model, wherein the first physiological parameter feature is a feature extracted from the physiological data.

8. The device according to claim 7, characterized in that The training module is specifically used for: Selecting a first target feature from the first physiological parameter features, or selecting a second target feature from the second physiological parameter features, where the second physiological parameter feature is a feature extracted from the annotated physiological data; inputting the second target feature, or the first target feature and the annotated physiological data into the sleep state recognition model to be trained, to obtain a third sleep state recognition result output by the sleep state recognition model to be trained; According to the third sleep state recognition result and the sleep state category label, adjusting the network parameters of the sleep state recognition model to be trained, and obtaining a target sleep state recognition model after multiple iterations of training; Using a test data set to test the sleep state classification accuracy of the target sleep state recognition model; If the sleep state classification accuracy meets the preset index requirement, the target sleep state recognition model is determined as the trained sleep state recognition model; If the sleep state classification accuracy rate does not meet the preset index requirement, return to the step of selecting the first target feature from the first physiological parameter feature, or selecting the second target feature from the second physiological parameter feature, until the sleep state classification accuracy rate meets the preset index requirement.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 or 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 or 6 is implemented.

Images