Sleep quality evaluation method and related device

Abstract

A sleep quality evaluation method and a related device. The sleep quality evaluation method comprises: generating first comprehensive sleep data on the basis of first sensor data from a first ear-worn device and second sensor data from a second ear-worn device (S105), and determining a sleep quality evaluation result on the basis of the first comprehensive sleep data (S107). By using multi-channel data from the two ear-worn devices for sleep quality evaluation, the accuracy of the sleep quality evaluation result is improved.

Description

Sleep quality assessment methods and related equipment Technical Field

[0001] This application relates to the field of sleep monitoring technology, and in particular to a method and related equipment for assessing sleep quality. Background Technology

[0002] Sleep is a vital part of human life. It helps us restore physical strength, promotes brain and body repair, and enhances memory and learning ability. Modern people are paying increasing attention to sleep, leading to a proliferation of corresponding monitoring products, especially with the development of wearable devices, resulting in a plethora of home sleep monitoring products, such as watches and headphones. However, existing sleep monitoring products suffer from inaccurate monitoring and an inability to accurately assess sleep quality. Summary of the Invention

[0003] This application improves the accuracy of sleep quality assessment results by using multi-channel data from two ear-worn devices to assess sleep quality.

[0004] This application provides a sleep quality assessment method, comprising: generating first comprehensive sleep data based on first sensor data from a first ear wearable device and second sensor data from a second ear wearable device; and determining a sleep quality assessment result based on the first comprehensive sleep data.

[0005] In some embodiments, generating first comprehensive sleep data based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device includes: generating first sleep data based on the first sensor data from the first ear-wearing device; generating second sleep data based on the second sensor data from the second ear-wearing device; and generating first comprehensive sleep data based on the first sleep data and the second sleep data.

[0006] In some embodiments, the method further includes: generating second comprehensive sleep data based on third sensor data from the first ear wearable device and fourth sensor data from the second ear wearable device, wherein determining the sleep quality assessment result based on the first comprehensive sleep data includes: determining the sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data.

[0007] In some embodiments, generating second comprehensive sleep data based on third sensor data from a first ear-wearing device and fourth sensor data from a second ear-wearing device includes: generating third sleep data based on the third sensor data from the first ear-wearing device; generating fourth sleep data based on the fourth sensor data from the second ear-wearing device; and generating second comprehensive sleep data based on the third sleep data and the fourth sleep data.

[0008] In some embodiments, the first sensor data and the second sensor data are first type data; the third sensor data and the fourth sensor data are second type data.

[0009] In some embodiments, the first sleep data, the second sleep data, the third sleep data, the fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state of each time window in the sleep interval, wherein the sleep state is selected from at least one of the following groups or from at least one of the following groups after adding uncertain states: wakefulness and sleep; wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3; wakefulness, REM sleep, light sleep, and deep sleep; or wakefulness, REM sleep, and non-REM sleep.

[0010] In some embodiments, the first ear-wearing device and the second ear-wearing device are a pair of earphones, one of which is the left earphone and the other is the right earphone.

[0011] In some embodiments, the first type of data is obtained by an accelerometer and the second type of data is obtained by a photoplethysmography (PPG) sensor, or the first type of data is obtained by a PPG sensor and the second type of data is obtained by an accelerometer.

[0012] In some embodiments, the first sleep data includes time periods with no data and / or time windows where the sleep state is uncertain. Generating first comprehensive sleep data based on the first sleep data and the second sleep data includes: using the second sleep data to determine the first comprehensive sleep data for the time periods with no data in the first sleep data; and using the second sleep data to determine the first comprehensive sleep data for the time windows where the sleep state in the first sleep data is uncertain.

[0013] In some embodiments, the first sleep data and the second sleep data each indicate the sleep state in each time window within a sleep interval. Generating first comprehensive sleep data based on the first sleep data and the second sleep data includes: if the first sleep data and the second sleep data have the same sleep state in a time window, then using the data of the first sleep data in the time window to determine the first comprehensive sleep data; if the first sleep data and the second sleep data have different sleep states in a time window, then using the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in the time window to determine the first comprehensive sleep data, or using the data of the first sleep data in the time window to determine the first comprehensive sleep data, wherein the feature value is the signal-to-noise ratio.

[0014] In some embodiments, the time period in which the first sensor data is missing within the sleep interval is shorter than the time period in which the second sensor data is missing within the sleep interval. Generating first comprehensive sleep data based on the first sensor data from the first ear wearable device and the second sensor data from the second ear wearable device includes: for the time period in the sleep interval in which the first sensor data is missing, extracting the second sensor data, and generating first comprehensive sleep data based on the first sensor data and the extracted second sensor data; or, generating first comprehensive sleep data based on the first sensor data.

[0015] In some embodiments, generating first comprehensive sleep data based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device includes: if the feature value of the first sensor data in a time window is higher than the feature value of the second sensor data in a time window, then using the sleep data corresponding to the first sensor data in the time window to determine the first comprehensive sleep data; if the feature value of the first sensor data in a time window is lower than the feature value of the second sensor data in the time window, then using the sleep data corresponding to the second sensor data in the time window to determine the first comprehensive sleep data, wherein the feature value is the signal-to-noise ratio.

[0016] In some embodiments, the average signal-to-noise ratio of the first sensor data is higher than that of the second sensor data, or the length of the data-free period of the first sleep data is shorter than that of the data-free period of the second sleep data.

[0017] In some embodiments, determining a sleep quality assessment result based on first comprehensive sleep data includes: normalizing the first comprehensive sleep data; modifying the normalized first comprehensive sleep data to a standard size; and obtaining a sleep quality assessment result based on the standard-sized first comprehensive sleep data through deep learning modeling.

[0018] In some embodiments, sleep quality assessment results include difficulty falling asleep, normal sleep, insomnia, or light sleep.

[0019] In some embodiments, the first comprehensive sleep data includes sleep states corresponding to multiple time windows, and determining the sleep quality assessment result based on the first comprehensive sleep data includes: inputting the first comprehensive sleep data into a preset sleep model to obtain the sleep type corresponding to the first comprehensive sleep data; wherein, the preset sleep model is a model trained based on a sleep data sample set and a deep learning model.

[0020] In some embodiments, the deep learning model includes a deep clustering model.

[0021] In some embodiments, the step of determining a preset sleep model includes: inputting sleep data from a sleep data sample set into an initial deep clustering model and training the initial deep clustering model to obtain a preset sleep model. The first comprehensive sleep data sample set includes sleep data from multiple users. The initial deep clustering model includes a cascaded preset feature extraction module and a comparison module. Inputting sleep data from the sleep data sample set into the initial deep clustering model and training the initial deep clustering model to obtain a preset sleep model includes: inputting sleep data from the first comprehensive sleep data sample set into the preset feature extraction module to obtain a first output result from the comparison module; obtaining a loss value based on the first output result; and training the preset feature extraction module and the comparison module based on the loss value to obtain the preset sleep model.

[0022] In some embodiments, sleep data from a first comprehensive sleep data sample set is input into a preset feature extraction module to obtain a first output result from a comparison module. This includes: randomly segmenting the sleep data from the first comprehensive sleep data sample set to obtain a first sleep segment set and a second sleep segment set; inputting the first sleep segment set and the second sleep segment set into a first preset feature extraction module and a second preset feature extraction module, respectively, to obtain a first feature vector and a second feature vector; inputting both the first feature vector and the second feature vector into a first convolutional neural network and a softmax function, respectively, to obtain a first predicted class probability vector and a second predicted class probability vector; and inputting the first feature vector and the second feature vector into a second convolutional neural network, respectively, to obtain a third feature vector and a fourth feature vector. The first output result includes the first predicted class probability vector, the second predicted class probability vector, the third feature vector, and the fourth feature vector.

[0023] In some embodiments, randomly segmenting sleep data in a first comprehensive sleep data sample set to obtain a first sleep segment set and a second sleep segment set includes: randomly segmenting each sleep data in the first comprehensive sleep data sample set to obtain a first sleep segment and a second sleep segment that correspond one-to-one with the first comprehensive sleep data; determining a first sleep segment set based on each first sleep segment; and determining a second sleep segment set based on each second sleep segment.

[0024] In some embodiments, the parameters in the first preset feature extraction module are the same as those in the second preset feature extraction module; the number of convolutional layers in the first convolutional neural network is greater than the number of convolutional layers in the second convolutional neural network; and the duration of random segmentation is greater than one sleep cycle.

[0025] In some embodiments, obtaining a loss value based on a first output result includes: determining the class similarity between a first predicted class probability vector and a second predicted class probability vector, the mean probability of the first predicted class probability vector, and the mean probability of the second predicted class probability vector; determining a class loss value based on the class similarity; determining the feature similarity between a third feature vector and a fourth feature vector, and determining a feature loss value based on the feature similarity; and determining a loss value based on the class loss value, the feature loss value, the mean probability of the first predicted class probability vector, and the mean probability of the second predicted class probability vector.

[0026] In some embodiments, determining a loss value based on the category loss value, feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector includes: determining the entropy of the cluster assignment probability based on the probability mean of the first predicted category probability vector and the probability mean of the second predicted category probability vector; and determining the loss value based on the category loss value, feature loss value, and the entropy of the cluster assignment probability.

[0027] In some embodiments, before the step of inputting the first sleep segment set and the second sleep segment set to the first preset feature extraction module and the second preset feature extraction module respectively, the method further includes: performing random Gaussian blurring, resampling and standardization processing on each sleep segment in the first sleep segment set and the second sleep segment set in sequence.

[0028] In some embodiments, prior to the step of inputting the first comprehensive sleep data into a preset sleep model, the method further includes: resampling and standardizing the first comprehensive sleep data.

[0029] This application also provides a sleep quality assessment method, comprising: receiving first comprehensive sleep data; receiving second comprehensive sleep data; and determining a sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data, wherein the first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, the first sensor data and the second sensor data being first type data, and the second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, the third sensor data and the fourth sensor data being second type data.

[0030] This application also provides a wearable device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any of the above embodiments of this application.

[0031] This application also provides a computer-readable storage medium having a computer program stored thereon, wherein in some embodiments, the computer program, when executed by a processor, implements the steps of the method of any of the above embodiments of this application.

[0032] This application also provides a sleep quality assessment device. In some embodiments, the sleep quality assessment device includes a wearable device and a terminal device. The wearable device includes a first ear-wearing device and a second ear-wearing device. The wearable device is used to generate first comprehensive sleep data based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. The terminal device is used to determine a sleep quality assessment result based on the first comprehensive sleep data.

[0033] This application also provides a terminal device, including a memory and a processor. The memory stores a computer program, and in some embodiments, the processor executes the computer program to implement the steps of the method of any of the above embodiments of this application.

[0034] This application assesses sleep quality based on multi-channel data (such as two-channel or four-channel data) from two ear-worn devices. If one ear-worn device fails to collect sensor data or provides inaccurate data due to issues such as poor fit, looseness, detachment, lack of battery power, or malfunction, sensor data from the other ear-worn device can supplement the assessment. In other words, the sensor data from the two ear-worn devices can complement each other, thereby improving the accuracy of the sleep quality assessment results. Furthermore, the multi-channel sensor data can be configured into different data types, fully utilizing the characteristics of each data type to obtain more accurate assessment results.

[0035] Furthermore, this application, based on user sleep data and a preset sleep model, can determine the sleep type corresponding to the sleep data. Users can directly determine their sleep type and sleep problems based on the output of the preset sleep model, meaning the aforementioned sleep type assessment method is intuitive and accurate. The preset sleep model is trained on a sleep data sample set and a deep learning model. The sleep data includes sleep states corresponding to multiple time windows. The deep learning model directly processes the sleep data without manual feature extraction, reducing information loss. Based on a large amount of sleep data, the sleep data sample set is processed according to the data characteristics of the sleep graph, then trained and validated on a large amount of sleep data to obtain the preset sleep model. The preset sleep model has high accuracy and strong generalization ability. This sleep type assessment method is obtained by inputting sleep data into the preset sleep model, resulting in high accuracy and strong generalization ability. It directly derives the sleep type from the sleep data, automatically completing the end-to-end sleep type classification, and is simple to operate. Attached Figure Description

[0036] The features, advantages, and technical effects of exemplary embodiments of this application will now be described with reference to the accompanying drawings.

[0037] Figure 1 is a flowchart of a sleep quality assessment method according to an embodiment of this application.

[0038] Figure 2 is a flowchart of a sleep quality assessment method according to an embodiment of this application.

[0039] Figure 3 is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of this application.

[0040] Figure 4 is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of this application.

[0041] Figure 5 is a schematic diagram of multi-channel sensor data in a sleep interval according to an embodiment of this application.

[0042] Figure 6 is a flowchart of a sleep quality assessment method according to an embodiment of this application.

[0043] Figure 7 is a flowchart of a sleep quality assessment method according to an embodiment of this application.

[0044] Figure 8 is a flowchart of a process for generating sleep data based on sensor data according to an embodiment of this application.

[0045] Figure 9 is a flowchart of a process for generating comprehensive sleep data based on sleep data according to an embodiment of this application.

[0046] Figure 10 is a schematic diagram of multi-channel sleep data within a sleep interval according to an embodiment of this application.

[0047] Figure 11 is a flowchart of the process for determining sleep quality assessment results according to an embodiment of this application.

[0048] Figure 12 is a flowchart of the process for determining sleep quality assessment results according to an embodiment of this application.

[0049] Figure 13 is a flowchart of a sleep quality assessment method according to an embodiment of this application.

[0050] Figure 14 is an application environment diagram of a sleep type assessment method according to an embodiment of this application;

[0051] Figure 15 is a flowchart illustrating a sleep type assessment method according to another embodiment of this application;

[0052] Figure 16 is a schematic diagram of sleep data according to an embodiment of this application;

[0053] Figure 17 is a flowchart illustrating the steps for determining a preset sleep model according to an embodiment of this application;

[0054] Figure 18 shows the results of data clustering according to an embodiment of this application;

[0055] Figure 19 is one of the sleep data after the sleep data in Figure 3 has been randomly segmented, randomly Gaussian blurred and resampled in sequence according to an embodiment of this application;

[0056] Figure 20 is a second type of sleep data after the sleep data in Figure 3 has been randomly segmented, randomly Gaussian blurred, and resampled according to an embodiment of this application.

[0057] Figure 21 is a third type of sleep data after the sleep data in Figure 3 has been randomly segmented, randomly Gaussian blurred, and resampled according to an embodiment of this application.

[0058] Figure 22 is one of the sleep type distribution diagrams according to an embodiment of this application;

[0059] Figure 23 is a second sleep type distribution diagram according to an embodiment of this application;

[0060] Figure 24 is a structural block diagram of a sleep type assessment device according to an embodiment of this application;

[0061] Figure 25 is an internal structural diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0062] The specific embodiments of this application are described below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals are used to denote the same or similar parts or steps, and repeated descriptions are omitted for simplicity.

[0063] Current smartwatches have several shortcomings when used for sleep monitoring. First, they suffer from significant data interference. Worn on the wrist, smartwatches typically rely on ACC (accelerometer) for sleep monitoring. However, arm movements monitored via ACC during sleep do not necessarily indicate a change in sleep state and can introduce noise interference, leading to inaccurate sleep tracking. Second, they suffer from low signal quality. Smartwatches collect PPG (photoplethysmography) data from the wrist. PPG sensor data is highly susceptible to temperature and movement. With frequent wrist movements and changes in wrist fit, plus sweating, the PPG signal quality remains poor.

[0064] Current headphone products monitor sleep by collecting physiological parameters of the ear, such as electrocardiogram (ECG), electroencephalogram (EEG), and respiratory data. However, these products have several problems. First, sleep monitoring data is inaccurate. Some headphone products lack mechanisms to handle situations where the earphones become loose, fall out, or run out of battery during sleep, resulting in incomplete sleep data and inaccurate sleep detection. Second, sleep quality assessment systems are inadequate. Assessing sleep quality requires a combination of sleep data collection, data processing methods, and sleep evaluation methods. Existing sleep headphones only collect sleep-related physiological data and determine sleep stages, failing to provide an effective assessment of sleep quality.

[0065] It is evident that accurately assessing sleep quality is one of the technical problems that this application urgently needs to solve.

[0066] Implementation Method 1:

[0067] Figure 1 is a flowchart of a sleep quality assessment method according to an embodiment of this application. As shown in Figure 1, in step S105, first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device. In step S107, a sleep quality assessment result is determined based on the first comprehensive sleep data.

[0068] The first and second ear-wearing devices can be headphones, hearing aids, or other ear-wearing devices worn on the ear. They can operate wirelessly, such as via Bluetooth, reducing the need for connections and simplifying wear. In one embodiment, the first and second ear-wearing devices are a pair of headphones, one being the left ear and the other the right ear. Using ear-wearing devices such as headphones to collect data has advantages over smartwatches. For example, when using motion data such as acceleration data for sleep monitoring, ear-wearing devices are directly correlated with head movements, directly collecting head movement data. Compared to wearable devices like smartwatches that collect wrist movement data, this reduces motion interference during sleep, resulting in higher-quality data. This is because some limb movements do not reflect key body movements, thus creating interference, while head movements are more likely to reflect key body movements. Moreover, the sensor data or sleep data from the two ear-wearing devices are complementary. If one ear-wearing device fails to collect sensor data or collects inaccurate sensor data due to problems such as poor fit, looseness, falling off, lack of power, or malfunction, the other ear-wearing device can collect sensor data normally, thereby supplementing the missing or inaccurate sensor data or sleep data.

[0069] In one embodiment, the first and second ear-wearing devices are a pair of custom-made earphones. Non-custom-made earphones, especially during sleep, are prone to loosening, falling out, or not fitting snugly, leading to incomplete and inaccurate sleep data, and consequently, inaccurate sleep quality assessment results. In this embodiment, using custom-made earphones as ear-wearing devices for sleep quality assessment ensures a good fit, secure wearing, and prevents them from falling out, improving the completeness and accuracy of sleep data detection, thus resulting in more accurate sleep quality assessment results. Furthermore, compared to non-custom-made earphones, custom-made earphones are tailored to the user's ear structure, offering greater wearing comfort and being more suitable for extended wear, thereby better meeting the requirements of sleep quality monitoring. In one embodiment, the first and second ear-wearing devices are a pair of wireless custom-made earphones.

[0070] The first and second sensor data can be data collected by sensors in the first and second ear-wearing devices, respectively. These sensors can be PPG sensors, ACC sensors, ECG (electrocardiogram) sensors, EEG (electroencephalogram) sensors, temperature sensors, sound sensors (such as microphones), etc. Those skilled in the art will understand that other types of sensors can also be used, as long as they can collect sleep-related data. The sensor data mentioned in this specification, including the data from the first to fourth sensors, can be data directly collected by the sensors, or data obtained after processing the data collected by the sensors; this application does not impose any limitations on this. Data collected by the PPG sensor can be referred to as PPG sensor data or PPG data, and the same applies to other types of data.

[0071] In one embodiment, the first sensor data and the second sensor data are first-type data, which are obtained, for example, by an accelerometer (i.e., ACC data) or by a photoplethysmography (PPG) sensor (i.e., PPG data). In other words, the first sensor data and the second sensor data can be the same type of data, such as both being ACC data, or both being PPG data or other types of data. In another embodiment, the first sensor data and the second sensor data can be different types of data, for example, the first sensor data is first-type data, such as ACC data, and the second sensor data is second-type data, such as PPG data.

[0072] The first comprehensive sleep data indicates the sleep state within each time window of a sleep interval. The first comprehensive sleep data, and other sleep data mentioned later, can be a sleep graph, i.e., a graph that includes both time and sleep state information, indicating the user's sleep state at different times. A sleep interval can be the time range from the start to the end of sleep for a user wearing an ear-worn device, such as 10:00 PM to 7:00 AM the next day. The start and end times of this sleep interval can be set by the user or automatically determined by sensor data and / or other data. A time window can be the temporal granularity of processing sensor data; for example, each time window can be 30 seconds or other durations, and the time windows typically do not overlap.

[0073] Sleep states can be selected from at least one of the following groups or from at least one of the following groups after adding the "uncertain" state: (1) wakefulness and sleep; (2) wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3; (3) wakefulness, REM sleep, light sleep, and deep sleep; or (4) wakefulness, REM sleep, and non-REM sleep. Four groups of sleep states are listed here as examples. Those skilled in the art will understand that these sleep states can be those currently used in the field of sleep monitoring, and other sleep state groupings can also be used. For a certain type of sensor data, a fixed sleep state grouping can be used to facilitate calculation.

[0074] In one embodiment, the first type of sleep state grouping can be used for ACC data. For example, when the first sensor data and the second sensor data are ACC data, the sleep state of the first comprehensive sleep data in each time window can be awake, asleep, or uncertain. In one embodiment, the second type of sleep state grouping can be used for PPG data. For example, when the first sensor data and the second sensor data are PPG data, the sleep state of the first comprehensive sleep data in each time window can be awake, REM sleep, sleep stage N1, sleep stage N2, sleep stage N3, or uncertain. PPG data can also be grouped using the third or fourth type of sleep state grouping. Other types of data can be grouped using one of the various sleep state groupings mentioned above. The choice of which sleep state grouping to use can be automatically determined based on the sensor data type, can be selected by the user, or can be selected by comparing the accuracy of the calculation results.

[0075] The processes in steps S105 and S107 will be described in detail below with reference to the accompanying drawings. In step S105, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. The first comprehensive sleep data in step S105 can be generated directly from the first sensor data and the second sensor data (see Figures 3 and 4 for details), or it can be generated by first generating the first sleep data and the second sleep data from the first sensor data and the second sensor data respectively, and then generating the data based on the first sleep data and the second sleep data (see the embodiments in Figures 6 and 7).

[0076] In step S107, a sleep quality assessment result is determined based on the first comprehensive sleep data. The sleep quality assessment result may include, for example, difficulty falling asleep, normal sleep, insomnia, or shallow sleep. In one embodiment, the sleep quality judgment criteria can adopt existing standards, such as the total duration of each sleep state, or whether each sleep state matches or is similar to a predetermined curve of each assessment result. In other embodiments, different sleep quality judgment criteria can be used for different users to consider individual differences in sleep quality requirements. For example, in one embodiment, sleep quality judgment criteria can be formed by having users confirm their sleep effectiveness after each sleep session and accumulating a certain amount of data. In another embodiment, sleep quality judgment criteria can be formed by monitoring the user's activity intensity after each sleep session.

[0077] The sleep quality assessment method shown in Figure 1 can be executed on an ear-worn device, or partly on the ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S105 and S107 are both executed on the first or second ear-worn device, wherein first sensor data is sent from the first ear-worn device to the second ear-worn device, or second sensor data is sent from the second ear-worn device to the first ear-worn device. In one embodiment, steps S105 and S107 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, step S105 is executed on a control module connected to the first ear-worn device, the second ear-worn device, or both, and step S107 is executed on a terminal device such as a mobile phone.

[0078] Sleep quality assessment results can be played to the user via voice on the first and / or second ear-worn device, or displayed on a mobile phone or other terminal device, allowing the user to easily understand their sleep quality. When using the display method, comparison charts or curves can be shown daily, weekly, or monthly to help users understand the comparison of their sleep over different periods.

[0079] In the embodiment shown in Figure 1, sleep quality is assessed based on two-channel sensor data (i.e., first and second sensor data) from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other, thereby improving the accuracy of the sleep quality assessment results.

[0080] Figure 2 is a flowchart of a sleep quality assessment method according to an embodiment of this application. As shown in Figure 2, in step S205, first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device. In step S206, second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device. In step S207, a sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data.

[0081] The first and second ear-wearing devices can be the same as those in the embodiment of Figure 1, and will not be described again here. In one embodiment, the first and second sensor data are first-type data, and the third and fourth sensor data are second-type data, different from the first-type data. For example, the first-type data is obtained through an accelerometer (i.e., ACC data), and the second-type data is obtained through a photoplethysmography (PPG) sensor (i.e., PPG data); or, the first-type data is obtained through a PPG sensor (i.e., PPG data), and the second-type data is obtained through an accelerometer (i.e., ACC data). In the embodiment shown in Figure 2, different types of sensor data can be used, such as ACC data and PPG data. The ACC sensor can monitor key body movement information during sleep, especially in the ear, reflecting head movement information, which is not affected by limb movement; the PPG sensor can monitor core physiological data of sleep, reflecting cardiac activity and the state of the cardiovascular system, which is closely related to the sleep state. That is, ACC data is body movement data, and PPG data is physiological data. Combining body movement data and physiological data for sleep quality assessment can effectively improve the accuracy of sleep quality assessment results.

[0082] The first and second comprehensive sleep data each indicate the sleep state within a specific time window of the sleep interval. The first and second comprehensive sleep data can be a sleep graph, i.e., a graph that includes both time and sleep state information, indicating the user's sleep state at different times. The meanings and settings of sleep intervals, time windows, and sleep states can be the same as in the embodiment shown in Figure 1, and will not be repeated here.

[0083] The processes in steps S205, S206, and S207 will be described in detail below with reference to the accompanying drawings. In step S205, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. In step S206, second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device. The first comprehensive sleep data in step S205 can be generated directly from the first and second sensor data (see Figures 3 and 4 for details), or it can be generated by first generating the first sleep data and the second sleep data from the first and second sensor data respectively, and then generating the second comprehensive sleep data based on the first and second sleep data (see the embodiments in Figures 6 and 7). Similarly, the second comprehensive sleep data in step S206 can be generated directly from the third and fourth sensor data (see Figures 3 and 4 for details), or it can be generated by first generating the third sleep data and the fourth sleep data from the third and fourth sensor data respectively, and then generating the second comprehensive sleep data based on the third and fourth sleep data (see the embodiment in Figure 7).

[0084] In the embodiment shown in Figure 2, the execution order of steps S205 and S206 is not explicitly limited. Step S205 may be executed before step S206, may be executed simultaneously with or at least partially overlap with step S206, or may be executed after step S206. Of course, step S207 is executed after steps S205 and S206.

[0085] In step S207, the sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data. The sleep quality assessment result and sleep quality judgment criteria in step S207 of Figure 2 can be implemented in the same way as in the embodiment of Figure 1.

[0086] Similar to the embodiment in Figure 1, the sleep quality assessment method shown in Figure 2 can be executed on an ear-worn device, or partly on the ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S205, S206, and S207 are all executed on the first or second ear-worn device, wherein the first and third sensor data are sent from the first ear-worn device to the second ear-worn device, or the second and fourth sensor data are sent from the second ear-worn device to the first ear-worn device. In one embodiment, step S205 is executed on the first ear-worn device, step S206 is executed on the second ear-worn device, and step S207 is executed on either the first or second ear-worn device. In one embodiment, step S205 is executed on the first ear-worn device, step S206 is executed on the second ear-worn device, and step S207 is executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, steps S205 and S206 are executed on the control module connected to the first ear-wearing device, the second ear-wearing device, or both, while step S207 is executed on a terminal device such as a mobile phone.

[0087] In the embodiment of Figure 2, the sleep quality assessment results can be played or displayed in the same manner as in the embodiment of Figure 1.

[0088] In the embodiment shown in Figure 2, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared to two-channel sensor data, in addition to the two ear-worn devices complementing each other's sensor data, using four-channel sensor data can further improve the accuracy of sleep quality assessment results. This is because four-channel sensor data is richer in information than two-channel sensor data, and more accurate assessment results can be obtained based on data with richer information. Moreover, the four-channel sensor data can be set to different types of data, such as including first type data (e.g., ACC data) and second type data (e.g., PPG data), which can fully utilize the characteristics of these different types of data to obtain more accurate assessment results.

[0089] Figure 3 is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of this application. As shown in Figure 3, the process for generating comprehensive sleep data based on sensor data includes: in step S301, determining a first ear-worn device as the main analysis device; in step S302, extracting second sensor data for time periods in the sleep interval where the first sensor data is missing; and in step S303, generating first comprehensive sleep data based on the first sensor data and the extracted second sensor data.

[0090] In step S301, the first ear-wearing device is determined to be the primary analysis device. The first ear-wearing device is determined to be the primary analysis device in the following situations: the time period during which the first sensor data is missing in the sleep interval is shorter than the time period during which the second sensor data is missing in the same sleep interval; in this case, the normal monitoring time of the first ear-wearing device is longer than that of the second ear-wearing device; or, the average signal-to-noise ratio of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device; in this case, the overall data quality of the first ear-wearing device is higher than that of the second ear-wearing device.

[0091] In step S302, for time periods within the sleep interval where the first sensor data is missing (i.e., the first sensor data is absent or empty during this time period), second sensor data is extracted. In step S303, first comprehensive sleep data is generated based on the first sensor data and the extracted second sensor data. In the embodiment of Figure 3, the first ear-wearing device is determined as the primary analysis device, meaning its corresponding first sensor data is determined as the primary analysis data. Second sensor data from the secondary analysis device (the second ear-wearing device) is used as supplementary data to fill in the time periods where the primary analysis data is missing. This approach reduces the number of time periods with missing data within the sleep interval; however, time periods where both first and second sensor data are missing are still considered data-missing. The process of generating the first comprehensive sleep data based on the first sensor data and the extracted second sensor data in step S303 is similar to the process of generating sleep data based on sensor data in Figure 8, and will be explained in detail later with reference to Figure 8.

[0092] In one embodiment, the process of generating first comprehensive sleep data based on first and second sensor data may include the following step instead of steps S302 and S303: generating first comprehensive sleep data based on first sensor data. In this embodiment, for example, because the effective monitoring time of the first ear wearable device is longer than that of the second ear wearable device (i.e., the time period in the sleep interval where the first sensor data is missing is shorter than the time period in the sleep interval where the second sensor data is missing), the first ear wearable device is determined as the main analysis device. In this case, the first sensor data from the first ear wearable device can be directly used to generate the first comprehensive sleep data, which can reduce the amount of computation. Especially when the effective monitoring time of the first ear wearable device is significantly longer than that of the second ear wearable device, the supplementary role of the second ear wearable device is limited. In this case, directly generating the first comprehensive sleep data based on the first sensor data from the first ear wearable device, which is the main analysis device, can reduce the computation of the second sensor data and the comparison of the first and second sensor data, thereby reducing the amount of computation. For example, if both the first sensor data and the second sensor data are ACC data, and the time period during which the first ear wearable device lacks the first sensor data in the sleep interval is shorter than the time period during which the second ear wearable device lacks the second sensor data in the same sleep interval, then the first comprehensive sleep data can be determined directly based on the ACC data of the first ear wearable device, which can reduce the amount of computation.

[0093] The process in Figure 3 can be applied not only to step S105 in Figure 1 or step S205 in Figure 2, but also to step S206 in Figure 2, simply by replacing the first sensor data, second sensor data, and first comprehensive sleep data with the third sensor data, fourth sensor data, and second comprehensive sleep data. When applied to step S206 in Figure 2, the primary analysis device determined in step S301 could be either the first or second ear-wearing device, depending primarily on the comparison results of the third and fourth sensor data. The comparison method for the third and fourth sensor data can be the same as that for the first and second sensor data. For example, the first ear-wearing device can be determined as the primary analysis device if: the time period during which the third sensor data is missing in the sleep interval is shorter than the time period during which the fourth sensor data is missing in the same sleep interval; or if the average signal-to-noise ratio of the third sensor data is higher than that of the fourth sensor data. Otherwise, the second ear-wearing device is determined as the primary analysis device. It is understood that different ear-worn devices may be used as the primary analysis device for generating the first and second comprehensive sleep data. This applies to four-channel or more channel data, especially when using four-channel or more channel data of different data types. For example, if the first and second sensor data are PPG data, and the third and fourth sensor data are ECG data, and the average signal-to-noise ratio (SNR) of the first sensor data is higher than that of the second sensor data, then when determining the first comprehensive sleep data, the first ear-worn device is designated as the primary analysis device, and the first sensor data is designated as the primary analysis data. Similarly, if the average SNR of the fourth sensor data is higher than that of the third sensor data, then when determining the second comprehensive sleep data, the second ear-worn device is designated as the primary analysis device, and the fourth sensor data is designated as the primary analysis data.

[0094] In one embodiment, the same ear-worn device can be used as the primary analysis device for generating the first and second comprehensive sleep data. For example, if the time period in which data from both the first and third sensors are missing simultaneously within a sleep interval is shorter than the time period in which data from both the second and fourth sensors are missing simultaneously within the same sleep interval, or if the combined average signal-to-noise ratio (SNR) of the first and third sensor data is higher than the combined average SNR of the second and fourth sensor data, then the first ear-worn device is determined as the primary analysis device, and the first and third sensor data are determined as the primary analysis data. In this case, the second ear-worn device serves as the secondary analysis device, and the second and fourth sensor data serve as the secondary analysis data.

[0095] For specific data types, such as ACC data, for time periods within the sleep interval where primary analysis data (e.g., first sensor data from the first ear-wearing device) exists, it is directly used as the sensor data for that time period, without considering secondary analysis data. This reduces computational load compared to considering both primary and secondary analysis data simultaneously within that time period. For time periods within the sleep interval where primary analysis data is missing, secondary analysis data (e.g., second sensor data from the second ear-wearing device) is used as the sensor data for that time period. Because this portion of secondary analysis data is a truncated version of the secondary analysis data only during the time periods where primary analysis data is missing, it is called "truncated secondary analysis data," and when the secondary analysis data is second sensor data, it is called "truncated second sensor data." In the embodiment of Figure 3, generating comprehensive sleep data (e.g., first comprehensive sleep data, or second comprehensive sleep data) directly from sensor data (e.g., first and second sensor data, or third and fourth sensor data) simplifies processing and reduces computational load compared to generating sleep data from sensor data first and then generating comprehensive sleep data from sleep data (as in the embodiments of Figures 6 and 7).

[0096] In the embodiment shown in Figure 3, comprehensive sleep data is generated based on two-channel sensor data from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other to improve the completeness of the comprehensive sleep data, thereby providing more complete data for subsequent sleep quality assessment.

[0097] Figure 4 is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of this application. As shown in Figure 4, the process for generating comprehensive sleep data based on sensor data includes: in step S401, determining a first ear-worn device as the main analysis device; in step S402, determining whether first sensor data exists in a time period within a sleep interval; if the determination result of step S402 is "no", in step S403, using second sensor data to generate first comprehensive sleep data in that time period; if the determination result of step S402 is "yes", in step S404, determining whether second sensor data exists in that time period; if the determination result of step S404 is "no", in step S405, using first sensor data to generate first comprehensive sleep data in that time period; if the determination result of step S404 is "yes", in step S406, dividing the time period into first sensor data and second sensor data according to a time window; in step S407, determining whether the feature value of the first sensor data in a time window is higher than that of the second sensor data; if the determination result of step S407 is "yes",... In step S408, the first comprehensive sleep data is determined using the sleep data corresponding to the first sensor data in the time window. If the judgment result in step S407 is "no", in step S409, the first comprehensive sleep data is determined using the sleep data corresponding to the second sensor data in the time window. After steps S408 and S409, in step S410, it is determined whether all time windows within the time period have been processed. If the judgment result in step S410 is "no" (i.e., all time windows within the time period have not been processed), the process returns to continue executing step S407. If the judgment result in step S410 is "yes" (i.e., all time windows within the time period have been processed), or after steps S403 and S405, in step S411, it is determined whether all time periods have been processed. If the judgment result in step S411 is "no" (i.e., all time periods have not been processed), the process returns to continue executing step S402. If the judgment result in step S411 is "yes" (i.e., all time periods have been processed), the process ends.

[0098] The criteria for determining the main analysis device in step S401 of Figure 4 can be the same as those in step S301 of Figure 3. Next, the time periods within the sleep interval are processed. These time periods can include periods with no data (neither first nor second sensor data exists), single-data periods with only one type of data (either only first or only second sensor data exists), and dual-data periods with both types of data (both first and second sensor data exist). In specific processing, these time periods can be merged based on the data used as the basis for analysis (such as the main analysis data). In the embodiment of Figure 4, the first sensor data, which serves as the main analysis data, is used as the basis for processing to simplify the process and reduce computation. For example, for time periods without first sensor data, second sensor data is used to generate the first comprehensive sleep data, regardless of whether second sensor data exists in that time period. This merges periods with no data and periods with only second sensor data into one case, simplifying the process and reducing computation. Of course, for time periods without second sensor data (i.e., periods with no data), they can be simply set to empty data without time window division or other processing to simplify the process.

[0099] In one embodiment, step S401, where the master analysis device is not determined, can be omitted. Instead, either the first or second sensor data can be used as the basis for subsequent processing steps. This simplifies the process, and compared to determining the master analysis device, this embodiment considers sensor data across all time periods and time windows, thus ensuring equally satisfactory processing results.

[0100] In step S402, it is determined whether first sensor data exists within a time period of the sleep interval. This determination can be performed sequentially for each time period within the sleep interval, either chronologically or by the type of time period. When the first ear-worn device is designated as the primary analysis device, the first sensor data, as the primary analysis data, forms the basis of the analysis. Therefore, the sleep interval can be divided into time periods without first sensor data (or periods with no first sensor data), time periods containing only first sensor data, and time periods where both first and second sensor data are present. The data can then be processed sequentially, either chronologically or according to the type of time period.

[0101] If it is determined in step S402 that no first sensor data exists within a time period, then step S403 is executed, that is, first comprehensive sleep data is generated using second sensor data within that time period. If it is determined in step S402 that first sensor data exists within a time period, then step S404 is executed to further determine whether second sensor data exists within that time period. If no second sensor data exists within that time period, that is, only first sensor data exists, then step S405 is executed, that is, first comprehensive sleep data is generated using first sensor data within that time period. The process of generating first comprehensive sleep data using second sensor data in step S403 and the process of generating first comprehensive sleep data using first sensor data in step S405 are similar to the process in Figure 8, and will be explained in detail later with reference to Figure 8.

[0102] If it is determined in step S404 that second sensor data exists within the time period, and combined with the determination in step S402, it is known that both first and second sensor data exist within this time period. In this case, step S406 and subsequent steps are executed. For time periods containing both first and second sensor data, the data quality of the two is compared, and the data with higher quality is selected as the basis for further processing. That is, the comprehensive sleep data is generated using the data with higher quality. The comparison of the first and second sensor data is usually based on time windows. That is, after dividing the first and second sensor data into time windows (e.g., 30 seconds), the data quality of the first and second sensor data is judged for each time window. This allows full utilization of the high-quality portions of the first and second sensor data. For example, in some time windows, the first sensor data has higher quality, while in other time windows, the second sensor data has higher quality. By comparing the first and second sensor data by time windows and selecting the sensor data with higher quality based on the comparison results, the high-quality portions of the first and second sensor data are combined, thereby improving the data quality of the generated first comprehensive sleep data. Data quality can be compared using features such as signal-to-noise ratio (SNR), with higher SNR indicating higher data quality. Alternatively, other features or comparison standards besides SNR can be used.

[0103] In step S406, the first sensor data and the second sensor data within the time period are divided according to time windows. The time window here is similar to other embodiments and can be the time granularity for processing the sensor data, such as 30 seconds, which will not be elaborated further here. After dividing the first and second sensor data according to time windows, step S407 and subsequent steps are executed, that is, determining which sensor data to use to generate the first comprehensive sleep data for each time window.

[0104] In step S407, it is determined whether the feature value of the first sensor data in a time window is higher than that of the second sensor data. If the feature value of the first sensor data in a time window is higher than that of the second sensor data in the same time window, the sleep data corresponding to the first sensor data in that time window is used to determine the first comprehensive sleep data (step S408). If the feature value of the first sensor data in a time window is lower than that of the second sensor data in the same time window, the sleep data corresponding to the second sensor data in that time window is used to determine the first comprehensive sleep data (step S409). The determination of the sleep data in the time window from the first sensor data in step S408 and the determination of the sleep data in the time window from the second sensor data in step S409 can be performed in the same way as step S804 in Figure 8, and will not be described in detail here.

[0105] After steps S408 and S409, in step S410, it is determined whether all time windows within the current time period have been processed. If it is determined that not all time windows within the current time period have been processed, the process returns to step S407 to continue processing the remaining time windows. If it is determined that all time windows within the current time period have been processed, the process continues to step S411.

[0106] If step S410 determines that all time windows within the current time period have been processed, or if, after steps S403 and S405, step S411 determines whether all time periods have been processed, and if it is determined that not all time periods have been processed, then the process returns to step S402 to continue processing the remaining time periods. If it is determined that all time periods have been processed, then the process ends.

[0107] The process in Figure 4 can be applied not only to step S105 in Figure 1 or step S205 in Figure 2, but also to step S206 in Figure 2, simply by replacing the first sensor data, second sensor data, and first comprehensive sleep data with third sensor data, fourth sensor data, and second comprehensive sleep data. In Figure 4, before dividing the first and second sensor data in step S406 (e.g., after step S404) or before calculating the feature values ​​of the first and second sensor data in step S407, the first and second sensor data can be filtered or otherwise processed. The specific filtering method can be the same as in the embodiment of Figure 8.

[0108] For specific data types, such as PPG or ECG data, by comparing the feature values ​​of multi-channel sensor data from two ear-worn devices and selecting the portion of sensor data with higher feature values, the high-quality portion of the multi-channel sensor data can be fully utilized, thereby improving the data quality of the generated comprehensive sleep data. Furthermore, in the embodiment of Figure 4, directly generating comprehensive sleep data (e.g., first comprehensive sleep data or second comprehensive sleep data) from sensor data (e.g., first and second sensor data, or third and fourth sensor data) simplifies the process and reduces computational load compared to generating sleep data from sensor data first and then generating comprehensive sleep data from sleep data (as in the embodiments of Figures 6 and 7).

[0109] Figure 5 is a schematic diagram of multi-channel sensor data within a sleep interval according to an embodiment of this application. In the following description, first and second sensor data are used as examples of multi-channel sensor data. Those skilled in the art will understand that third and fourth sensor data are also applicable in other embodiments. For clarity and simplicity, the term "time period" is sometimes omitted and directly referred to by letters.

[0110] Figure 5 illustrates the data from the first and second sensors within the sleep interval T. As shown in Figure 5, for the first sensor data, there is a data-containing time period T1 (T1 = A + B + C) and a data-free time period T2 (T2 = D + E) within the sleep interval T; for the second sensor data, there is a data-containing time period T1' (T1' = A), a data-free time period T2' (T2' = B), a data-containing time period T3' (T3' = C + D), and a data-free time period T4' (= E) within the sleep interval T. Although Figure 5 shows that both the first and second sensor data have data-free time periods within the sleep interval T, those skilled in the art will understand that in other embodiments, the first and / or second sensor data may not have data-free time periods within the sleep interval T. The data-containing time periods shown as rectangles in Figure 5 only indicate that data was present within that time period, and do not indicate that the data values ​​are all the same; sensor data can have different values ​​at different times. Similarly, the two rectangles for the second sensor data are only used to illustrate that they are different time periods, and do not indicate whether the data values ​​are higher or lower.

[0111] Based on the simultaneous presence of data from the first and second sensors, the sleep interval T contains the following time periods: time period E, a period with no data where neither type of data exists; time periods B and D, single-data periods where only one type of data exists; and time periods A and C, dual-data periods where both types of data exist. The single-data periods containing only one type of data include time periods containing only the first sensor data (time period B) and time periods containing only the second sensor data (time period D). In specific processing, some of these time periods can be merged based on the data used as the basis for analysis.

[0112] The sensor data in Figure 5 can be applied to the embodiments in Figure 3 or Figure 4. In the case of the embodiment in Figure 3, the first sensor data is missing in time period T2(D+E) within the sleep interval T. Therefore, in step S302, the second sensor data is directly used as the extracted second sensor data for this time period, i.e., the second sensor data in time period D and the empty data (i.e., no data) in time period E. Then, in step S303, the first comprehensive sleep data is generated based on the first sensor data in time period T1(A+B+C), the second sensor data in time period D, and the empty data in time period E.

[0113] When applied to the embodiment shown in Figure 4, the time periods within the sleep interval can be processed sequentially according to the order of the time periods, or according to the order of first processing the data-free time period of the main analysis data (T2 = D + E), then processing the data-single time period of the main analysis data (B), and finally processing the data-double time periods (A and C). The following descriptions of each time period are provided separately, but their order is not limited; the order described in this paragraph or other orders may be used.

[0114] Time period A contains both first sensor data and second sensor data. Therefore, for time period A, it is determined in step S402 that first sensor data exists and in step S404 that second sensor data exists. Then, step S406 is executed to divide the first sensor data and second sensor data in this time period into time windows. Then, for the divided first and second sensor data, the feature values ​​of the two are compared one time window at a time. When it is determined in step S407 that the feature value of the first sensor data in a time window (time window W1 as shown in Figure 5) is higher than the feature value of the second sensor data in the same time window, step S408 is executed to use the sleep data corresponding to the first sensor data in that time window to determine the first comprehensive sleep data. When it is determined in step S407 that the feature value of the first sensor data in a time window (time window W1 as shown in Figure 5) is lower than the feature value of the second sensor data in that time window, step S409 is executed to use the sleep data corresponding to the second sensor data in that time window to determine the first comprehensive sleep data. For example, assuming that in time window W1 shown in Figure 5, the feature value of the first sensor data is lower than the feature value of the second sensor data in that time window, then the second sensor data is used in time window W1 to determine the value of the first comprehensive sleep data in that time window, that is, the sleep state in that time window.

[0115] In time period B, only the first sensor data exists. Therefore, for time period B, it is determined in step S402 that the first sensor data exists and in step S404 that the second sensor data does not exist. Then, the process proceeds to step S405, where the first sensor data is used to generate the first comprehensive sleep data in this time period.

[0116] Time period C is similar to time period A, containing both first and second sensor data; therefore, its processing is similar to that of time period A. Figure 5 shows time window W2. In time window W2, sensor data with higher eigenvalues ​​is used to determine the first comprehensive sleep data. For example, if the eigenvalue of the first sensor data in time window W2 is higher than the eigenvalue of the second sensor data in that time window, then the first sensor data is used in time window W2 to determine the value of the first comprehensive sleep data in that time window, i.e., the sleep state in that time window.

[0117] Since there is no first sensor data in time period T2, in step S402 it is determined that there is no first sensor data in time period T2, and then the process proceeds to step S403, where the first comprehensive sleep data is generated using the second sensor data (including the second sensor data in time period D and the empty data in time period E) during this time period. For example, for time period D, the first comprehensive sleep data can be generated using the second sensor data through the process shown in Figure 8, and for time period E, the value of the first comprehensive sleep data is set to empty.

[0118] Figure 6 is a flowchart of a sleep quality assessment method according to an embodiment of this application. As shown in Figure 6, in step S601, first sleep data is generated based on first sensor data from a first ear-wearing device. In step S603, second sleep data is generated based on second sensor data from a second ear-wearing device. In step S605, first comprehensive sleep data is generated based on the first sleep data and the second sleep data. In step S607, a sleep quality assessment result is determined based on the first comprehensive sleep data.

[0119] Sensor data from the ear-worn device is processed to generate sleep data. In the embodiment shown in Figure 6, the first sleep data is generated based on the first sensor data, the second sleep data is generated based on the second sensor data, and the first and second sleep data are combined to generate the first comprehensive sleep data. The first sleep data, the second sleep data, and the first comprehensive sleep data each indicate the sleep state at each time window within a sleep interval. The first sleep data, the second sleep data, and the first comprehensive sleep data can be a sleep graph, i.e., a graph that includes both time and sleep state information, which can indicate the user's sleep state at different times. A sleep interval can be the time interval from when the user wearing the ear-worn device begins to fall asleep to when they end to fall asleep, for example, from 22:00 to 7:00 the next day. The start and end times of this sleep interval can be set by the user or automatically determined by sensor data and / or other data. A time window can be the time granularity for processing the sensor data; for example, each time window can be 30 seconds or other durations, and the time windows typically do not overlap.

[0120] Sleep states can be selected from at least one of the following groups or from at least one of the following groups after adding the "uncertain" state: (1) wakefulness and sleep; (2) wakefulness, REM sleep, sleep stage N1, sleep stage N2 and sleep stage N3; (3) wakefulness, REM sleep, light sleep and deep sleep; or (4) wakefulness, REM sleep and non-REM sleep. Four groups of sleep states are listed here by way of example. Those skilled in the art will understand that these sleep states can be the sleep states currently used in the field of sleep monitoring, and other sleep state groupings can also be used. In one embodiment, the first sleep data, the second sleep data and the first comprehensive sleep data can be grouped using the same sleep state, such as one of the groups (1) to (4) above. In this case, the sleep stage method of the first sleep data, the second sleep data and the first comprehensive sleep data is the same, which is convenient for calculation and comparison. Different sleep state groupings can be used for different types of sensor data. In one embodiment, the first type of sleep state grouping can be used for ACC data. For example, when the first sensor data and the second sensor data are ACC data, the sleep states of the first sleep data, the second sleep data, and the first comprehensive sleep data in each time window can be awake, asleep, or uncertain. In one embodiment, the PPG data can be grouped into a second type of sleep state. For example, when the first sensor data and the second sensor data are PPG data, the sleep states of the first sleep data, the second sleep data, and the first comprehensive sleep data in each time window can be awake, REM sleep, sleep stage N1, sleep stage N2, sleep stage N3, or uncertain.

[0121] The processing of steps S601, S603, S605, and S607 will be described in detail below with reference to the accompanying drawings. In the embodiment of FIG6, the execution order of steps S601 and S603 is not explicitly limited. Considering that the first sensor data and the second sensor data come from different ear-wearing devices, the order of steps S601 and S603 can be set based on their generation order or the convenience of data processing. Step S601 can be executed before step S603, can be executed simultaneously with or at least partially overlap with step S603, or can be executed after step S603. Step S605 is executed after steps S601 and S603, and step S607 is executed after step S605. In step S607, a sleep quality assessment result is determined based on the first comprehensive sleep data. The sleep quality assessment result is, for example, difficulty falling asleep, normal sleep, insomnia, light sleep, etc. The judgment criteria for sleep quality can be similar to those in the embodiment of FIG1.

[0122] The sleep quality assessment method shown in Figure 6 can be executed on an ear-worn device, or partly on an ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S601, S603, S605, and S607 are all executed on the first or second ear-worn device, wherein first sensor data is sent from the first ear-worn device to the second ear-worn device, or second sensor data is sent from the second ear-worn device to the first ear-worn device. In one embodiment, step S601 is executed on the first ear-worn device, step S603 is executed on the second ear-worn device, and steps S605 and S607 are executed on either the first or second ear-worn device. In one embodiment, step S601 is executed on the first ear-worn device, step S603 is executed on the second ear-worn device, and steps S605 and S607 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, step S601 is executed on the first ear-wearing device, step S603 is executed on the second ear-wearing device, step S605 is executed on the control module connected to the first ear-wearing device, the second ear-wearing device, or both, and step S607 is executed on a terminal device such as a mobile phone.

[0123] In the embodiment shown in Figure 6, the sleep quality assessment results can be played or displayed in the same manner as in the embodiment shown in Figure 1.

[0124] In the embodiment shown in Figure 6, sleep quality assessment is performed based on two-channel sensor data (i.e., first and second sensor data) from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other, thereby improving the accuracy of the sleep quality assessment results.

[0125] Figure 7 is a flowchart of a sleep quality assessment method according to an embodiment of this application. As shown in Figure 7, in step S701, first sleep data is generated based on first sensor data from a first ear-wearing device. In step S703, second sleep data is generated based on second sensor data from a second ear-wearing device. In step S702, third sleep data is generated based on third sensor data from the first ear-wearing device. In step S704, fourth sleep data is generated based on fourth sensor data from the second ear-wearing device. In step S705, first comprehensive sleep data is generated based on the first sleep data and the second sleep data. In step S706, second comprehensive sleep data is generated based on the third sleep data and the fourth sleep data. In step S707, a sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data.

[0126] In one embodiment, the first sensor data and the second sensor data are first-type data, and / or, the third sensor data and the fourth sensor data are second-type data, different from the first-type data. For example, the first-type data is obtained through an accelerometer (i.e., ACC data), and the second-type data is obtained through a photoplethysmography (PPG) sensor (i.e., PPG data); or, the first-type data is obtained through a PPG sensor (i.e., PPG data), and the second-type data is obtained through an accelerometer (i.e., ACC data). In the embodiment shown in Figure 7, different types of sensor data can be used, such as ACC data and PPG data. The ACC sensor can monitor key body movement information during sleep, especially in the ear, reflecting head movement information, unaffected by limb movements; the PPG sensor can monitor core physiological data of sleep, reflecting cardiac activity and the state of the cardiovascular system, which is closely related to the sleep state. That is, ACC data is body movement data, and PPG data is physiological data. Combining body movement data and physiological data for sleep quality assessment can effectively improve the accuracy of sleep quality assessment results.

[0127] Sensor data from the wearable ear device is processed to generate sleep data. In the embodiment shown in Figure 7, the first sleep data is generated based on the first sensor data, the second sleep data is generated based on the second sensor data, and the first and second sleep data are combined to generate the first comprehensive sleep data; the third sleep data is generated based on the third sensor data, the fourth sleep data is generated based on the fourth sensor data, and the third and fourth sleep data are combined to generate the second comprehensive sleep data. The first, second, third, and fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state at each time window within a sleep interval. The first, second, third, and fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data can be a sleep graph, i.e., a graph that includes both time and sleep state information, which can indicate the user's sleep state at different times. The meaning and setting method of sleep intervals, time windows, and sleep states can be the same as in the embodiment of Figure 1, and will not be repeated here. In one embodiment, the first sleep data, the second sleep data, and the first comprehensive sleep data can be grouped using the same sleep state. In this case, the sleep staging methods for the first sleep data, the second sleep data, and the first comprehensive sleep data are the same, which facilitates calculation and comparison. Similarly, the third sleep data, the fourth sleep data, and the second comprehensive sleep data can be grouped using the same sleep state. In this case, the sleep staging methods for the third sleep data, the fourth sleep data, and the second comprehensive sleep data are the same, which facilitates calculation and comparison.

[0128] Similar to the embodiment in Figure 6, the execution order of steps S701, S702, S703, and S704 is not explicitly limited in the embodiment in Figure 7. Step S701 can be executed before step S703, can be executed simultaneously with or at least partially overlap with step S703, or can be executed after step S703; step S702 can be executed before step S704, can be executed simultaneously with or at least partially overlap with step S704, or can be executed after step S704. The order of steps S701 with steps S702 and S704, and step S702 with steps S701 and S703, is also not explicitly limited. The execution order of steps S705 and S706 is also not explicitly limited. Of course, step S705 is executed after steps S701 and S703, step S706 is executed after steps S702 and S704, and step S707 is executed after steps S705 and S706. The sleep quality assessment results in step S707 of Figure 7 can be obtained in the same way as those in Figure 1, Figure 2, or Figure 6.

[0129] Similar to the embodiment in Figure 6, the sleep quality assessment method shown in Figure 7 can be executed on an ear-worn device, or partly on an ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S701 to S707 are all executed on the first or second ear-worn device, wherein the first and third sensor data are sent from the first ear-worn device to the second ear-worn device, or the second and fourth sensor data are sent from the second ear-worn device to the first ear-worn device. In one embodiment, steps S701 and S702 are executed on the first ear-worn device, steps S703 and S704 are executed on the second ear-worn device, and steps S705, S706, and S707 are executed on the first or second ear-worn device. In one embodiment, steps S701 and S702 are executed on the first ear-worn device, steps S703 and S704 are executed on the second ear-worn device, and steps S705, S706, and S707 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, steps S701 and S702 are executed on the first ear-wearing device, steps S703 and S704 are executed on the second ear-wearing device, steps S705 and S706 are executed on the control module connected to the first ear-wearing device, the second ear-wearing device, or both, and step S707 is executed on a terminal device such as a mobile phone.

[0130] In the embodiment of Figure 7, the sleep quality assessment results can be played or displayed in the same manner as in the embodiment of Figure 1.

[0131] In the embodiment shown in Figure 7, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared to two-channel sensor data, in addition to the two ear-worn devices complementing each other's sensor data, using four-channel sensor data can further improve the accuracy of sleep quality assessment results. This is because four-channel sensor data is richer in information than two-channel sensor data. More accurate assessment results can be obtained based on data with richer information. Moreover, four-channel sensor data can be set to different types of data, such as including first type data (e.g., ACC data) and second type data (e.g., PPG data), which can make full use of the characteristics of these different types of data to obtain more accurate assessment results.

[0132] Figure 8 is a flowchart of a process for generating sleep data based on sensor data according to an embodiment of this application. As shown in Figure 8, the process for generating sleep data based on sensor data includes: in step S801, determining a sleep interval and the sensor data within that sleep interval; in step S802, filtering the sensor data within the sleep interval; in step S803, dividing the filtered sensor data into time windows; and in step S804, determining the sleep state for the sensor data in each time window. At this point, sleep data is generated based on the sensor data.

[0133] The process in Figure 8 can be applied to steps S601 and S603 in Figure 6 or steps S701, S702, S703, and S704 in Figure 7. For clarity, when describing specific sensor data or sleep data, the first sensor data and first sleep data in step S601 or S701 and the first comprehensive sleep data in step S605 or S705 are used as examples for illustration, and the differences in other steps are supplemented with explanations. These explanations can also be applied to other sensor data and / or sleep data. The process in Figure 8 generates sleep data based on sensor data. Therefore, in addition to being applicable to embodiments where sleep data is first generated from sensors and then comprehensive sleep data is generated based on the sleep data (as in the embodiments of Figures 6 and 7), it can also be applied to embodiments where comprehensive sleep data is directly generated from sensor data (as in the embodiments of Figures 3 and 4), for example, to step S303 in Figure 3 or steps S403, S405, S408, and S409 in Figure 4. Of course, when applying steps S408 and S409 in Figure 4, since the determination process of step S801, the filtering process of step S802, and the partitioning process of step S803 may have been performed before, only the sleep state determination process of step S804 can be performed.

[0134] In step S801, the sleep interval and the sensor data within that sleep interval are determined. As mentioned earlier, the sleep interval can be the time range from when the user wearing the ear-worn device begins to fall asleep to when they end their sleep, and can be determined in various ways. The reason for determining the sleep interval is that the start and end times of sensor data collection do not necessarily coincide with the start and end times of the sleep interval. For example, the sleep interval may be set by the user to 22:00 to 7:00 the next day. However, the user may actually put on the ear-worn device at 23:00 and go to bed, and wake up and remove the device at 7:30 the next day. In this case, the sensor data collection start time is 23:00, meaning no data is collected between 22:00 and 23:00. Since the user sets the end time of sleep to 7:00, no data may be collected between 7:00 and 7:30. In this scenario, sensor data from 11:00 PM to 7:00 AM can be used as the sensor data within the sleep interval, and the period from 10:00 PM to 11:00 PM within this sleep interval can be marked as a data-free period. This sensor data can be from the first and second sensors, or from the first to the fourth sensors. For example, the sleep interval is set by the user to be from 10:00 PM to 7:00 AM the next day. The user puts on the ear-worn device at 10:00 PM and goes to bed, with the first ear-worn device always worn, the second ear-worn device falling off between 6:00 AM and 7:00 AM the next day, and the user waking up at 7:00 AM the next day. In this scenario, sensor data from the first ear-wearing device between 22:00 and 7:00 the next day can be used as the sensor data within the sleep interval. This sensor data can be the first sensor data and / or the third sensor data. Sensor data from the second ear-wearing device between 22:00 and 6:00 the next day can be used as the sensor data within the sleep interval, and the period from 6:00 to 7:00 the next day is marked as a data-free period. This sensor data can be the second sensor data and / or the fourth sensor data.

[0135] Sleep intervals can be set by the system, for example, by setting a fixed value (e.g., always set to 22:00 to 7:00 the next day), by setting different periods of time (e.g., different settings according to the season), by automatically determining them based on sleep data monitoring within a certain period, or even by automatically determining them based on sleep data after each sleep period. Sleep intervals can also be set by the user, for example, by setting a fixed value, by setting different periods of time, or by setting them before or after each sleep period. For example, setting them by the user after each sleep period or by automatic determination can cover the user's actual sleep time and ensure the integrity of the sensor-collected data as much as possible.

[0136] Sleep intervals can be set based on the user's sleep habits, ensuring that the set sleep intervals cover the user's usual sleep time. This makes it easy to determine the time interval for data processing when conducting sleep quality assessments. For ease of processing, the same sleep interval can be set for the sensor data that need to be processed. As an example, in the embodiments of Figure 1 or Figure 6, the same sleep interval can be set for the first and second sensor data, for example, 22:00 to 7:00 the next day. In the embodiments of Figure 2 or Figure 7, the same sleep interval can be set for the first to fourth sensor data, or one sleep interval can be set for the first and second sensor intervals, and another sleep interval can be set for the third and fourth sensor data. In the latter case, when determining the sleep quality assessment result in step S207 or S707, it may be necessary to consider the differences in sleep intervals between the first comprehensive sleep data and the second comprehensive sleep data, for example, to unify them into the same interval for subsequent processing.

[0137] In step S802, the sensor data within the sleep interval is filtered. For example, a Butterworth filter can be used to filter the sensor data to remove noise interference. The sensor data here may include data from the first and second sensors, or data from the first to the fourth sensors; the same applies to subsequent steps, and will not be elaborated further. For the data-free periods within the sleep interval, since there is no data, they are not processed in step S802 and subsequent steps.

[0138] In step S803, the filtered sensor data is divided into time windows. The time window can be the time granularity for processing the sensor data; for example, each time window can be 30 seconds or other durations, and the time windows usually do not overlap.

[0139] In step S804, a sleep state is determined for the sensor data of each time window. The sleep state can be set to be the same as in the embodiment of Figure 1.

[0140] In one embodiment, when determining sleep state based on sensor data for each time window, time windows where sleep state cannot be determined can be identified in a relatively simple way. These time windows may be unable to determine sleep state through subsequent calculations due to reasons such as weak signals. For example, the approximate signal-to-noise ratio (SNR) can be calculated for the filtered sensor data of each time window. If the approximate SNR of a time window is less than a threshold t, the time window is marked as "uncertain," so that it is not calculated and judged again when determining sleep state in subsequent calculations, thus saving overall computational load. In another embodiment, this "uncertain" state judgment can be performed uniformly on all sensor data instead of separately.

[0141] Different calculation methods can be used to determine sleep states for different types of sensor data. The following explanation assumes that an "uncertain" state is determined first, meaning that time windows with an "uncertain" state are excluded from the calculations and determinations below. If an "uncertain" state is not determined first, the "uncertain" state needs to be added to the sleep state determination result.

[0142] For PPG data, endpoint detection can be performed on sensor data for each time window to calculate PRV (pulse rate variability) feature data. Then, a machine learning model (such as Gradient Boosting Decision Tree (GBDT)) can be used to determine sleep state. As mentioned earlier, sleep states in PPG data can include, for example, wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3, and sometimes also include uncertain states. For ACC data, time-domain features (such as mean, variance, zero-crossing rate, etc.) and / or frequency-domain features (amplitude, frequency, mean, etc. after Fourier transform) can be extracted from sensor data for each time window. Then, a machine learning model (such as GBDT) can be used to determine sleep state. Sleep states in ACC data can include wakefulness and sleep, and sometimes also include uncertain states.

[0143] Sleep state is determined by sensor data for each time window, and corresponding sleep data is generated based on the sensor data. Specifically, first sleep data is generated based on first sensor data, second sleep data is generated based on second sensor data, third sleep data is generated based on third sensor data, and fourth sleep data is generated based on fourth sensor data.

[0144] In one embodiment, the process of generating sleep data based on sensor data may not include the filtering in step S802, or in addition to the filtering in step S802, it may include amplification, normalization and other processing before step S803 to facilitate subsequent processing.

[0145] When the flowchart in Figure 8 is applied to steps S601 and S603 of Figure 6, the division of the first sensor data and the division of the second sensor data in step S803 usually requires aligning the time windows. This ensures that the data in the corresponding time windows of the first and second sensor data are more comparable and complementary when they are integrated in step S605. If the time windows of the first sensor data and the second sensor data are not aligned, the data in the corresponding time windows actually target different sleep time intervals of the user, making the data incomparable. Using one to supplement the other may cause errors. This alignment can be achieved by setting the same sleep interval or the same time base for different sensor data. For example, when dividing the sensor data, it can start from the beginning of the sleep interval determined in step S801 and divide it sequentially according to the time windows. If the starting point of the sleep interval is the same, subsequent time windows can be aligned. For periods with no data in the sleep interval, their markings can be retained without dividing them, as dividing them may be meaningless. When a time window contains data only for a portion of its duration due to periods without data (e.g., a 30-second window with data for the first 20 seconds and none for the last 10), this time window can be simply marked as having no data (empty data) for easier processing. Assuming the start of the data-free period is actually at the 20th second of this time window, if the window is marked as data-free, the start of that data-free period can be moved forward by 20 seconds to align with the boundaries (start and end) of the time window. Conversely, when only a portion of a time window contains data (e.g., the duration of the data-containing portion is longer than the duration of the data-free portion, or the signal-to-noise ratio of the time window is greater than the average signal-to-noise ratio of the overall data), the data-containing portion can be used as the entire data for that time window. In this case, the start of the data-free period can be moved forward or backward based on its positional relationship with the time window to avoid the data-free period.

[0146] Figure 9 is a flowchart of a process for generating comprehensive sleep data based on sleep data according to an embodiment of this application. In Figure 9, the process of generating first comprehensive sleep data based on first sleep data and second sleep data in step S605 of Figure 6 or step S705 of Figure 7 is used as an example for illustration. Those skilled in the art will understand that the process in Figure 9 is also applicable to step S706 of Figure 7, simply by replacing the first sleep data, second sleep data, and first comprehensive sleep data with third sleep data, fourth sleep data, and second comprehensive sleep data.

[0147] As shown in Figure 9, the process of generating first comprehensive sleep data based on first sleep data and second sleep data includes: In step S901, determining the first ear-worn device as the main analysis device; in step S902, determining whether a time period is a data-free period for the first sleep data; if the determination result of step S902 is "yes", in step S903, using the second sleep data to determine the first comprehensive sleep data; if the determination result of step S902 is "no", in step S904, determining whether second sleep data exists in the time period; if the determination result of step S904 is "no", in step S905, using the first sleep data to determine the first comprehensive sleep data; if the determination result of step S904 is "yes", in step S906, determining whether the sleep state of the first sleep data within a time window is uncertain; if the determination result of step S906 is "yes", in step S907, using the second sleep data to determine the first comprehensive sleep data; if the determination result of step S906 is "no", in step... Step S908: Determine if the sleep states of the first sleep data and the second sleep data are the same within the time window. If the determination result of step S908 is "yes", in step S909, use the first sleep data to determine the first comprehensive sleep data. If the determination result of step S908 is "no", in step S910, use the first sleep data to determine the first comprehensive sleep data, or use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within the time window to determine the first comprehensive sleep data. After steps S907, S909, and S910, in step S911, determine if all time windows within the time period have been processed. If not, return to step S906 to continue processing; if processed, continue to step S912. If the determination result of step S911 is "yes" or after steps S903 and S905, in step S912, determine if all time periods have been processed. If not, return to step S902 to continue processing; if processed, the process ends. The following provides a detailed explanation of each step.

[0148] In step S901, the first ear-worn device is determined as the primary analysis device. By determining the primary analysis device, sleep data from the primary analysis device can be used as the main source, supplemented by sleep data from secondary analysis devices, simplifying the processing and reducing computational load. The first ear-wearing device is designated as the primary analysis device in the following situations: The average signal-to-noise ratio (SNR) of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device. In this case, the overall data quality of the first ear-wearing device is higher than that of the second ear-wearing device, thus it is designated as the primary analysis device. This applies to both two-channel and four-channel sensor data. Alternatively, for four-channel sensor data, the average SNR of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device, and the average SNR of the third sensor data from the first ear-wearing device is higher than that of the fourth sensor data from the second ear-wearing device. Or, the length of the data-free period in the first sleep data (or the total length of multiple data-free periods if multiple data-free periods exist) is shorter than the length of the data-free period in the second sleep data. In this case, the normal monitoring time of the first ear-wearing device is longer than that of the second ear-wearing device. This applies to both two-channel and four-channel sensor data. In other cases, the user may designate either the first or second ear-wearing device as the primary analysis device; this application is not limited to these limitations. Similar to the situation in Figure 2 where the main analysis device may differ in steps S205 and S206, when the process in Figure 9 is applied to steps S705 and S706 in Figure 7, the main analysis device may also differ. That is, the processes for generating the first and second comprehensive sleep data may use different ear-worn devices as the main analysis device. For example, the first and second sensor data are PPG data, and the third and fourth sensor data are ECG data. If the average signal-to-noise ratio (SNR) of the first sensor data is higher than that of the second sensor data, then when determining the first comprehensive sleep data, the first ear-worn device is determined as the main analysis device, and the first sensor data and the first sleep data are determined as the main analysis data. Simultaneously, if the average SNR of the fourth sensor data is higher than that of the third sensor data, then when determining the second comprehensive sleep data, the second ear-worn device is determined as the main analysis device, and the fourth sensor data and the fourth sleep data are determined as the main analysis data.

[0149] In step S902, it is determined whether a time period is a data-free period for the first sleep data. The determination in step S902 can be performed sequentially by time period within the sleep interval, or sequentially by time period type. When the first ear-worn device is designated as the primary analysis device, the first sleep data, as the primary analysis data, forms the basis of the analysis. Therefore, the sleep interval can be divided into data-free periods for the first sleep data, periods containing only the first sleep data, and periods where both the first and second sleep data exist. Then, processing can be performed sequentially by time period or by these time period types.

[0150] If, in step S902, a time period is determined to be a period without data for the first sleep data, then step S903 is executed, that is, the second sleep data is used to determine the first comprehensive sleep data during this time period. For example, if the period without data for the first sleep data is from 22:00 to 23:00, then the data from the second sleep data between 22:00 and 23:00 is used as the data for the first comprehensive sleep data between 22:00 and 23:00.

[0151] Next, the time periods containing the first sleep data within the sleep interval (i.e., the normal monitoring time of the first ear-worn device) are processed. If, in step S902, it is determined that a time period is not a data-free period for the first sleep data, meaning that the first sleep data exists in that time period, then step S904 is executed to further determine whether the second sleep data exists in that time period. If the second sleep data does not exist in that time period, meaning that only the first sleep data exists, then step S905 is executed, meaning that the first sleep data is used in that time period to determine the first comprehensive sleep data.

[0152] Starting from step S906, the time period in which both first and second sleep data exist simultaneously within the sleep interval is processed. This processing can be performed sequentially, window by window, according to a certain order. This order could be, for example, the chronological order of the time windows within the processed time period. In step S906, it is determined whether the sleep state of the first sleep data within a time window is uncertain. If the determination result of step S906 is "yes," then step S907 is executed. In step S907, the second sleep data is used to determine the first comprehensive sleep data; that is, the sleep state of the second sleep data within that time window is used as the sleep state of the first comprehensive sleep data within that time window. If the determination result of step S906 is "no," then step S908 is executed. In step S908, it is determined whether the sleep states of the first and second sleep data are the same within that time window.

[0153] If the judgment result of step S908 is "yes", then step S909 is executed. In step S909, the first sleep data is used to determine the first comprehensive sleep data, that is, the sleep state of the first sleep data in this time window is used as the sleep state of the first comprehensive sleep data in this time window. Although the first sleep data is used to determine the first comprehensive sleep data in step S909, since the prerequisite for executing step S909 is that the first sleep data and the second sleep data are in the same judgment time window, it can also be said that the second sleep data is used here, which does not affect the result of step S909. The reason for using the first sleep data in step S909 is, on the one hand, for ease of practical operation, and on the other hand, to save computation. For example, in actual operation, the first sleep data, which is the main analysis data, is first copied directly to the temporary storage address of the first comprehensive sleep data, and then the second sleep data is used to make partial adjustments to the first sleep data to obtain the final first comprehensive sleep data.

[0154] If the judgment result of step S908 is "no", then step S910 is executed. In step S910, the first sleep data can be used to determine the first comprehensive sleep data, that is, the sleep state of the first sleep data in this time window is used as the sleep state of the first comprehensive total sleep data in this time window. The reason for using the first sleep data is that when the first ear wearable device is determined to be the main analysis device in step S901, the first sleep data from it can be used as the main analysis data because it may be more accurate overall.

[0155] In step S910, the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within the time window can also be used to determine the first comprehensive sleep data. For example, if the feature value of the first sensor data is higher within the time window, the sleep state of the first sleep data corresponding to the first sensor data within that time window is used as the sleep state of the first comprehensive sleep data within that time window; if the feature value of the second sensor data is higher within the time window, the sleep state of the second sleep data corresponding to the second sensor data within that time window is used as the sleep state of the first comprehensive sleep data within that time window. Here, the feature value can be, for example, the signal-to-noise ratio, or other indicators that can evaluate data quality.

[0156] When the first ear-worn device is determined to be the main analysis device, the corresponding first sleep data of the main analysis device is used as the basis. Missing data (such as data from periods without data in the first sleep data, processed in step S903) and / or inaccurate data (such as data indicating an uncertain sleep state, processed in step S907; data indicating inconsistent sleep states, processed in step S910) are supplemented using second sleep data. This makes the synthesized first sleep data more complete and accurate compared to either unsynthesized first sleep data or unsynthesized second sleep data. Similar to the previous description, periods without data in the first sleep data refer to those periods within the sleep interval where no first sleep data is available due to missing first sensor data. An uncertain sleep state refers to a situation where the sensor data signal is too weak within a time window (e.g., the approximate signal-to-noise ratio is less than a threshold t, where t can be a predetermined value) to determine the sleep state through calculation.

[0157] After steps S907, S908, and S910, step S911 is executed, which determines whether all time windows within the current time period have been processed. If not, the process returns to step S906 to continue processing; if processed, the process continues to step S912. If the result of step S911 is "yes" or after steps S903 and S905, step S912 is executed, which determines whether all time periods have been processed. If not, the process returns to step S902 to continue processing; if processed, the process ends.

[0158] The embodiment in Figure 9 generates comprehensive sleep data based on two-channel sleep data. That is, by combining the sleep data from the two channels, the sleep data from the two channels can complement each other, thereby making the comprehensive sleep data more complete and accurate.

[0159] Figure 10 is a schematic diagram of multichannel sleep data within a sleep interval according to an embodiment of this application. In the following description, first and second sleep data are used as examples of multichannel sleep data. Those skilled in the art will understand that third and fourth sleep data are also applicable in other embodiments. For clarity and simplicity, the term "time period" is sometimes omitted and directly referred to by letters.

[0160] Figure 10 illustrates the data distribution of the first and second sleep data within the sleep interval T. As shown in Figure 10, for the first sleep data, there is a data-containing time period T1 (T1 = A + B + C) and a data-free time period T2 (T2 = D + E) within the sleep interval T; for the second sleep data, there is a data-containing time period T1' (T1' = A), a data-free time period T2' (T2' = B), a data-containing time period T3' (T3' = C + D), and a data-free time period T4' (T4' = E) within the sleep interval T. Although Figure 10 shows that both the first and second sleep data have data-free time periods within the sleep interval T, those skilled in the art will understand that in other embodiments, the first and / or second sleep data may not have data-free time periods within the sleep interval T. The data-containing time periods shown as rectangles in Figure 10 only indicate that there is data within that time period, and do not mean that the data values ​​are all the same; sleep data can have different values ​​at different times. Similarly, the two rectangles for the second sleep data are only used to illustrate that they are different time periods, and do not indicate that the data values ​​are higher or lower.

[0161] Based on the simultaneous presence of first and second sleep data, the time periods within sleep interval T are as follows: Time periods A and C contain both first and second sleep data; time period B contains only first sleep data; time period D contains only second sleep data; and time period E contains neither first nor second sleep data. In specific processing, some of these time periods can be merged based on the data used as the basis for analysis.

[0162] The sleep data in Figure 10 can be applied to the embodiment in Figure 9. When applied to the embodiment in Figure 9, for the data-free time period T2 (=D+E) of the first sleep data, in step S903, the data of the second sleep data within that time period (including the second sleep data in time period D and the empty data in time period E) is used as the first comprehensive sleep data. For the data-rich time period T1 (=A+B+C) of the first sleep data, two cases can be handled: if the second sleep data is absent (i.e., for the time period (B) where only the first sleep data exists), the first sleep data is used to determine the first comprehensive sleep data in step S905; if the second sleep data exists (i.e., for the time periods (A and C) where both the first and second sleep data exist), the first comprehensive sleep data is determined by processing the first and second sleep data window by window (steps S906 to S911). The processing order of each time period and each time window in Figure 10 can be the same as that in Figure 5, or other orders can be used.

[0163] Figure 10 shows four time windows W1, W2, W3, and W4. In time window W1, the sleep state of the first sleep data is "uncertain," while in time windows W2, W3, and W4, the sleep state of the first sleep data is not "uncertain." Specifically, in time window W2, the sleep state of the first sleep data is the same as that of the second sleep data; in time window W3, the feature value of the first sensor data is lower than that of the second sensor data; and in time window W4, the feature value of the first sensor data is higher than that of the second sensor data.

[0164] For time window W1, in step S906, the sleep state of the first sleep data is determined to be "uncertain". Therefore, step S907 is executed to use the second sleep data in time window W1 as the first comprehensive sleep time.

[0165] For time window W2, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are the same. Therefore, step S909 is executed, and the first sleep data in time window W2 is used as the first comprehensive sleep data.

[0166] For time window W3, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are different. Therefore, step S910 is executed to use the first sleep data to determine the first comprehensive sleep data, or to use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in time window W3 (i.e., the second sleep data) to determine the first comprehensive sleep data.

[0167] For time window W4, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are different. Therefore, step S910 is executed to use the first sleep data to determine the first comprehensive sleep data, or to use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in time window W4 (i.e., the first sleep data) to determine the first comprehensive sleep data.

[0168] Figure 11 is a flowchart of the process for determining sleep quality assessment results according to an embodiment of this application. As shown in Figure 11, the process for determining sleep quality assessment results includes: in step S1101, normalizing the first comprehensive sleep data; in step S1102, modifying the normalized first comprehensive sleep data to a standard size; and in step S1103, obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data through deep learning modeling.

[0169] The process in Figure 11 can be applied to step S107 in Figure 1 and step S607 in Figure 6, which will be explained in detail below.

[0170] In step S1101, the first comprehensive sleep data is normalized, for example, by the following method:

[0171] Where, x new Let x = [x1, x2, x3, ..., xn] represent the first comprehensive sleep data after normalization. x1 to xn represent the sleep state of the first comprehensive sleep data in each time window, respectively. n represents the data length of the first comprehensive sleep data, μ represents the average value of the first comprehensive sleep data, and σ represents the variance of the first comprehensive sleep data.

[0172] In step S1102, the normalized first comprehensive sleep data is modified to a standard size. Since everyone's sleep time is different, in subsequent processing, for example considering model requirements, it is necessary to convert it to the same data length. This can be done, for example, by using an interpolation algorithm to modify the data to a standard size, the size of which is determined by the model.

[0173] In step S1103, sleep quality assessment results are obtained based on standard-sized comprehensive sleep data through deep learning modeling. These results include difficulty falling asleep, normal sleep, insomnia, or light sleep. Through deep learning modeling, the data enters a deep learning clustering model to determine the sleep type. The model can be an unsupervised clustering algorithm. During model training, sleep data type modeling can be completed end-to-end without labels, aiming to increase the distance between different categories of data and decrease the distance between data of the same category until convergence. The trained model can then be directly used for application testing to determine the sleep quality assessment results of the data, such as difficulty falling asleep, normal sleep, insomnia, or light sleep. As shown in the embodiment of Figure 1, the sleep quality assessment results can be played to the user via voice on an ear-worn device or displayed on a mobile phone or other terminal device, facilitating the user's understanding of their sleep quality.

[0174] Figure 12 is a flowchart of the process for determining a sleep quality assessment result according to an embodiment of this application. As shown in Figure 12, the process for determining the sleep quality assessment result includes: in step S1201, normalizing the first comprehensive sleep data and the second comprehensive sleep data respectively; in step S1202, modifying the normalized first comprehensive sleep data and the second comprehensive sleep data to a standard size; in step S1203, obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data and the second comprehensive sleep data through deep learning modeling.

[0175] The process in Figure 12 can be applied to steps S207 in Figure 2 and S707 in Figure 7, which will be explained in detail below.

[0176] In step S1201, the first comprehensive sleep data and the second comprehensive sleep data are normalized respectively, for example, by the following method:

[0177] Where, x new Let x = [x1, x2, x3, ..., xn] represent the first comprehensive sleep data, where x1 to xn represent the sleep state of the first comprehensive sleep data in each time window, and n represents the data length of the first comprehensive sleep data. Let x = [x1, x2, x3, ..., xm] represent the second comprehensive sleep data, where x1 to xm represent the sleep state of the second comprehensive sleep data in each time window, and m represents the data length of the second comprehensive sleep data. m and n can be the same or different. μ represents the average value of the first or second comprehensive sleep data, and σ represents the variance of the first or second comprehensive sleep data.

[0178] In step S1202, the normalized first and second comprehensive sleep data are modified to a standard size. Since each person's sleep time is different, in subsequent processing, for example considering model requirements, it is necessary to convert them to the same data length. This can be done, for example, by using an interpolation algorithm to modify the data to a standard size, the size of which is determined by the model.

[0179] In step S1203, sleep quality assessment results are obtained through deep learning modeling based on standard-sized first and second comprehensive sleep data. These results include difficulty falling asleep, normal sleep, insomnia, or light sleep. Through deep learning modeling, the data enters a deep learning clustering model to determine the sleep type. The model can be an unsupervised clustering algorithm. During model training, the first and second comprehensive sleep data can be fused without labels to complete end-to-end modeling of sleep data types. The goal is to increase the distance between different categories of data and decrease the distance between data of the same category until convergence. The trained model can then be directly used for application testing to determine the sleep quality assessment results of the data, such as difficulty falling asleep, normal sleep, insomnia, or light sleep. As shown in the embodiment of Figure 1, the sleep quality assessment results can be played to the user via voice on an ear-worn device or displayed on a mobile phone or other terminal device, facilitating the user's understanding of their sleep quality.

[0180] Figure 13 is a flowchart of a sleep quality assessment method according to an embodiment of this application. As shown in Figure 13, in step S1301, first comprehensive sleep data is received. In step S1302, second comprehensive sleep data is received. In step S1303, a sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data. The first comprehensive sleep data may be generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, wherein the first sensor data and the second sensor data are first type data. The second comprehensive sleep data may be generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, wherein the third sensor data and the fourth sensor data are second type data.

[0181] The data in the embodiment of Figure 13 can be the same as those in the foregoing embodiments. The embodiment of Figure 13 can be executed on a terminal device such as a mobile phone. In this case, the terminal device can receive first and second integrated sleep data from the first and / or second ear-wearing devices, for example, via Bluetooth, and process the received data. Executing the sleep quality assessment method of Figure 13 on the terminal device can fully utilize the significantly higher processing power of the terminal device compared to the ear-wearing device (e.g., deep learning or artificial intelligence clustering algorithms can be used), thereby obtaining more accurate sleep quality assessment results and facilitating the subsequent presentation of the assessment results to the user.

[0182] Similar to the embodiments in Figures 2 and 7, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared with two-channel sensor data, the accuracy of sleep quality assessment results can be further improved. Moreover, the four-channel sensor data can be set to different types of data, which can make full use of the characteristics of these different types of data, thereby obtaining more accurate assessment results.

[0183] Implementation Method Two:

[0184] As mentioned above, sleep is a vital part of human life. Sleep helps us restore physical strength, promotes the repair of the brain and body, and enhances memory and learning ability. Therefore, modern people are paying more and more attention to sleep, and related sleep analysis algorithms are constantly being developed to help users understand their sleep problems.

[0185] However, existing sleep analysis algorithms typically follow the AASM (American Academy of Sleep Medicine) standards to analyze sleep, resulting in a series of sleep states such as Awake, REM (Rapid Eye Movement), N1 (N1 Sleep, Stage 1 sleep), N2 (N2 Sleep, Stage 2 sleep), and N3 (N3 Sleep, Stage 3 sleep, deep sleep). These sleep states are difficult to understand, making it hard for ordinary users to determine their sleep type and sleep problems. Existing methods for obtaining sleep types require pre-designing multiple sleep metrics (such as variability in bedtime, minutes to fall asleep, hours to sleep, and number of awakenings per hour), and then classifying them into multiple sleep types based on traditional clustering or manual statistical methods. However, classification based on manually designed sleep metrics relies entirely on the rationality and comprehensiveness of the metrics, and these metrics inevitably lead to information loss, resulting in low accuracy and reliability. Furthermore, the lack of large-scale data validation results in low generalization ability.

[0186] Therefore, this application embodiment also provides another sleep quality assessment method, hereinafter also referred to as the "sleep type assessment method," which can be applied to the application environment shown in Figure 14. The wearable device 102 or electronic device 104 obtains the sleep type corresponding to the first comprehensive sleep data based on a preset sleep model stored internally and the user's first comprehensive sleep data. When the above process is implemented on the wearable device 102, after obtaining the sleep type, the wearable device 102 can transmit the sleep type to the electronic device 104 via a communication network for the user to view, thereby determining their corresponding sleep type. Alternatively, the wearable device can announce the sleep type to the user via voice, or the wearable device can have a display screen to present the sleep type to the user in the form of text, charts, or images. The user's first comprehensive sleep data can be data determined after processing data related to the user's sleep state collected by sensors. Correspondingly, the first comprehensive sleep data sample set can also be a data sample set determined after processing data related to the user's sleep state collected by sensors. The preset sleep model obtained based on the first comprehensive sleep data sample set and the deep learning model can be determined in the server 106, and then the trained preset sleep model is sent to the wearable device 102 via the communication network, thereby reducing the memory usage and power consumption of the wearable device. Of course, the process of determining the preset sleep model can also be implemented by the wearable device 102 or the electronic device 104. Correspondingly, the aforementioned sensors can also be built into the wearable device 102 or the electronic device 104. Among them, the wearable device 102 can be, but is not limited to, a smartwatch, a smart bracelet, a head-mounted device, an ear-wearing device, etc. The ear-wearing device can be, but is not limited to, in-ear headphones, semi-in-ear headphones, open-back headphones, custom headphones, headphones, ear-hook headphones, clip-on headphones, hearing aids, or other devices worn on the ear. The electronic device 104 can be, but is not limited to, a mobile phone, a computer, a medical device (such as a PSG (Pulse Shape Generator) device), etc. For example, wearable devices can be integrated into electronic devices, such as medical devices. For instance, a PSG device can integrate sensors that come into contact with the human body to acquire sensor data, thereby obtaining the first comprehensive sleep data. Server 106 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The data storage system can be mounted on devices such as headphones, hard drives, hospital equipment (e.g., PSG (Pulse Shape Generator) devices), and mobile phones.

[0187] In an exemplary embodiment, as shown in FIG15, a sleep type assessment method is provided, including:

[0188] S202, Obtain the user's first comprehensive sleep data, which includes sleep states corresponding to multiple time windows.

[0189] The "first comprehensive sleep data" and its acquisition method can be similar to or the same as the "first comprehensive sleep data" and its acquisition method in Embodiment 1 above. For details, please refer to Embodiment 1 above, and it will not be repeated here. Of course, the "first comprehensive sleep data" and its acquisition method can be different from the "first comprehensive sleep data" and its acquisition method in Embodiment 1 above, and this application does not impose specific restrictions on this.

[0190] For example, the first comprehensive sleep data may be a sleep graph.

[0191] As shown in Figure 16, the first comprehensive sleep data can be a sleep map determined based on physiological data such as heart rate, electroencephalogram, or skin conductance response; it can also be a sleep map determined based on environmental data such as noise level or light level in the user's sleep environment; or it can be a sleep map determined based on behavioral data such as activity frequency during the user's sleep.

[0192] Furthermore, the aforementioned physiological, environmental, and behavioral data can all be processed by data processing algorithms such as filtering and denoising to generate corresponding sleep maps.

[0193] For example, during sleep, the resting heart rate is lower than that during wakefulness, the electroencephalogram (EEG) differs from that during wakefulness, and the skin is more relaxed compared to when awake. Therefore, these physiological data can be used to determine whether a user is asleep. This physiological data can be obtained using devices or sensors such as PPG (Photoplethysmography) sensors, ECG (Electrocardiogram) sensors, EEG (Electroencephalogram) sensors, temperature sensors, sound sensors (such as microphones), pressure sensors, and polysomnography (PSG) devices.

[0194] For example, since ambient light intensity and noise levels are low during sleep, it is possible to determine whether a user is asleep by analyzing the ambient light intensity and noise levels. This environmental data can be acquired using sound sensors (such as microphones), optical sensors, etc.

[0195] For example, since human activity decreases during sleep, motion sensors can be used to analyze a user's movement patterns. These motion sensors can be accelerometers, gyroscopes, etc. They can be integrated into watches or ear-worn devices worn by the user. Alternatively, pressure sensors installed under mattresses or pillows can monitor changes in body pressure to determine whether the user is asleep.

[0196] S202 can be executed on wearable devices, mobile phones, computers, tablets, computers, hospital equipment (such as PSG equipment), or servers.

[0197] For example, wearable devices include ear-worn devices, watches, wristbands, glasses, VR or AR devices, etc. Ear-worn devices may include in-ear headphones, semi-in-ear headphones, open-back headphones, custom headphones, over-ear headphones, ear-hook headphones, clip-on headphones, hearing aids, or other devices worn on the ear, etc., without specific limitations. For example, the above-mentioned wearable devices are equipped with sensors that can collect at least one of the above-mentioned physiological data, environmental data, and motion data, thereby enabling the wearable device or terminal device to determine the user's sleep state based on these data, thereby obtaining first comprehensive sleep data, which may be a sleep graph.

[0198] The aforementioned first comprehensive sleep data can be the first comprehensive sleep data during a short nap, the first comprehensive sleep data during a whole night's sleep period, the first comprehensive sleep data over a 24-hour period, or the first comprehensive sleep data over a week. Of course, it can also be any other first comprehensive sleep data that requires a long-term sleep type assessment, such as 2 days, 3 days, a week, a month, etc.

[0199] The time window can be determined based on the sleep stage segmentation method. For example, a time window of 30 seconds means that the user's sleep state is determined every 30 seconds.

[0200] S204, input the first comprehensive sleep data into the preset sleep model to obtain the sleep type corresponding to the first comprehensive sleep data; wherein, the preset sleep model is a model trained based on the first comprehensive sleep data sample set and a deep learning model.

[0201] For example, the first comprehensive sleep data may be a sleep graph, and the first comprehensive sleep data sample set may be a sleep graph sample set.

[0202] During a sleep cycle, a user's sleep state changes. Inputting first comprehensive sleep data, including sleep states across multiple time windows, into a preset sleep model allows for the determination of a predicted sleep type corresponding to the first comprehensive sleep data from these multiple time windows. Sleep states are selected from at least one of the following groups, or from at least one of the following groups after adding uncertainties: wakefulness and sleep; wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3; wakefulness, REM sleep, light sleep, and deep sleep; or wakefulness, REM sleep, and non-REM sleep; wakefulness and sleep.

[0203] More specifically, the first comprehensive sleep data can be a sleep graph, and the first comprehensive sleep data sample set can be a sample set of the sleep graph.

[0204] Therefore, the aforementioned sleep type assessment method, based on the user's first comprehensive sleep data and a preset sleep model, can determine the sleep type corresponding to the first comprehensive sleep data. Users can directly determine their sleep type and sleep problems based on the output of the preset sleep model, meaning the method is intuitive and accurate. Furthermore, the preset sleep model is trained on a first comprehensive sleep data sample set. The first comprehensive sleep data includes sleep states corresponding to multiple time windows. The deep learning model directly processes the first comprehensive sleep data, eliminating the need for manual feature extraction and reducing information loss. Based on a large amount of first comprehensive sleep data, the model processes the first comprehensive sleep data sample set based on the data characteristics of sleep graphs, then trains it, and validates it on a large amount of first comprehensive sleep data to obtain the preset sleep model. The preset sleep model has high accuracy and strong generalization ability. This sleep type assessment method is obtained by inputting the first comprehensive sleep data into the preset sleep model, resulting in high accuracy and strong generalization ability. It directly derives the sleep type from the first comprehensive sleep data, automatically completing the end-to-end sleep type classification, and is simple to operate.

[0205] In one exemplary embodiment, the deep learning model includes a deep clustering model.

[0206] Deep clustering models combine deep learning methods with clustering concepts to better discover latent structures in first-round comprehensive sleep data. The basic idea is to use deep neural networks to automatically extract features from high-dimensional data (such as physiological data images, noise, etc.) in the first-round comprehensive sleep data, and then apply these features for classification. The automatic feature extraction and classification processes are jointly optimized to obtain the sleep type.

[0207] Compared to traditional clustering methods, deep clustering models can better handle large-scale first comprehensive sleep data, thereby improving the generalization ability of sleep type assessment methods.

[0208] Deep clustering models can automatically extract deep features from the first comprehensive sleep data, eliminating the need for manual feature extraction and sleep type labeling. They can cluster the first comprehensive sleep data, automatically identifying and obtaining the sleep type. When the first comprehensive sleep data is a sleep graph, deep clustering models are well-suited and exhibit high accuracy. The sleep type assessment method using deep clustering models also possesses strong generalization ability.

[0209] In one exemplary embodiment, the step of determining the preset sleep model includes:

[0210] The first comprehensive sleep data in the first comprehensive sleep data sample set is input into the initial deep clustering model, and the initial deep clustering model is trained to obtain the preset sleep model. The first comprehensive sleep data sample set includes the first comprehensive sleep data of multiple users.

[0211] The first comprehensive sleep data is divided into a first comprehensive sleep data sample set and a first comprehensive sleep data validation set. The first comprehensive sleep data from the first comprehensive sleep data sample set is input into an initial deep clustering model for training to obtain a preset sleep model. Then, the obtained preset sleep model is validated based on the first comprehensive sleep data validation set. When the evaluation index obtained during validation is within a reasonable range, the preset sleep model can be considered to have high reliability. If the evaluation index exceeds the reasonable range, the parameters of each dimension in the initial deep clustering model can be automatically optimized, and a new preset sleep model can be determined again based on the first comprehensive sleep data from the first comprehensive sleep data sample set, until the obtained evaluation index is within a reasonable range.

[0212] The phrase "within a reasonable range" for the aforementioned evaluation metrics can be understood as the accuracy of the preset sleep model obtained based on the first comprehensive sleep data sample set meeting the accuracy requirements for practical application. For example, the evaluation metric could be the loss value. Of course, other data can also be used as evaluation metrics, which will not be elaborated upon here.

[0213] The first comprehensive sleep data sample set can be a collection of all or part of the first comprehensive sleep data of one or more users. Preferably, the first comprehensive sleep data sample set is a collection of partial sleep maps from multiple different users. This determined first comprehensive sleep data sample set can cover more sleep types, thereby improving the generalization ability of the sleep type assessment method. For example, a user's partial sleep map can refer to measuring the user's sleep state over multiple nights to obtain multiple sleep maps. Selecting one or more sleep maps from the multiple sleep maps constitutes the user's partial sleep map.

[0214] Furthermore, the first comprehensive sleep data sample set used in each training session can be the same or different.

[0215] In an exemplary embodiment, the initial deep clustering model includes a cascaded preset feature extraction module and a comparison module. The steps of inputting first comprehensive sleep data from a first comprehensive sleep data sample set into the initial deep clustering model and training the initial deep clustering model to obtain a preset sleep model include:

[0216] The first comprehensive sleep data in the first comprehensive sleep data sample set is input into the preset feature extraction module to obtain the first output result of the comparison module.

[0217] The loss value is obtained based on the first output result, and the preset feature extraction module and comparison module are trained based on the loss value to obtain the preset sleep model.

[0218] The loss value quantifies the difference between the sleep type of the preset sleep model and the target sleep type. A lower loss value indicates that the preset sleep model fits the first comprehensive sleep data well, while a higher loss value indicates that the preset sleep model's prediction is inaccurate. Therefore, based on the loss value, the adjustment direction and magnitude of each parameter in the preset feature extraction module and the comparison module can be determined. This allows for retraining based on the updated preset feature extraction module and comparison module, resulting in a reliable preset sleep model.

[0219] In an exemplary embodiment, the step of training a preset feature extraction module and a comparison module based on the loss value to obtain a preset sleep model includes:

[0220] Using stochastic gradient descent and backpropagation algorithms, the preset feature extraction module and comparison module are iteratively optimized based on the loss value until the loss value reaches the preset range, thus obtaining the preset sleep model.

[0221] Stochastic gradient descent is an optimization algorithm used to minimize a loss function to obtain the minimum loss value. Specifically, in each iteration, stochastic gradient descent uses only a randomly selected small subset (one or a few) of the first comprehensive sleep data samples to calculate the gradient. This method can accelerate the convergence of the pre-defined sleep model and effectively avoid the pre-defined sleep model getting trapped in local optima.

[0222] Backpropagation is a gradient calculation method in deep learning used to train multi-layer neural networks. Based on the chain rule, it calculates the contribution of each layer's parameters to the loss function by backpropagating the loss value of a pre-defined sleep model from the output layer to the input layer.

[0223] Therefore, iteratively optimizing the preset feature extraction module and comparison module using stochastic gradient descent and backpropagation algorithms can improve the training efficiency and accuracy of the preset sleep model while optimizing the loss value.

[0224] In an exemplary embodiment, the step of inputting first comprehensive sleep data from a first comprehensive sleep data sample set into a preset feature extraction module and obtaining a first output result from a comparison module includes:

[0225] The first comprehensive sleep data in the first comprehensive sleep data sample set is randomly segmented to obtain a first sleep segment set and a second sleep segment set. The first sleep segment set includes multiple first sleep segments, each determined based on multiple first comprehensive sleep data. Correspondingly, the second sleep segment set includes multiple second sleep segments, each determined based on multiple first comprehensive sleep data.

[0226] The first comprehensive sleep data is randomly segmented at least once. That is, the first sleep segment and the second sleep segment corresponding to the first comprehensive sleep data can be obtained after one random segmentation, or they can be obtained separately after two random segments. For example, the first comprehensive sleep data is randomly segmented once to obtain one or more sleep segments. When multiple sleep segments are obtained, two sleep segments are selected from the multiple sleep segments to obtain the first sleep segment and the second sleep segment. For example, the first comprehensive sleep data is randomly segmented twice; the first sleep segment is obtained after the first random segmentation, and the second sleep segment is obtained after the second random segmentation. The above are merely examples and do not constitute a limitation on the scope of protection; other implementation methods are also possible.

[0227] The first and second sleep segments were obtained by randomly segmenting the same first comprehensive sleep dataset. They are derived from the same dataset and belong to the same data category. The high similarity between these two randomly segmented sleep segments from the same first comprehensive sleep dataset aligns well with deep clustering models, allowing for reliable sleep type identification.

[0228] The first comprehensive sleep data sample set can be a collection of all or part of the first comprehensive sleep data of one or more users. Preferably, the first comprehensive sleep data sample set is a collection of partial sleep maps from multiple different users. This determined first comprehensive sleep data sample set can cover more sleep types, thereby improving the generalization ability of the sleep type assessment method.

[0229] For example, the first comprehensive sleep data sample set used in a training session is x = {x} 1 ,x 2 ,x 3 ,…,xN}, where N represents the amount of data in one training iteration, x 1 Represents the first sleep graph, x 2 Representing the second sleep pattern..., x N This represents the Nth sleep map, which can come from 1 to N people. Preferably, these sleep maps come from different people, for example, N people. Then, through random data partitioning, the data is processed to form two parts, namely the first sleep segment set. With the second sleep fragment set Among them, the same first comprehensive sleep data x i This will create two data fragments. For example, the first sleep graph x 1 The sleep was randomly split twice to form two sleep segments. Represents the first sleep graph x 1 The corresponding first sleep segment, Represents the first sleep graph x 1 The corresponding second sleep segment, the second sleep graph x 2 The sleep was randomly split twice to form two sleep segments. This represents the second sleep graph x. 2 The corresponding first sleep segment, This represents the second sleep graph x. 2 The corresponding second sleep segment, ..., the Nth sleep image x N The sleep was randomly split twice to form two sleep segments. This represents the first sleep segment corresponding to the Nth sleep map. This represents the second sleep segment corresponding to the Nth sleep map.

[0230] The first sleep segment set and the second sleep segment set are respectively input into the first preset feature extraction module and the second preset feature extraction module to obtain the first feature vector and the second feature vector.

[0231] The first and second preset feature extraction modules can automatically extract highly abstract data features, thereby determining the first and second feature vectors after layer-by-layer data processing. For example, the first and second preset feature extraction modules can be CNN (Convolutional Neural Network) feature extraction modules. The core structure of a CNN feature extraction module includes stacked convolutional layers, pooling layers, activation layers, etc. Parameters of each layer, such as the number and size of convolutional kernels, and the stride and pooling type of pooling layers, can be adjusted according to the confidence and accuracy of the prediction results of the preset sleep model to improve the prediction accuracy of the preset sleep model. This CNN feature extraction module can be designed independently or can utilize classic network structures, such as AlexNet or ResNet.

[0232] As shown in Figure 17, the first sleep fragment set After processing by the first preset feature extraction module, the first feature vector is output. Second Sleep Fragment Collection After processing by the second preset feature extraction module, the second feature vector is output.

[0233] The first feature vector and the second feature vector are sequentially input into the first convolutional neural network and the softmax function to obtain the first predicted class probability vector and the second predicted class probability vector. The first feature vector and the second feature vector are respectively input into the second convolutional neural network to obtain the third feature vector and the fourth feature vector. The first output result includes the first predicted class probability vector, the second predicted class probability vector, the third feature vector and the fourth feature vector.

[0234] As shown in Figure 17, the first feature vector After inputting into the first convolutional neural network, the output is... Then, the data is input into the softmax function, which converts it into a probability distribution to obtain the classification result, thus yielding the first predicted class probability vector. Correspondingly, the second feature vector After inputting into the first convolutional neural network, the output is... Then, the data is input into the softmax function, which converts it into a probability distribution to obtain the classification result, thus yielding the second predicted class probability vector.

[0235] More specifically, It can be represented in matrix form: Where N represents the number of sleep segments in a sleep segment cluster, and M represents the number of sleep types. That is: in, Represented as the first sleep fragment set x a Sleep segments The probability of it being the first type of sleep; Represented as the first sleep fragment set x a Sleep segments The probability of it being the second type of sleep; Represented as the first sleep fragment set x a Sleep segments The probability of it being the third type of sleep; ... Represented as the first sleep fragment set x a Sleep segments The probability that it is the Mth sleep type; Represented as the first sleep fragment set x a Sleep segments The probability of it being the first type of sleep; Represented as the first sleep fragment set x a Sleep segments The probability of it being the first type of sleep; Represented as the first sleep fragment set x a Sleep segments This represents the probability of the Mth sleep type. It should be noted that the specific meanings of the matrix elements not exhaustively listed above can be found in the examples above; they will not be repeated here. For example... The content and meaning refer to

[0236] As shown in Figure 17, the first feature vector After inputting into the second convolutional neural network, the third feature vector is output. Correspondingly, the second feature vector After inputting into the second convolutional neural network, the output is the fourth feature vector.

[0237] Furthermore, if each piece of first comprehensive sleep data in the first comprehensive sleep data sample set contains corresponding personal information, such as age, gender, region, and race, this personal information can be digitally encoded and its features normalized, and then concatenated with the z in the above output results. a , z b h a ′, h b By combining these methods, the accuracy of predictions can be improved.

[0238] In an exemplary embodiment, the step of randomly segmenting the first comprehensive sleep data in the first comprehensive sleep data sample set to obtain a first sleep fragment set and a second sleep fragment set includes:

[0239] Each piece of first comprehensive sleep data in the first comprehensive sleep data sample set is randomly segmented to obtain a first sleep segment and a second sleep segment that correspond one-to-one with the first comprehensive sleep data.

[0240] The first sleep segment set is determined based on each first sleep segment.

[0241] The set of second sleep segments is determined based on each second sleep segment.

[0242] Taking the first comprehensive sleep data as an example for explanation: If the first comprehensive sleep data sample set used in one training session is x = {x} 1 ,x 2 ,x 3 ,…,x N}, where N represents the amount of data in one training iteration, x 1 Represents the first sleep graph, x 2 Representing the second sleep pattern..., x N This represents the Nth sleep map, which can come from 1 to N people. Preferably, these sleep maps come from different people, for example, N people. The sleep maps in this first comprehensive sleep data sample set are randomly divided, i.e., based on the first sleep map x... 1 Obtain the first sleep segment Second sleep segment According to the second sleep chart x 2 Obtain the first sleep segment Second sleep segment ...; based on the Nth sleep graph x N Obtain the first sleep segment Second sleep segment Then, based on the first sleep segment corresponding to each sleep map, the first sleep segment set is determined, i.e., the first sleep segment set is... Correspondingly, based on the second sleep segment that corresponds one-to-one with each sleep map, the second sleep segment set is determined, that is, the second sleep segment set is...

[0243] The first sleep segment and the second sleep segment are randomly segmented from the same first comprehensive sleep data. The first sleep segment set and the second sleep segment set are obtained from the first sleep segment and the second sleep segment corresponding to each first comprehensive sleep data one by one. Based on the random segmentation method, the data of different time periods of the first comprehensive sleep data are cropped as sample similar data to enhance the generalization ability of the algorithm.

[0244] In an exemplary embodiment, the parameters in the first preset feature extraction module are the same as the parameters in the second preset feature extraction module.

[0245] The same parameters can be understood as weight sharing between the first preset feature extraction module and the second preset feature extraction module.

[0246] Using the same parameters can significantly reduce the total number of parameters of the deep clustering model, thereby reducing the complexity of the deep clustering model and also reducing the resource requirements for training and storage.

[0247] In an exemplary embodiment, the number of convolutional layers of the first convolutional neural network is greater than the number of convolutional layers of the second convolutional neural network.

[0248] The first convolutional neural network is related to the final sleep type. Therefore, setting a deeper number of convolutional layers can ensure that the preset sleep model can output reliable sleep types.

[0249] The second convolutional neural network is related to the optimization of the preset sleep model. Setting a shallower number of convolutional layers can improve the optimization efficiency of the preset sleep model.

[0250] In an exemplary embodiment, the time length of the random segmentation is greater than one sleep cycle.

[0251] During sleep, a person's sleep state switches periodically in different states, resulting in periodicity in the first comprehensive sleep data. Generally, 1.5h is considered one sleep cycle, and the first comprehensive sleep data can be a sleep graph. Usually, the time length of the random segmentation is greater than one sleep cycle (e.g., 1.5h). The number of sleep states in the sleep segments obtained by randomly segmenting the sleep graph once is l, L min <l < L, where L min is the number of sleep states corresponding to one sleep cycle, and L is the number of sleep states corresponding to the total time length of one sleep graph. There is no special limitation on the number of sleep states l in the sleep segments obtained by randomly segmenting the sleep graph once in this application. l can be greater than zero, such as l ≤ L min , or l = L.

[0252] Furthermore, the starting point of the random segmentation can be any value less than L min of any value.

[0253] For example, consider a sleep graph with a total sleep duration of 8 hours. If the sleep state within each 30-second interval can be determined, then the data length L of the sleep graph is 960, meaning there are 960 sleep states. If the random segmentation time is set to be greater than 1.5 hours, then the length of the randomly segmented data can be any value greater than 180 and less than 960. During random segmentation, if l is chosen to be 200 and the starting point of the random segmentation data length is 10, then the first sleep segment can be determined to be a segment of data within the range of 10 to 210 of the sleep graph. Similarly, the sleep graph can be randomly segmented again to obtain a second sleep segment. The second sleep segment can be a segment of data within the range of 200 to 900 of the sleep graph, or any other range. The second sleep segment can also be the entire sleep graph.

[0254] In an exemplary embodiment, the step of obtaining the loss value based on the first output result includes:

[0255] Determine the class similarity between the first predicted class probability vector and the second predicted class probability vector, the mean probability of the first predicted class probability vector, and the mean probability of the second predicted class probability vector.

[0256] The class similarity between the first predicted class probability vector and the second predicted class probability vector can be determined using the following formula:

[0257] Where i ≠ j.

[0258] Where i ≠ j.

[0259] Where i ≠ j.

[0260] Where i ≠ j.

[0261] Where i = 1, 2, 3, ..., M; j = 1, 2, 3, ..., M; T is the mathematical transpose.

[0262] Category similarity includes

[0263] The category loss value is determined based on the category similarity.

[0264] Category loss value as well as It can be determined using the following formula:

[0265] Where i ≠ j.

[0266] Where i ≠ j.

[0267] Where i = 1, 2, 3, ..., M; j = 1, 2, 3, ..., M, M represents the number of clusters; τ c The category loss is the temperature coefficient; specifically, τ c It can be set to 1 by default, or it can be adjusted based on experience and the performance requirements of the preset sleep model.

[0268] Determine the feature similarity between the third and fourth feature vectors, and determine the feature loss value based on the feature similarity.

[0269] Feature similarity between the third and fourth feature vectors It can be determined using the following formula:

[0270] Where i ≠ j.

[0271] Where i ≠ j.

[0272] Where i ≠ j.

[0273] Where i ≠ j.

[0274] Where i = 1, 2, 3, ..., N; j = 1, 2, 3, ..., N; T is the mathematical transpose.

[0275] Feature loss value as well as It can be determined using the following formula:

[0276] Where i ≠ j.

[0277] Where i ≠ j.

[0278] Where i = 1, 2, 3, ..., N; j = 1, 2, 3, ..., N; τ τ is the temperature coefficient for the feature loss. Specifically, τ can be set to 1 by default, or it can be adjusted based on experience and the performance requirements of the preset sleep model.

[0279] The loss value is determined based on the category loss value, the feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector.

[0280] The reliability of the preset sleep model obtained during training is determined based on the loss value. If the loss value does not meet the preset conditions, the parameters in the deep clustering model are modified through a parameter optimization algorithm. Then, based on the adjusted deep clustering model, the preset sleep model is re-determined until the loss value meets the preset conditions.

[0281] The preset condition is that the loss value reaches its minimum. For example, determining whether the loss value has reached its minimum can be done by comparing the loss value obtained after parameter tuning with the loss value obtained before parameter tuning. If the upward or downward trend of the loss value is greater than the reasonable trend level, and the deviation level is greater than the reasonable deviation level, then the loss value is considered not yet at its minimum. If the upward or downward trend of the loss value is less than or equal to the reasonable trend level, and the deviation level is less than or equal to the reasonable deviation level, then the loss value is considered to have reached its minimum. Here, the trend level refers to the change trend of the loss value obtained this time compared to the loss values ​​obtained previously; the reasonable trend level can be determined based on historical experience or the performance requirements of the preset sleep model. The deviation level is the difference between the loss value obtained this time and the loss value obtained the previous time or the time before that; the deviation level can also be the difference between the loss value obtained this time and the best loss value obtained previously; the reasonable deviation level can be determined based on historical experience or the performance requirements of the preset sleep model. Determining whether the loss value has reached its minimum can also be done when the obtained loss value shows a convergence trend and the loss value is within a reasonable range. Here, the reasonable range can be determined based on historical experience or the performance requirements of the preset sleep model.

[0282] Based on the aforementioned loss values, the preset sleep model can be optimized. After the sleep model is optimized, the data clustering results can be obtained. The number of clusters is set according to the data and experience. Figure 18 shows the clustering results of 5 classes obtained according to the sleep type assessment method, that is, 5 sleep types are obtained, such as A-type sleep, B-type sleep, C-type sleep, D-type sleep, and E-type sleep. For example, A-type sleep, B-type sleep, C-type sleep, D-type sleep, and E-type sleep represent normal sleep, light sleep, difficulty falling asleep, insomnia, and sleep disorder, respectively. Each cluster represents a class, and each point represents a data point, that is, a person's sleep map.

[0283] In an exemplary embodiment, the step of determining the loss value based on the category loss value, the feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector includes:

[0284] The entropy of the cluster assignment probability is determined based on the mean probability of the first predicted category probability vector and the mean probability of the second predicted category probability vector.

[0285] The loss value is determined based on the category loss value, feature loss value, and entropy of the cluster assignment probability.

[0286] The entropy H(Y) of the cluster assignment probability can be determined according to the following formula:

[0287] Where i ≠ j.

[0288] in, This represents the mean probability of each class in the classification prediction. Specifically, Let be the mean probability of the i-th predicted category among the sleep segments in the first sleep segment set. Let M be the mean probability of the predicted category of the sleep segment in the second sleep segment set, and let M represent the number of clusters, i = 1, 2, 3, ..., M. M is an integer greater than or equal to 1, and generally, M is an integer greater than or equal to 2. Specifically, when M is 5, it represents 5 sleep types; when M is 3, it represents 3 sleep types; and when M is 2, it represents 2 sleep types.

[0289] Specifically, It can be determined using the following formula:

[0290] It can be determined using the following formula:

[0291] The loss value L can be determined using the following formula:

[0292] Where i ≠ j.

[0293] in, and The feature loss value, and Let H(Y) be the category loss value, and H(Y) be the entropy of the cluster assignment probability. The entropy of the cluster assignment probability can control the clusters from becoming overly concentrated in one class.

[0294] In an exemplary embodiment, before the step of inputting the first sleep segment set and the second sleep segment set to the first preset feature extraction module and the second preset feature extraction module, respectively, the method further includes:

[0295] Each sleep segment in the first sleep segment set and the second sleep segment set is sequentially subjected to random Gaussian blurring, resampling, and standardization.

[0296] The window range length k of the random Gaussian blur is set according to the actual prediction performance, and the weight G of each sleep state of each first sleep segment and second sleep segment within the window range is set.

[0297] Specifically, the weight G can be determined according to the following formula:

[0298] Where, d i The distance from the center of the window is given as an example for the d-th data point within a random Gaussian blurred window; σ is a parameter of G.

[0299] Where σ = (0.3*((k-1)*0.5-1)+0.8)*p, p is a random value of the random Gaussian blur, 0≤p≤1. Furthermore, Gaussian blur can be performed when p is greater than a certain value, such as when p > 0.2.

[0300] The more blurred the first and second sleep fragment sets are after random Gaussian blurring, the greater the deviation between them and the original data, and the more difficult it is to identify the data, thereby enhancing the generalization ability and robustness of the preset feature extraction module.

[0301] Resample each sleep segment in the first and second sleep segment sets after random Gaussian blurring, so that all sleep segments in the first and second sleep segment sets are resampled. The lengths are consistent, that is The lengths are consistent.

[0302] Resampling can be achieved through linear interpolation or other interpolation methods to ensure that all sleep segments in the first sleep segment set and the second sleep segment set are resampled. The lengths of all data are consistent, thus ensuring the consistency of data length before entering the preset feature extraction module.

[0303] For example, by sequentially performing random segmentation, random Gaussian blurring, and resampling on the first comprehensive sleep data in Figure 16, the data shown in Figures 19, 20, and 21 can be obtained. Although the data lengths of the first comprehensive sleep data in Figures 19, 20, and 21 are consistent after resampling, based on the sparsity of the first comprehensive sleep data, the random segmentation length corresponding to Figure 20 is greater than that corresponding to Figure 19, but less than that corresponding to Figure 21.

[0304] Standardization refers to shifting the first comprehensive sleep data to a uniform scale to improve the convergence speed and performance of the preset sleep model.

[0305] Specifically, all data can be labeled using a standardization formula, as follows:

[0306] Where x represents the first comprehensive sleep data segment μ represents the mean of each first comprehensive sleep data point in the first comprehensive sleep data sample set, and σ1 represents the variance of each first comprehensive sleep data point in the first comprehensive sleep data sample set. The first comprehensive sleep data can be a sleep graph. For example, u and σ1 can be calculated from the statistical overall first comprehensive sleep data sample set.

[0307] Based on the characteristics of the first comprehensive sleep data and the working mode of contrastive learning in the deep clustering model, this solution designs a first comprehensive sleep data processing scheme, including random segmentation, random Gaussian blurring, resampling, and standardization, to fit the first comprehensive sleep data with the model and enable the model to converge quickly.

[0308] In one embodiment, after the step of acquiring the user's first comprehensive sleep data and before the step of inputting the first comprehensive sleep data into a preset sleep model, the method further includes:

[0309] The first comprehensive sleep data was resampled and standardized.

[0310] The processing method is consistent with that of the first comprehensive sleep data in the training phase, which is conducive to outputting reasonable sleep type assessment results.

[0311] After resampling and standardizing the first comprehensive sleep data, the processed first comprehensive sleep data is input into a preset sleep model to obtain the sleep type corresponding to the first comprehensive sleep data.

[0312] In one embodiment, the above sleep type assessment method further includes:

[0313] The distribution of sleep types is determined based on the identified sleep types.

[0314] Users can obtain multiple sleep types by using wearable devices equipped with this sleep type assessment method during multiple sleep cycles. By statistically analyzing these multiple sleep types, the distribution of sleep types corresponding to the user can be determined, allowing the user to clearly know which sleep type they prefer.

[0315] A defined distribution of sleep types can be represented numerically, such as as a percentage, or graphically, such as in a bar chart or pie chart. Of course, sleep type distribution can be represented in other ways as well; the examples above are not intended to limit it.

[0316] For example, as shown in Figures 22 and 23, which are sleep type distribution maps representing the distribution of sleep types, if type A sleep type represents normal sleep, type B sleep type represents light sleep, type C sleep type represents difficulty falling asleep, type D sleep type represents insomnia, and type E sleep type represents sleep disorder, then the sleep type distribution map in Figure 22 indicates that the user prefers the light sleep type represented by type B sleep type, and the sleep type distribution map in Figure 23 indicates that the user prefers the light sleep type represented by type B sleep type and the insomnia type represented by type D sleep type.

[0317] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0318] Based on the same inventive concept, this application also provides a sleep type assessment device for implementing the sleep type assessment method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more sleep type assessment device embodiments provided below can be found in the limitations of the sleep type assessment method described above, and will not be repeated here.

[0319] In an exemplary embodiment, as shown in FIG24, a sleep type assessment device 900 is provided, including: an acquisition module 902 and a prediction module 904, wherein:

[0320] The acquisition module 902 is used to acquire the user's first comprehensive sleep data, which includes sleep states corresponding to multiple time windows.

[0321] The prediction module 904 is used to input the first comprehensive sleep data into a preset sleep model to obtain the sleep type corresponding to the first comprehensive sleep data; wherein, the preset sleep model is a model trained based on the first comprehensive sleep data sample set and a deep learning model.

[0322] In one exemplary embodiment, the deep learning model in the sleep type assessment device 900 described above includes a deep clustering model.

[0323] In an exemplary embodiment, the sleep type assessment device 900 further includes a training module.

[0324] The training module is used to input the first comprehensive sleep data from the first comprehensive sleep data sample set into the initial deep clustering model and train the initial deep clustering model to obtain the preset sleep model. The first comprehensive sleep data sample set includes the first comprehensive sleep data of multiple users.

[0325] In an exemplary embodiment, the initial deep clustering model includes a cascaded preset feature extraction module and a comparison module, and the training module includes a first output result determination module and an optimization module.

[0326] The first output result determination module is used to input the first comprehensive sleep data in the first comprehensive sleep data sample set into the preset feature extraction module to obtain the first output result output by the comparison module.

[0327] The optimization module is used to obtain the loss value based on the first output result, and to train the preset feature extraction module and the comparison module based on the loss value to obtain the preset sleep model.

[0328] In one exemplary embodiment, the optimization module includes an iterative optimization module.

[0329] The iterative optimization module uses stochastic gradient descent and backpropagation algorithms to iteratively optimize the preset feature extraction module and comparison module based on the loss value until the loss value reaches the preset range, thus obtaining the preset sleep model.

[0330] In an exemplary embodiment, the first output result determination module includes: a sleep fragment set determination module, a first data determination module, and a second data determination module.

[0331] The sleep fragment set determination module is used to randomly segment the first comprehensive sleep data in the first comprehensive sleep data sample set to obtain the first sleep fragment set and the second sleep fragment set.

[0332] The first data determination module is used to input the first sleep segment set and the second sleep segment set into the first preset feature extraction module and the second preset feature extraction module, respectively, to obtain the first feature vector and the second feature vector.

[0333] The second data determination module is used to input the first feature vector and the second feature vector into the first convolutional neural network and the softmax function in sequence to obtain the first predicted class probability vector and the second predicted class probability vector. The first feature vector and the second feature vector are respectively input into the second convolutional neural network to obtain the third feature vector and the fourth feature vector. The first output result includes the first predicted class probability vector, the second predicted class probability vector, the third feature vector and the fourth feature vector.

[0334] In an exemplary embodiment, the sleep fragment set determination module includes: a random segmentation module, a first sleep fragment set determination module, and a second sleep fragment set determination module.

[0335] The random segmentation module is used to randomly segment each piece of first comprehensive sleep data in the first comprehensive sleep data sample set to obtain first sleep segments and second sleep segments that correspond one-to-one with the first comprehensive sleep data.

[0336] The first sleep segment set determination module is used to determine the first sleep segment set based on each first sleep segment.

[0337] The second sleep segment set determination module is used to determine the second sleep segment set based on each second sleep segment.

[0338] In one exemplary embodiment, the parameters in the first preset feature extraction module are the same as those in the second preset feature extraction module.

[0339] In one exemplary embodiment, the number of convolutional layers in the first convolutional neural network is greater than the number of convolutional layers in the second convolutional neural network.

[0340] In one exemplary embodiment, the duration of the random segmentation is longer than one sleep cycle.

[0341] In an exemplary embodiment, the optimization module includes: a third data determination module, a category loss value determination module, a feature loss value determination module, and a loss value determination module.

[0342] The third data determination module is used to determine the class similarity between the first predicted class probability vector and the second predicted class probability vector, the mean probability of the first predicted class probability vector, and the mean probability of the second predicted class probability vector.

[0343] The category loss value determination module is used to determine the category loss value based on category similarity.

[0344] The feature loss value determination module is used to determine the feature similarity between the third and fourth feature vectors, and to determine the feature loss value based on the feature similarity.

[0345] The loss value determination module is used to determine the loss value based on the category loss value, feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector.

[0346] In one exemplary embodiment, the loss value determination module includes an entropy determination module and a loss value determination submodule.

[0347] The entropy determination module is used to determine the entropy of the cluster assignment probability based on the probability mean of the first predicted category probability vector and the probability mean of the second predicted category probability vector.

[0348] The loss value determination submodule is used to determine the loss value based on the category loss value, feature loss value, and entropy of the cluster assignment probability.

[0349] In an exemplary embodiment, the sleep type assessment device 900 further includes a first data processing module.

[0350] The first data processing module is used to perform random Gaussian blurring, resampling, and standardization on each sleep segment in the first sleep segment set and the second sleep segment set in sequence.

[0351] In an exemplary embodiment, the sleep type assessment device 900 further includes a second data processing module.

[0352] The second data processing module is used to resample and standardize the first comprehensive sleep data.

[0353] Each module in the aforementioned sleep type assessment device 900 can be implemented entirely or partially through software, hardware, or a combination thereof. As shown in Figure 25, each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0354] In one exemplary embodiment, a wearable device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement any of the steps in the above-described sleep type assessment method.

[0355] Wearable devices equipped with the aforementioned sleep type assessment method can monitor in real time whether a user has entered a sleep state and perform targeted sleep type monitoring for that user, thereby achieving personalized sleep type assessment.

[0356] In some embodiments, wearable devices may be, but are not limited to, smartwatches, smart bracelets, headbands, ear-worn devices, etc. Ear-worn devices may include, but are not limited to, in-ear headphones, semi-in-ear headphones, open-back headphones, custom headphones, over-ear headphones, ear-hook headphones, clip-on headphones, hearing aids, or other devices worn on the ear.

[0357] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above for sleep type assessment.

[0358] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the methods described above for sleep type assessment.

[0359] In one embodiment, a sleep quality assessment device is provided, comprising a wearable device and a terminal device. The wearable device includes a first ear-wearing device and a second ear-wearing device. The wearable device is used to generate first comprehensive sleep data based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. The terminal device is used to determine a sleep quality assessment result based on the first comprehensive sleep data.

[0360] In one embodiment, a terminal device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the sleep type assessment method described above.

[0361] In some embodiments, the terminal device may include, but is not limited to, a mobile phone, a computer, a medical device (such as a PSG (Pulse Shape Generator) device), or a server. The server may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0362] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0363] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0364] The above describes the sleep quality assessment method and its specific process of this application. Those skilled in the art will understand that some or all of these methods and processes can be fixed to ear wearable devices, mobile phones or other devices by means of software, firmware or hardware, and can be run when needed to perform the methods and / or processes described and / or processes.

[0365] In one embodiment, this application discloses a sleep quality assessment device, which includes a module for performing the above-described method steps. Similar to the method described above, this sleep quality assessment device can improve the accuracy of sleep quality assessment results.

[0366] Although the technical solutions of this application have been described in detail above with two-channel data, four-channel data and two types of data, those skilled in the art will understand that other numbers of multi-channel data (such as six-channel data, eight-channel data, etc.) and more types of data (such as three types of data, four types of data, etc.) can also be used.

[0367] Obviously, the above embodiments of this application are merely examples for clear illustration and are not intended to limit the implementation of this application. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

[0368] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be interpreted as indicating or implying relative importance.

Claims

1. A sleep quality assessment method, characterized by, include: First comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device; and, The sleep quality assessment result is determined based on the first comprehensive sleep data.

2. The sleep quality evaluation method according to claim 1, characterized in that, The generation of the first comprehensive sleep data based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device includes: First sleep data is generated based on the first sensor data from the first ear-wearing device; Second sleep data is generated based on the second sensor data from the second ear-wearing device; and, First comprehensive sleep data is generated based on the first sleep data and the second sleep data.

3. The sleep quality evaluation method according to claim 1 or 2, characterized by, Also includes: A second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, wherein... Determining the sleep quality assessment result based on the first comprehensive sleep data includes: determining the sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data.

4. The sleep quality evaluation method according to claim 3, characterized in that, The second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, including: Third sleep data is generated based on third sensor data from the first ear-wearing device; Fourth sleep data is generated based on fourth sensor data from the second ear-wearing device; and, A second comprehensive sleep data is generated based on the third sleep data and the fourth sleep data.

5. The sleep quality evaluation method according to claim 3, wherein The first sensor data and the second sensor data are of the first type; the third sensor data and the fourth sensor data are of the second type.

6. The sleep quality evaluation method according to claim 4, wherein The first sleep data, the second sleep data, the third sleep data, the fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state of each time window in the sleep interval, wherein the sleep state is selected from at least one of the following groups or from at least one of the following groups after adding uncertain states: wakefulness and sleep; wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3; wakefulness, REM sleep, light sleep, and deep sleep; or wakefulness, REM sleep, and non-REM sleep.

7. The sleep quality evaluation method according to claim 1, wherein The first ear-wearing device and the second ear-wearing device are a pair of earphones, one of which is the left earphone and the other is the right earphone.

8. The sleep quality evaluation method according to claim 5, wherein The first type of data is obtained through an accelerometer, and the second type of data is obtained through a photoplethysmography (PPG) sensor; or, the first type of data is obtained through a PPG sensor, and the second type of data is obtained through an accelerometer.

9. The sleep quality evaluation method according to claim 2, wherein The first sleep data includes time periods with no data and / or time windows where sleep state is uncertain. Generating the first comprehensive sleep data based on the first sleep data and the second sleep data includes: For the time periods when there is no data in the first sleep data, the second sleep data is used to determine the first comprehensive sleep data; For time windows where the sleep state of the first sleep data is uncertain, the second sleep data is used to determine the first comprehensive sleep data.

10. The sleep quality evaluation method according to claim 2, wherein, The first sleep data and the second sleep data each indicate the sleep state at each time window within the sleep interval. Generating the first comprehensive sleep data based on the first sleep data and the second sleep data includes: If the first sleep data and the second sleep data have the same sleep state in a time window, then the data of the first sleep data in that time window is used to determine the first comprehensive sleep data; If the sleep states of the first sleep data and the second sleep data are different within a time window, the first comprehensive sleep data is determined by using the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within the time window, or by using the data of the first sleep data within the time window, wherein the feature value is the signal-to-noise ratio.

11. The sleep quality evaluation method according to claim 1, wherein The period of missing data from the first sensor within the sleep interval is shorter than the period of missing data from the second sensor within the same sleep interval. The generation of first comprehensive sleep data based on the first sensor data from the first ear-wearing device and the second sensor data from the second ear-wearing device includes: For time periods within the sleep interval where the first sensor data is missing, the second sensor data is extracted, and the first comprehensive sleep data is generated based on the first sensor data and the extracted second sensor data; or... The first comprehensive sleep data is generated based on the data from the first sensor.

12. The sleep quality evaluation method of claim 1, wherein, The first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, including: If the feature value of the first sensor data in a time window is higher than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the first sensor data in that time window is used to determine the first comprehensive sleep data. If the feature value of the first sensor data in a time window is lower than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the second sensor data in that time window is used to determine the first comprehensive sleep data, wherein the feature value is the signal-to-noise ratio.

13. The sleep quality evaluation method according to claim 9 or 10, characterized by, The average signal-to-noise ratio of the first sensor data is higher than that of the second sensor data, or the length of the data-free period of the first sleep data is shorter than that of the data-free period of the second sleep data.

14. The sleep quality evaluation method of claim 1, wherein, The sleep quality assessment results determined based on the first comprehensive sleep data include: The first comprehensive sleep data is normalized; The normalized first comprehensive sleep data was modified to standard size; and, The sleep quality assessment result is obtained by modeling with deep learning based on the first comprehensive sleep data of the standard size.

15. The sleep quality evaluation method of claim 1, wherein, The sleep quality assessment results include difficulty falling asleep, normal sleep, insomnia, or light sleep.

16. The sleep quality assessment method according to claim 1, characterized in that, The first comprehensive sleep data includes sleep states corresponding to multiple time windows, and the step of determining the sleep quality assessment result based on the first comprehensive sleep data includes: The first comprehensive sleep data is input into a preset sleep model to obtain the sleep type corresponding to the first comprehensive sleep data; wherein, the preset sleep model is a model trained based on a sleep data sample set and a deep learning model.

17. The sleep quality evaluation method according to claim 16, wherein The deep learning model includes a deep clustering model.

18. The sleep quality evaluation method according to claim 17, wherein, The steps for determining the preset sleep model include: The process involves inputting sleep data from a sleep data sample set into an initial deep clustering model and training the initial deep clustering model to obtain the preset sleep model. The first comprehensive sleep data sample set includes sleep data from multiple users. The initial deep clustering model includes a cascaded preset feature extraction module and a comparison module. The process of inputting sleep data from the sleep data sample set into the initial deep clustering model and training the initial deep clustering model to obtain the preset sleep model includes: The sleep data in the first comprehensive sleep data sample set is input into the preset feature extraction module to obtain the first output result output by the comparison module; The loss value is obtained based on the first output result, and the preset feature extraction module and the comparison module are trained based on the loss value to obtain the preset sleep model.

19. The sleep quality evaluation method according to claim 18, wherein, The step of inputting sleep data from the first comprehensive sleep data sample set into the preset feature extraction module to obtain the first output result from the comparison module includes: The sleep data in the first comprehensive sleep data sample set is randomly segmented to obtain a first sleep segment set and a second sleep segment set; The first sleep segment set and the second sleep segment set are respectively input into the first preset feature extraction module and the second preset feature extraction module to obtain the first feature vector and the second feature vector; The first feature vector and the second feature vector are sequentially input into the first convolutional neural network and the softmax function to obtain the first predicted class probability vector and the second predicted class probability vector. The first feature vector and the second feature vector are respectively input into the second convolutional neural network to obtain the third feature vector and the fourth feature vector. The first output result includes the first predicted class probability vector, the second predicted class probability vector, the third feature vector and the fourth feature vector.

20. The sleep quality evaluation method according to claim 19, wherein, The step of randomly segmenting the sleep data in the first comprehensive sleep data sample set to obtain a first sleep fragment set and a second sleep fragment set includes: The sleep data in the first comprehensive sleep data sample set are randomly segmented to obtain a first sleep segment and a second sleep segment that correspond one-to-one with the first comprehensive sleep data. Based on each of the first sleep segments, determine the first sleep segment set; The second sleep segment set is determined based on each of the second sleep segments.

21. The sleep quality evaluation method of claim 19, wherein, The parameters in the first preset feature extraction module are the same as those in the second preset feature extraction module; the number of convolutional layers in the first convolutional neural network is greater than the number of convolutional layers in the second convolutional neural network; and the duration of the random segmentation is greater than one sleep cycle.

22. The sleep quality evaluation method of claim 19, wherein, The step of obtaining the loss value based on the first output result includes: Determine the class similarity between the first predicted class probability vector and the second predicted class probability vector, the mean probability of the first predicted class probability vector, and the mean probability of the second predicted class probability vector; Based on the category similarity, determine the category loss value; Determine the feature similarity between the third feature vector and the fourth feature vector, and determine the feature loss value based on the feature similarity; The loss value is determined based on the category loss value, the feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector.

23. The sleep quality assessment method of claim 22, wherein, Determining the loss value based on the category loss value, the feature loss value, the probability mean of the first predicted category probability vector, and the probability mean of the second predicted category probability vector includes: The entropy of the clustering assignment probability is determined based on the probability mean of the first predicted category probability vector and the probability mean of the second predicted category probability vector. The loss value is determined based on the category loss value, the feature loss value, and the entropy of the cluster assignment probability.

24. The sleep quality evaluation method of claim 19, wherein, Before the step of inputting the first sleep segment set and the second sleep segment set into the first preset feature extraction module and the second preset feature extraction module respectively, the method further includes: Each sleep segment in the first sleep segment set and the second sleep segment set is sequentially subjected to random Gaussian blurring, resampling, and standardization.

25. The sleep quality assessment method of any one of claims 16 to 24, wherein, Before the step of inputting the first comprehensive sleep data into the preset sleep model, the method further includes: The first comprehensive sleep data is resampled and standardized.

26. A sleep quality assessment method, characterized by, include: Receive the first comprehensive sleep data; Receive second comprehensive sleep data; and, Based on the first comprehensive sleep data and the second comprehensive sleep data, a sleep quality assessment result is determined, wherein... The first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device. The first sensor data and the second sensor data are of type first data. The second comprehensive sleep data is generated based on third sensor data from the first ear wearable device and fourth sensor data from the second ear wearable device, wherein the third sensor data and the fourth sensor data are second type data. 27.A wearable device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes a computer program, it implements the steps of the method according to any one of claims 1 to 26.

28. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 26.

29. A sleep quality assessment apparatus, characterized by, The sleep quality assessment device includes a wearable device and a terminal device. The wearable device includes a first ear-wearing device and a second ear-wearing device. The wearable device is used to generate first comprehensive sleep data based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. The terminal device is used to determine the sleep quality assessment result based on the first comprehensive sleep data.

30. A terminal device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes a computer program, it implements the steps of the method according to any one of claims 1 to 26.

Images