Sleep quality evaluation method and device

Abstract

The invention relates to a sleep quality assessment method and device. The sleep quality assessment method includes generating first comprehensive sleep data based on first sensor data from a first ear wearable device and second sensor data from a second ear wearable device, and determining a sleep quality assessment result based on the first comprehensive sleep data. Sleep quality evaluation is performed by using multi-channel data from the two ear wearable devices, so that the accuracy of a sleep quality evaluation result is improved.

Description

Technical Field

[0001] This invention relates to the field of sleep monitoring technology, and more particularly to a method and apparatus for assessing sleep quality based on ear-worn devices. Background Technology

[0002] Sleep is a vital part of human life. It helps us restore physical strength, promotes brain and body repair, and enhances memory and learning ability. Modern people are paying increasing attention to sleep, leading to a proliferation of corresponding monitoring products, especially with the development of wearable devices, resulting in a plethora of home sleep monitoring products, such as watches and headphones. However, existing sleep monitoring products suffer from inaccurate monitoring and an inability to accurately assess sleep quality. Summary of the Invention

[0003] This invention improves the accuracy of sleep quality assessment results by using multi-channel data from two ear-worn devices to perform sleep quality assessment.

[0004] In one embodiment, a sleep quality assessment method is provided, including generating first comprehensive sleep data based on first sensor data from a first ear wearable device and second sensor data from a second ear wearable device, and determining a sleep quality assessment result based on the first comprehensive sleep data.

[0005] In one embodiment, generating first comprehensive sleep data based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device includes: generating first sleep data based on the first sensor data from the first ear-wearing device; generating second sleep data based on the second sensor data from the second ear-wearing device; and generating first comprehensive sleep data based on the first sleep data and the second sleep data.

[0006] In one embodiment, the sleep quality assessment method further includes generating second comprehensive sleep data based on third sensor data from a first ear-wearing device and fourth sensor data from a second ear-wearing device, wherein determining the sleep quality assessment result based on the first comprehensive sleep data includes: determining the sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data.

[0007] In one embodiment, generating second comprehensive sleep data based on third sensor data from a first ear-wearing device and fourth sensor data from a second ear-wearing device includes: generating third sleep data based on the third sensor data from the first ear-wearing device; generating fourth sleep data based on the fourth sensor data from the second ear-wearing device; and generating second comprehensive sleep data based on the third sleep data and the fourth sleep data.

[0008] In one embodiment, the first sensor data and the second sensor data are first type data, and / or, the third sensor data and the fourth sensor data are second type data.

[0009] In one embodiment, the first sleep data, the second sleep data, the third sleep data, the fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state of each time window in the sleep interval.

[0010] In one embodiment, the sleep state is selected from at least one of the following groups or from at least one of the following groups after adding the "uncertain" state: wakefulness and sleep; wakefulness, REM sleep, sleep stage N1, sleep stage N2 and sleep stage N3; wakefulness, REM sleep, light sleep and deep sleep; or wakefulness, REM sleep and non-REM sleep.

[0011] In one embodiment, the first ear-wearing device and the second ear-wearing device are a pair of earphones, one of which is the left earphone and the other is the right earphone.

[0012] In one embodiment, the first type of data is obtained by an accelerometer and the second type of data is obtained by a photoplethysmography (PPG) sensor; or, the first type of data is obtained by a PPG sensor and the second type of data is obtained by an accelerometer.

[0013] In one embodiment, generating first sleep data based on first sensor data from a first ear-wearing device includes: determining a sleep interval and the first sensor data within the sleep interval; filtering the first sensor data within the sleep interval; dividing the filtered first sensor data into time windows; and determining a sleep state for the first sensor data in each time window. Furthermore, generating second sleep data based on second sensor data from a second ear-wearing device includes: determining a sleep interval and the second sensor data within the sleep interval; filtering the second sensor data within the sleep interval; dividing the filtered second sensor data into time windows; and determining a sleep state for the second sensor data in each time window.

[0014] In one embodiment, generating third sleep data based on third sensor data from a first ear-wearing device includes: determining a sleep interval and the third sensor data within that sleep interval; filtering the third sensor data within the sleep interval; dividing the filtered third sensor data into time windows; and determining a sleep state for the third sensor data in each time window. Furthermore, generating fourth sleep data based on fourth sensor data from a second ear-wearing device includes: determining a sleep interval and the fourth sensor data within that sleep interval; filtering the fourth sensor data within the sleep interval; dividing the filtered fourth sensor data into time windows; and determining a sleep state for the fourth sensor data in each time window.

[0015] In one embodiment, the first sleep data includes time periods with no data and / or time windows where the sleep state is uncertain. Generating first comprehensive sleep data based on the first sleep data and the second sleep data includes: using the second sleep data to determine the first comprehensive sleep data for the time periods with no data in the first sleep data; and / or using the second sleep data to determine the first comprehensive sleep data for the time windows where the sleep state in the first sleep data is uncertain.

[0016] In one embodiment, the first sleep data and the second sleep data each indicate the sleep state in each time window within the sleep interval. Generating first comprehensive sleep data based on the first sleep data and the second sleep data includes: if the first sleep data and the second sleep data have the same sleep state in a time window, then using the data of the first sleep data in that time window to determine the first comprehensive sleep data; and / or, if the first sleep data and the second sleep data have different sleep states in a time window, then using the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in that time window to determine the first comprehensive sleep data, or using the data of the first sleep data in that time window to determine the first comprehensive sleep data.

[0017] In one embodiment, the time period in which the first sensor data is missing within the sleep interval is shorter than the time period in which the second sensor data is missing within the sleep interval. Generating the first comprehensive sleep data based on the first sensor data from the first ear-wearing device and the second sensor data from the second ear-wearing device includes: for the time period in the sleep interval in which the first sensor data is missing, extracting the second sensor data, and generating the first comprehensive sleep data based on the first sensor data and the extracted second sensor data; or, generating the first comprehensive sleep data based on the first sensor data.

[0018] In one embodiment, generating first comprehensive sleep data based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device includes: if the feature value of the first sensor data in a time window is higher than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the first sensor data in that time window is used to determine the first comprehensive sleep data; if the feature value of the first sensor data in a time window is lower than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the second sensor data in that time window is used to determine the first comprehensive sleep data.

[0019] In one embodiment, the average signal-to-noise ratio of the first sensor data is higher than that of the second sensor data, or the length of the data-free period of the first sleep data is shorter than that of the data-free period of the second sleep data.

[0020] In one embodiment, the characteristic value is the signal-to-noise ratio.

[0021] In one embodiment, determining the sleep quality assessment result based on the first comprehensive sleep data includes: normalizing the first comprehensive sleep data; modifying the normalized first comprehensive sleep data to a standard size; and obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data through deep learning modeling.

[0022] In one embodiment, determining the sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data includes: normalizing the first comprehensive sleep data and the second comprehensive sleep data respectively; modifying the normalized first comprehensive sleep data and the second comprehensive sleep data to a standard size; and obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data and the second comprehensive sleep data through deep learning modeling.

[0023] In one embodiment, sleep quality assessment results include difficulty falling asleep, normal sleep, insomnia, or light sleep.

[0024] In one embodiment, a sleep quality assessment method is provided, comprising: receiving first comprehensive sleep data; receiving second comprehensive sleep data; and determining a sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data. The first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, wherein the first and second sensor data are first type data; and the second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, wherein the third and fourth sensor data are second type data.

[0025] In one embodiment, a sleep quality assessment device is provided, which includes a module for performing the above-described method steps.

[0026] This invention assesses sleep quality based on multi-channel data (such as two-channel or four-channel data) from two ear-worn devices. If one ear-worn device fails to collect sensor data or provides inaccurate data due to issues such as poor fit, looseness, detachment, lack of battery power, or malfunction, sensor data from the other ear-worn device can supplement the assessment. In other words, the sensor data from the two ear-worn devices can complement each other, thereby improving the accuracy of the sleep quality assessment results. Furthermore, the multi-channel sensor data can be configured into different data types, fully utilizing the characteristics of each data type to obtain more accurate assessment results. Attached Figure Description

[0027] The features, advantages, and technical effects of exemplary embodiments of this application will now be described with reference to the accompanying drawings.

[0028] Figure 1 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention.

[0029] Figure 2 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention.

[0030] Figure 3 This is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of the present invention.

[0031] Figure 4 This is a flowchart of a process for generating comprehensive sleep data based on sensor data according to an embodiment of the present invention.

[0032] Figure 5 This is a schematic diagram of multi-channel sensor data within a sleep zone according to an embodiment of the present invention.

[0033] Figure 6 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention.

[0034] Figure 7 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention.

[0035] Figure 8 This is a flowchart of a process for generating sleep data based on sensor data according to an embodiment of the present invention.

[0036] Figure 9 This is a flowchart of a process for generating comprehensive sleep data based on sleep data according to an embodiment of the present invention.

[0037] Figure 10 This is a schematic diagram of multi-channel sleep data within a sleep interval according to an embodiment of the present invention.

[0038] Figure 11 This is a flowchart of the process for determining sleep quality assessment results according to an embodiment of the present invention.

[0039] Figure 12 This is a flowchart of the process for determining sleep quality assessment results according to an embodiment of the present invention.

[0040] Figure 13 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Detailed Implementation

[0041] The specific embodiments of this application will now be described with reference to the accompanying drawings. In the drawings, the same or similar reference numerals are used to denote the same or similar parts or steps, and repeated descriptions are omitted for simplicity.

[0042] Current smartwatches have several shortcomings when used for sleep monitoring. First, they suffer from significant data interference. Worn on the wrist, smartwatches typically rely on ACC (accelerometer) for sleep monitoring. However, arm movements monitored via ACC during sleep do not necessarily indicate a change in sleep state and can introduce noise interference, leading to inaccurate sleep tracking. Second, they suffer from low signal quality. Smartwatches collect PPG (photoplethysmography) data from the wrist. PPG sensor data is highly susceptible to temperature and movement. With frequent wrist movements and changes in wrist fit, plus sweating, the PPG signal quality remains poor.

[0043] Current headphone products monitor sleep by collecting physiological parameters of the ear, such as electrocardiogram (ECG), electroencephalogram (EEG), and respiratory data. However, these products have several problems. First, sleep monitoring data is inaccurate. Some headphone products lack mechanisms to handle situations where the earphones become loose, fall out, or run out of battery during sleep, resulting in incomplete sleep data and inaccurate sleep detection. Second, sleep quality assessment systems are inadequate. Assessing sleep quality requires a combination of sleep data collection, data processing methods, and sleep evaluation methods. Existing sleep headphones only collect sleep-related physiological data and determine sleep stages, failing to provide an effective assessment of sleep quality.

[0044] It is evident that accurately assessing sleep quality is one of the technical problems that this invention urgently needs to solve.

[0045] Figure 1 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Figure 1 As shown, in step S105, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. In step S107, a sleep quality assessment result is determined based on the first comprehensive sleep data.

[0046] The first and second ear-wearing devices can be headphones, hearing aids, or other ear-wearing devices worn on the ear. They can operate wirelessly, such as via Bluetooth, reducing the need for connections and simplifying wear. In one embodiment, the first and second ear-wearing devices are a pair of headphones, one being the left ear and the other the right ear. Using ear-wearing devices such as headphones to collect data has advantages over smartwatches. For example, when using motion data such as acceleration data for sleep monitoring, ear-wearing devices are directly correlated with head movements, directly collecting head movement data. Compared to wearable devices like smartwatches that collect wrist movement data, this reduces motion interference during sleep, resulting in higher-quality data. This is because some limb movements do not reflect key body movements, thus creating interference, while head movements are more likely to reflect key body movements. Moreover, the sensor data or sleep data from the two ear-wearing devices are complementary. If one ear-wearing device fails to collect sensor data or collects inaccurate sensor data due to problems such as poor fit, looseness, falling off, lack of power, or malfunction, the other ear-wearing device can collect sensor data normally, thereby supplementing the missing or inaccurate sensor data or sleep data.

[0047] In one embodiment, the first and second ear-wearing devices are a pair of custom-made earphones. Non-custom-made earphones, especially during sleep, are prone to loosening, falling out, or not fitting snugly, leading to incomplete and inaccurate sleep data, and consequently, inaccurate sleep quality assessment results. In this embodiment, using custom-made earphones as ear-wearing devices for sleep quality assessment ensures a good fit, secure wearing, and prevents them from falling out, improving the completeness and accuracy of sleep data detection, thus resulting in more accurate sleep quality assessment results. Furthermore, compared to non-custom-made earphones, custom-made earphones are tailored to the user's ear structure, offering greater wearing comfort and being more suitable for extended wear, thereby better meeting the requirements of sleep quality monitoring. In one embodiment, the first and second ear-wearing devices are a pair of wireless custom-made earphones.

[0048] The first and second sensor data can be data collected by sensors in the first and second ear-wearing devices, respectively. These sensors can be PPG sensors, ACC sensors, ECG (electrocardiogram) sensors, EEG (electroencephalogram) sensors, temperature sensors, sound sensors (such as microphones), etc. Those skilled in the art will understand that other types of sensors can also be used, as long as they can collect sleep-related data. The sensor data mentioned in this specification, including the data from the first to fourth sensors, can be data directly collected by the sensors, or data obtained after processing the data collected by the sensors; this invention does not impose any limitations on this. Data collected by the PPG sensor can be referred to as PPG sensor data or PPG data, and the same applies to other types of data.

[0049] In one embodiment, the first sensor data and the second sensor data are first-type data, which are obtained, for example, by an accelerometer (i.e., ACC data) or by a photoplethysmography (PPG) sensor (i.e., PPG data). In other words, the first sensor data and the second sensor data can be the same type of data, such as both being ACC data, or both being PPG data or other types of data. In another embodiment, the first sensor data and the second sensor data can be different types of data, for example, the first sensor data is first-type data, such as ACC data, and the second sensor data is second-type data, such as PPG data.

[0050] The first comprehensive sleep data indicates the sleep state within each time window of a sleep interval. The first comprehensive sleep data, and other sleep data mentioned later, can be a sleep graph, i.e., a graph that includes both time and sleep state information, indicating the user's sleep state at different times. A sleep interval can be the time range from the start to the end of sleep for a user wearing an ear-worn device, such as 10:00 PM to 7:00 AM the next day. The start and end times of this sleep interval can be set by the user or automatically determined by sensor data and / or other data. A time window can be the temporal granularity of processing sensor data; for example, each time window can be 30 seconds or other durations, and the time windows typically do not overlap.

[0051] Sleep states can be selected from at least one of the following groups or from at least one of the following groups after adding the "uncertain" state: (1) wakefulness and sleep; (2) wakefulness, REM sleep, sleep stage N1, sleep stage N2 and sleep stage N3; (3) wakefulness, REM sleep, light sleep and deep sleep; or (4) wakefulness, REM sleep and non-REM sleep. Four groups of sleep states are listed here by way of example. Those skilled in the art will understand that these sleep states can be those currently used in the field of sleep monitoring, and other sleep state groupings can also be used. For a certain type of sensor data, a fixed sleep state grouping can be used to facilitate calculation. In one embodiment, the first sleep state grouping can be used for ACC data. For example, when the first sensor data and the second sensor data are ACC data, the sleep state of the first composite sleep data in each time window can be wakefulness, sleep or uncertainty. In one embodiment, the second sleep state grouping can be used for PPG data. For example, when the first and second sensor data are PPG data, the sleep state of the first comprehensive sleep data in each time window can be wakefulness, REM sleep, sleep stage N1, sleep stage N2, sleep stage N3, or uncertain. PPG data can also be grouped using the third or fourth type of sleep state grouping. Other types of data can be grouped using one of the above-mentioned sleep state groupings. The choice of which sleep state grouping to use can be automatically determined based on the sensor data type, can be selected by the user, or can be selected by comparing the accuracy of the calculation results.

[0052] The processes in steps S105 and S107 will be described in detail below with reference to the accompanying drawings. In step S105, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. The first comprehensive sleep data in step S105 can be directly generated from the first sensor data and the second sensor data (see details). Figure 3 and Figure 4 Alternatively, it can be that first sleep data and second sleep data are generated from the first sensor data and second sensor data respectively, and then the data is generated based on the first sleep data and second sleep data (see...). Figure 6 and Figure 7 (Example).

[0053] In step S107, a sleep quality assessment result is determined based on the first comprehensive sleep data. The sleep quality assessment result may include, for example, difficulty falling asleep, normal sleep, insomnia, or shallow sleep. In one embodiment, the sleep quality judgment criteria can adopt existing standards, such as the total duration of each sleep state, or whether each sleep state matches or is similar to a predetermined curve of each assessment result. In other embodiments, different sleep quality judgment criteria can be used for different users to consider individual differences in sleep quality requirements. For example, in one embodiment, sleep quality judgment criteria can be formed by having users confirm their sleep effectiveness after each sleep session and accumulating a certain amount of data. In another embodiment, sleep quality judgment criteria can be formed by monitoring the user's activity intensity after each sleep session.

[0054] Figure 1 The sleep quality assessment method shown can be executed on an ear-worn device, or partly on the ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S105 and S107 are both executed on the first or second ear-worn device, wherein first sensor data is sent from the first ear-worn device to the second ear-worn device, or second sensor data is sent from the second ear-worn device to the first ear-worn device. In one embodiment, steps S105 and S107 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, step S105 is executed on a control module connected to the first ear-worn device, the second ear-worn device, or both, and step S107 is executed on a terminal device such as a mobile phone.

[0055] Sleep quality assessment results can be played to the user via voice on the first and / or second ear-worn device, or displayed on a mobile phone or other terminal device, allowing the user to easily understand their sleep quality. When using the display method, comparison charts or curves can be shown daily, weekly, or monthly to help users understand the comparison of their sleep over different periods.

[0056] exist Figure 1 In this embodiment, sleep quality is assessed based on two-channel sensor data (i.e., first and second sensor data) from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other, thereby improving the accuracy of the sleep quality assessment results.

[0057] Figure 2This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Figure 2 As shown, in step S205, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. In step S206, second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device. In step S207, a sleep quality assessment result is determined based on the first and second comprehensive sleep data.

[0058] The first ear-wearing device and the second ear-wearing device can be... Figure 1 The embodiments are the same and will not be repeated here. In one embodiment, the first sensor data and the second sensor data are first type data, and the third sensor data and the fourth sensor data are second type data, different from the first type data. For example, the first type data is obtained by an accelerometer (i.e., ACC data), and the second type data is obtained by a photoplethysmography (PPG) sensor (i.e., PPG data); or, the first type data is obtained by a PPG sensor (i.e., PPG data), and the second type data is obtained by an accelerometer (i.e., ACC data). Figure 2 In the illustrated embodiment, different types of sensor data can be used, such as ACC data and PPG data. The ACC sensor can monitor key body movement information during sleep, especially around the ear, reflecting head movement information unaffected by limb movements; the PPG sensor can monitor core physiological data of sleep, reflecting heart activity and the state of the cardiovascular system, which is closely related to sleep state. In other words, ACC data is body movement data, and PPG data is physiological data. Combining body movement data with physiological data for sleep quality assessment can effectively improve the accuracy of sleep quality assessment results.

[0059] The first and second comprehensive sleep data points each indicate the sleep state within a specific time window of the sleep interval. These data points can be used to create a sleep graph, a diagram that includes both time and sleep state information, indicating the user's sleep state at different times. The meaning and setting methods of sleep intervals, time windows, and sleep states can be discussed in conjunction with... Figure 1 The implementation methods are the same, and will not be repeated here.

[0060] The processes in steps S205, S206, and S207 will be described in detail below with reference to the accompanying drawings. In step S205, first comprehensive sleep data is generated based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device. In step S206, second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device. The first comprehensive sleep data in step S205 can be directly generated from the first sensor data and the second sensor data (see details). Figure 3 and Figure 4 Alternatively, it can be that first sleep data and second sleep data are generated from the first sensor data and second sensor data respectively, and then the data is generated based on the first sleep data and second sleep data (see...). Figure 6 and Figure 7 (Example). Similarly, the second comprehensive sleep data in step S206 can be generated directly from the data from the third sensor and the fourth sensor (see the specific implementation). Figure 3 and Figure 4 Alternatively, third sleep data and fourth sleep data can be generated first from data from the third sensor and data from the fourth sensor, respectively, and then the data generated can be based on the third sleep data and fourth sleep data (see...). Figure 7 (Example).

[0061] exist Figure 2 In this embodiment, the execution order of steps S205 and S206 is not explicitly limited. Step S205 may be executed before step S206, may be executed simultaneously with or at least partially overlap with step S206, or may be executed after step S206. Of course, step S207 is executed after steps S205 and S206.

[0062] In step S207, the sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data. Figure 2 The sleep quality assessment results and sleep quality judgment criteria in step S207 can be adopted with... Figure 1 The same method applies to the embodiments.

[0063] and Figure 1 The implementation examples are similar, Figure 2The sleep quality assessment method shown can be executed on an ear-worn device, or partly on an ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S205, S206, and S207 are all executed on the first or second ear-worn device, wherein the first and third sensor data are transmitted from the first ear-worn device to the second ear-worn device, or the second and fourth sensor data are transmitted from the second ear-worn device to the first ear-worn device. In one embodiment, step S205 is executed on the first ear-worn device, step S206 is executed on the second ear-worn device, and step S207 is executed on either the first or second ear-worn device. In one embodiment, step S205 is executed on the first ear-worn device, step S206 is executed on the second ear-worn device, and step S207 is executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, steps S205 and S206 are executed on a control module connected to the first ear-worn device, the second ear-worn device, or both, and step S207 is executed on a terminal device such as a mobile phone.

[0064] exist Figure 2 In the embodiments, the same as can be used. Figure 1 The sleep quality assessment results are played or displayed in the same manner as in the embodiments.

[0065] exist Figure 2 In this embodiment, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared to two-channel sensor data, in addition to the two ear-worn devices complementing each other's sensor data, using four-channel sensor data can further improve the accuracy of sleep quality assessment results. This is because four-channel sensor data is richer in information than two-channel sensor data, and more accurate assessment results can be obtained based on the richer information. Moreover, the four-channel sensor data can be set to different types of data, such as including first type data (e.g., ACC data) and second type data (e.g., PPG data), which can fully utilize the characteristics of these different types of data to obtain more accurate assessment results.

[0066] Figure 3 This is a flowchart illustrating the process of generating comprehensive sleep data based on sensor data according to an embodiment of the present invention. Figure 3As shown, the process of generating comprehensive sleep data based on sensor data includes: in step S301, determining the first ear-wearable device as the main analysis device; in step S302, extracting second sensor data for time periods in the sleep interval where the first sensor data is missing; and in step S303, generating first comprehensive sleep data based on the first sensor data and the extracted second sensor data.

[0067] In step S301, the first ear-wearing device is determined to be the primary analysis device. The first ear-wearing device is determined to be the primary analysis device in the following situations: the time period during which the first sensor data is missing in the sleep interval is shorter than the time period during which the second sensor data is missing in the same sleep interval; in this case, the normal monitoring time of the first ear-wearing device is longer than that of the second ear-wearing device; or, the average signal-to-noise ratio of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device; in this case, the overall data quality of the first ear-wearing device is higher than that of the second ear-wearing device.

[0068] In step S302, for time periods within the sleep interval where the first sensor data is missing (i.e., the first sensor data is absent or empty during that time period), second sensor data is extracted; in step S303, first comprehensive sleep data is generated based on the first sensor data and the extracted second sensor data. Figure 3 In this embodiment, the first ear-wearing device is designated as the primary analysis device, meaning its corresponding first sensor data is determined as the primary analysis data. Second sensor data from the secondary analysis device (the second ear-wearing device) is used as supplementary data to fill in time periods where the primary analysis data is missing. This approach reduces the time periods with missing data within the sleep interval; however, data is still missing for time periods where both first and second sensor data are missing. The process of generating the first comprehensive sleep data based on the first sensor data and the extracted second sensor data in step S303 is similar to... Figure 8 The process of generating sleep data based on sensor data is similar and will be referenced later. Figure 8 Please provide a detailed explanation.

[0069] In one embodiment, the process of generating first comprehensive sleep data based on first and second sensor data may include the following step instead of steps S302 and S303: generating first comprehensive sleep data based on first sensor data. In this embodiment, for example, because the effective monitoring time of the first ear wearable device is longer than that of the second ear wearable device (i.e., the time period in the sleep interval where the first sensor data is missing is shorter than the time period in the sleep interval where the second sensor data is missing), the first ear wearable device is determined as the main analysis device. In this case, the first sensor data from the first ear wearable device can be directly used to generate the first comprehensive sleep data, which can reduce the amount of computation. Especially when the effective monitoring time of the first ear wearable device is significantly longer than that of the second ear wearable device, the supplementary role of the second ear wearable device is limited. In this case, directly generating the first comprehensive sleep data based on the first sensor data from the first ear wearable device, which is the main analysis device, can reduce the computation of the second sensor data and the comparison of the first and second sensor data, thereby reducing the amount of computation. For example, if both the first sensor data and the second sensor data are ACC data, and the time period during which the first ear wearable device lacks the first sensor data in the sleep interval is shorter than the time period during which the second ear wearable device lacks the second sensor data in the same sleep interval, then the first comprehensive sleep data can be determined directly based on the ACC data of the first ear wearable device, which can reduce the amount of computation.

[0070] Figure 3 The process can be applied to Figure 1 Step S105 or Figure 2 In addition to step S205, it can also be applied to Figure 2 Step S206 simply involves replacing the first sensor data, second sensor data, and first comprehensive sleep data with the third sensor data, fourth sensor data, and second comprehensive sleep data. In applications... Figure 2In step S206, the primary analysis device determined in step S301 may be either a first ear-worn device or a second ear-worn device, depending primarily on the comparison results of the third and fourth sensor data. The comparison method for the third and fourth sensor data can be the same as that for the first and second sensor data. For example, the first ear-worn device may be determined as the primary analysis device if: the time period during which the third sensor data is missing in the sleep interval is shorter than the time period during which the fourth sensor data is missing in the same sleep interval; or if the average signal-to-noise ratio of the third sensor data is higher than that of the fourth sensor data. Otherwise, the second ear-worn device is determined as the primary analysis device. It is understood that different ear-worn devices may be used as the primary analysis device for generating the first and second comprehensive sleep data. This applies to four-channel or more channel data, especially when four-channel or more channel data of different data types are used. For example, if the data from the first and second sensors are PPG data, and the data from the third and fourth sensors are ECG data, and if the average signal-to-noise ratio (SNR) of the first sensor data is higher than that of the second sensor data, then when determining the first comprehensive sleep data, the first ear-worn device is designated as the primary analysis device, and the first sensor data is designated as the primary analysis data. Simultaneously, if the average SNR of the fourth sensor data is higher than that of the third sensor data, then when determining the second comprehensive sleep data, the second ear-worn device is designated as the primary analysis device, and the fourth sensor data is designated as the primary analysis data.

[0071] In one embodiment, the same ear-worn device can be used as the primary analysis device for generating the first and second comprehensive sleep data. For example, if the time period in which data from both the first and third sensors are missing simultaneously within a sleep interval is shorter than the time period in which data from both the second and fourth sensors are missing simultaneously within the same sleep interval, or if the combined average signal-to-noise ratio (SNR) of the first and third sensor data is higher than the combined average SNR of the second and fourth sensor data, then the first ear-worn device is determined as the primary analysis device, and the first and third sensor data are determined as the primary analysis data. In this case, the second ear-worn device serves as the secondary analysis device, and the second and fourth sensor data serve as the secondary analysis data.

[0072] For specific data types, such as ACC data, for time periods within the sleep interval where primary analysis data (e.g., first sensor data from the first ear-wearing device) exists, this data is directly used as the sensor data for that time period, without considering secondary analysis data. This reduces computational load compared to considering both primary and secondary analysis data simultaneously within that time period. For time periods within the sleep interval where primary analysis data is missing, secondary analysis data (e.g., second sensor data from the second ear-wearing device) is used as the sensor data for that time period. Because this secondary analysis data is a truncated portion of the secondary analysis data only during the time periods where primary analysis data is missing, it is called "truncated secondary analysis data," and when the secondary analysis data is second sensor data, it is called "truncated second sensor data." Figure 3 In some embodiments, comprehensive sleep data (e.g., first comprehensive sleep data or second comprehensive sleep data) is generated directly from sensor data (e.g., first and second sensor data, or third and fourth sensor data), while comprehensive sleep data is generated from sensor data first and then from the sleep data (e.g., first comprehensive sleep data or second comprehensive sleep data). Figure 6 and Figure 7 Compared to the previous embodiment, the processing is simplified and the amount of computation is reduced.

[0073] exist Figure 3 In this embodiment, comprehensive sleep data is generated based on two-channel sensor data from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other, improving the completeness of the comprehensive sleep data and thus providing more complete data for subsequent sleep quality assessment.

[0074] Figure 4 This is a flowchart illustrating the process of generating comprehensive sleep data based on sensor data according to an embodiment of the present invention. Figure 4As shown, the process of generating comprehensive sleep data based on sensor data includes: in step S401, determining the first ear-worn device as the main analysis device; in step S402, determining whether first sensor data exists within a time period in the sleep interval; if the determination result of step S402 is "no", in step S403, using second sensor data to generate first comprehensive sleep data within that time period; if the determination result of step S402 is "yes", in step S404, determining whether second sensor data exists within that time period; if the determination result of step S404 is "no", in step S405, using first sensor data to generate first comprehensive sleep data within that time period; if the determination result of step S404 is "yes", in step S406, dividing the first sensor data and second sensor data within that time period according to a time window; in step S407, determining whether the feature value of the first sensor data within a time window is higher than that of the second sensor data; if the determination result of step S407 is "yes", in... In step S408, the first comprehensive sleep data is determined using the sleep data corresponding to the first sensor data within the time window. If the judgment result in step S407 is "no", in step S409, the first comprehensive sleep data is determined using the sleep data corresponding to the second sensor data within the time window. After steps S408 and S409, in step S410, it is determined whether all time windows within the time period have been processed. If the judgment result in step S410 is "no" (i.e., all time windows within the time period have not been processed), the process returns to continue executing step S407. If the judgment result in step S410 is "yes" (i.e., all time windows within the time period have been processed), or after steps S403 and S405, in step S411, it is determined whether all time periods have been processed. If the judgment result in step S411 is "no" (i.e., all time periods have not been processed), the process returns to continue executing step S402. If the judgment result in step S411 is "yes" (i.e., all time periods have been processed), the process ends.

[0075] Figure 4 In step S401, the criteria for determining the main analytical equipment can be consistent with... Figure 3 The process is the same as step S301. Next, the time periods within the sleep interval are processed. These time periods can include periods with no data (neither first nor second sensor data exists), single-data periods with only one type of data (either only first sensor data or only second sensor data), and dual-data periods with both types of data (both first and second sensor data exist). In specific processing, these time periods can be merged based on the data used as the basis for analysis (such as master analysis data). Figure 4In this embodiment, processing is based on the first sensor data, which serves as the primary analysis data, to simplify the process and reduce computation. For example, for time periods where the first sensor data is absent, the second sensor data is used to generate the first comprehensive sleep data, regardless of whether the second sensor data exists during that time period. This combines time periods with no data and time periods with only the second sensor data into a single case, simplifying the process and reducing computation. Of course, for time periods where the second sensor data is also absent (i.e., time periods with no data), they can be simply set to empty data without time window division, further simplifying the process. In one embodiment, step S401 can be omitted, i.e., the primary analysis device is not determined, and either the first or second sensor data is used as the basis for subsequent processing steps. In this way, processing is simplified, and compared to the case where the primary analysis device is determined, this embodiment considers sensor data for all time periods and time windows, thus ensuring the same processing results.

[0076] In step S402, it is determined whether first sensor data exists within a time period of the sleep interval. This determination can be performed sequentially for each time period within the sleep interval, either chronologically or by the type of time period. When the first ear-worn device is designated as the primary analysis device, the first sensor data, as the primary analysis data, forms the basis of the analysis. Therefore, the sleep interval can be divided into time periods without first sensor data (or periods with no first sensor data), time periods containing only first sensor data, and time periods where both first and second sensor data are present. The data can then be processed sequentially, either chronologically or according to the type of time period.

[0077] If it is determined in step S402 that no first sensor data exists within a time period, then step S403 is executed, i.e., first comprehensive sleep data is generated using second sensor data within that time period. If it is determined in step S402 that first sensor data exists within a time period, then step S404 is executed to further determine whether second sensor data exists within that time period. If no second sensor data exists within that time period, i.e., only first sensor data exists, then step S405 is executed, i.e., first comprehensive sleep data is generated using first sensor data within that time period. The processes of generating first comprehensive sleep data using second sensor data in step S403 and generating first comprehensive sleep data using first sensor data in step S405 are similar to... Figure 8 The process is similar and will be referenced later. Figure 8 Please provide a detailed explanation.

[0078] If it is determined in step S404 that second sensor data exists within the time period, and combined with the determination in step S402, it is known that both first and second sensor data exist within this time period. In this case, step S406 and subsequent steps are executed. For time periods containing both first and second sensor data, the data quality of the two is compared, and the data with higher quality is selected as the basis for further processing. That is, the comprehensive sleep data is generated using the data with higher quality. The comparison of first and second sensor data is usually based on time windows. That is, after dividing the first and second sensor data into time windows (e.g., 30 seconds), the data quality of the first and second sensor data is judged for each time window. This fully utilizes the high-quality portions of the first and second sensor data. For example, in some time windows, the first sensor data has higher quality, while in other time windows, the second sensor data has higher quality. By comparing the first and second sensor data by time windows and selecting the sensor data with higher quality based on the comparison results, the high-quality portions of the first and second sensor data are combined, thereby improving the data quality of the generated first comprehensive sleep data. Data quality can be compared using features such as signal-to-noise ratio (SNR), with higher SNR indicating higher data quality. Alternatively, other features or comparison standards besides SNR can be used.

[0079] In step S406, the first sensor data and the second sensor data within the time period are divided according to time windows. The time window here is similar to other embodiments and can be the time granularity for processing the sensor data, such as 30 seconds, which will not be elaborated further here. After dividing the first and second sensor data according to time windows, step S407 and subsequent steps are executed, that is, determining which sensor data to use to generate the first comprehensive sleep data for each time window.

[0080] In step S407, it is determined whether the feature value of the first sensor data in a time window is higher than that of the second sensor data. If the feature value of the first sensor data in a time window is higher than that of the second sensor data in the same time window, then the sleep data corresponding to the first sensor data in that time window is used to determine the first comprehensive sleep data (step S408). If the feature value of the first sensor data in a time window is lower than that of the second sensor data in the same time window, then the sleep data corresponding to the second sensor data in that time window is used to determine the first comprehensive sleep data (step S409). The determination of the sleep data from the first sensor data in the same time window in step S408 and the determination of the sleep data from the second sensor data in the same time window in step S409 can be achieved using... Figure 8 The same procedure applies to step S804, and will not be described in detail here.

[0081] After steps S408 and S409, in step S410, it is determined whether all time windows within the current time period have been processed. If it is determined that not all time windows within the current time period have been processed, the process returns to step S407 to continue processing the remaining time windows. If it is determined that all time windows within the current time period have been processed, the process continues to step S411.

[0082] If step S410 determines that all time windows within the current time period have been processed, or if, after steps S403 and S405, step S411 determines whether all time periods have been processed, and if it is determined that not all time periods have been processed, then the process returns to step S402 to continue processing the remaining time periods. If it is determined that all time periods have been processed, then the process ends.

[0083] Figure 4 The process can be applied to Figure 1 Step S105 or Figure 2 In addition to step S205, it can also be applied to Figure 2 Step S206 simply involves replacing the data from the first sensor, the second sensor, and the first comprehensive sleep data with the data from the third sensor, the fourth sensor, and the second comprehensive sleep data. Figure 4 In step S406, before dividing the data from the first and second sensors (e.g., after step S404) or before calculating the feature values ​​of the data from the first and second sensors in step S407, the data from the first and second sensors can be filtered or otherwise processed. The specific filtering method can be the same as described above. Figure 8 The embodiments are the same.

[0084] For specific data types, such as PPG or ECG data, by comparing the feature values ​​of multi-channel sensor data from two ear-worn devices and selecting the portion of sensor data with higher feature values, the high-quality portion of the multi-channel sensor data can be fully utilized, thereby improving the overall data quality of the generated sleep data. Furthermore, in Figure 4 In some embodiments, comprehensive sleep data (e.g., first comprehensive sleep data or second comprehensive sleep data) is generated directly from sensor data (e.g., first and second sensor data, or third and fourth sensor data), while comprehensive sleep data is generated from sensor data first and then from the sleep data (e.g., first comprehensive sleep data or second comprehensive sleep data). Figure 6 and Figure 7 Compared to the previous embodiment, the processing is simplified and the amount of computation is reduced.

[0085] Figure 5This is a schematic diagram of multi-channel sensor data within a sleep interval according to an embodiment of the present invention. In the following description, first and second sensor data are used as examples of multi-channel sensor data. Those skilled in the art will understand that third and fourth sensor data are also applicable in other embodiments. For clarity and simplicity, the term "time period" is sometimes omitted and directly referred to by letters.

[0086] Figure 5 The data from the first and second sensors are shown within the sleep interval T. For example... Figure 5 As shown, for the first sensor data, within the sleep interval T, there is a data-containing time period T1 (T1 = A + B + C) and a data-free time period T2 (T2 = D + E); for the second sensor data, within the sleep interval T, there is a data-containing time period T1' (T1' = A), a data-free time period T2' (T2' = B), a data-containing time period T3' (T3' = C + D), and a data-free time period T4' (= E). Although Figure 5 The diagram shows that both the first and second sensor data have periods of no data within the sleep interval T. However, those skilled in the art will understand that in other embodiments, the first and / or second sensor data may not have periods of no data within the sleep interval T. Figure 5 The rectangles representing data periods only indicate that data was available within those periods, not that all data values ​​were the same. Sensor data can have different values ​​at different times. Similarly, the two rectangles representing the second sensor data only indicate that they represent different time periods, not that the data values ​​are higher or lower.

[0087] Based on the simultaneous presence of data from the first and second sensors, the sleep interval T contains the following time periods: time period E, a period with no data where neither type of data exists; time periods B and D, single-data periods where only one type of data exists; and time periods A and C, dual-data periods where both types of data exist. The single-data periods containing only one type of data include time periods containing only the first sensor data (time period B) and time periods containing only the second sensor data (time period D). In specific processing, some of these time periods can be merged based on the data used as the basis for analysis.

[0088] Figure 5 Sensor data can be applied to Figure 3 or Figure 4 Examples of implementations. Applied to... Figure 3In this embodiment, first sensor data is missing during time period T2(D+E) within the sleep interval T. Therefore, in step S302, second sensor data is directly used as the extracted second sensor data for this time period, i.e., the second sensor data within time period D and the empty data (i.e., no data) within time period E. Then, in step S303, first comprehensive sleep data is generated based on the first sensor data of time period T1(A+B+C), the second sensor data of time period D, and the empty data of time period E.

[0089] In application Figure 4 In the implementation example, the time periods within the sleep interval can be processed sequentially according to the order of the time periods, or in the order of first processing the data-free time period of the main analysis (T2 = D + E), then processing the data-single time period of the main analysis (B), and finally processing the data-double time periods (A and C). The following descriptions of each time period are not limited to a specific order; the order described in this paragraph or other orders may be used.

[0090] Time period A contains both first sensor data and second sensor data. Therefore, for time period A, step S402 determines that first sensor data exists, and step S404 determines that second sensor data exists. Then, proceed to step S406 to divide the first and second sensor data within the time period into time windows. Then, for the divided first and second sensor data, compare the characteristic values ​​of the two data points for each time window. When, in step S407, it is determined that the first sensor data is within one time window (e.g., ... Figure 5 When the feature value in the time window W1 shown is higher than the feature value of the second sensor data in the same time window, step S408 is executed to determine the first comprehensive sleep data using the sleep data corresponding to the first sensor data in that time window; when it is determined in step S407 that the first sensor data is in a time window (e.g., Figure 5 When the feature value in the time window W1 is lower than the feature value of the second sensor data in that time window, step S409 is executed to determine the first comprehensive sleep data using the sleep data corresponding to that time window from the second sensor data. For example, suppose that in Figure 5 In the time window W1 shown, if the feature value of the first sensor data is lower than the feature value of the second sensor data in the same time window, then the second sensor data is used in the time window W1 to determine the value of the first comprehensive sleep data in that time window, i.e., the sleep state in that time window.

[0091] In time period B, only the first sensor data exists. Therefore, for time period B, it is determined in step S402 that the first sensor data exists and in step S404 that the second sensor data does not exist. Then, the process proceeds to step S405, where the first sensor data is used to generate the first comprehensive sleep data in this time period.

[0092] Time period C is similar to time period A, as it contains both data from the first sensor and data from the second sensor, so its processing is similar to that of time period A. Figure 5 The diagram shows a time window W2, in which sensor data with higher eigenvalues ​​is used to determine the first comprehensive sleep data. For example, if the eigenvalue of the first sensor data in time window W2 is higher than the eigenvalue of the second sensor data in that time window, then the first sensor data is used in time window W2 to determine the value of the first comprehensive sleep data in that time window, i.e., the sleep state in that time window.

[0093] Since there is no first sensor data in time period T2, in step S402 it is determined that there is no first sensor data in time period T2, and then the process proceeds to step S403, where the first comprehensive sleep data is generated using the second sensor data (including the second sensor data in time period D and the empty data in time period E). For example, for time period D, it can be done by... Figure 8 The process uses data from the second sensor to generate the first comprehensive sleep data. For time period E, the value of the first comprehensive sleep data is set to empty.

[0094] Figure 6 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Figure 6 As shown, in step S601, first sleep data is generated based on first sensor data from the first ear-wearing device. In step S603, second sleep data is generated based on second sensor data from the second ear-wearing device. In step S605, first comprehensive sleep data is generated based on the first sleep data and the second sleep data. In step S607, a sleep quality assessment result is determined based on the first comprehensive sleep data.

[0095] Sensor data from wearable ear devices is processed to generate sleep data. Figure 6In the illustrated embodiment, the first sleep data is generated based on the first sensor data, and the second sleep data is generated based on the second sensor data. The first and second sleep data are then combined to generate the first comprehensive sleep data. The first sleep data, the second sleep data, and the first comprehensive sleep data each indicate the sleep state within a specific time window of a sleep interval. The first sleep data, the second sleep data, and the first comprehensive sleep data can be a sleep graph, i.e., a graph including both time and sleep state information, which can indicate the user's sleep state at different times. A sleep interval can be the time interval from when the user wearing the ear-worn device begins to fall asleep to when they end to fall asleep, for example, from 10:00 PM to 7:00 AM the next day. The start and end times of this sleep interval can be set by the user or automatically determined by sensor data and / or other data. A time window can be the time granularity for processing the sensor data; for example, each time window can be 30 seconds or other durations, and the time windows typically do not overlap.

[0096] Sleep states can be selected from at least one of the following groups or from at least one of the following groups after adding the "uncertain" state: (1) wakefulness and sleep; (2) wakefulness, REM sleep, sleep stage N1, sleep stage N2 and sleep stage N3; (3) wakefulness, REM sleep, light sleep and deep sleep; or (4) wakefulness, REM sleep and non-REM sleep. Four groups of sleep states are listed here by way of example. Those skilled in the art will understand that these sleep states can be the sleep states currently used in the field of sleep monitoring, and other sleep state groupings can also be used. In one embodiment, the first sleep data, the second sleep data and the first comprehensive sleep data can be grouped using the same sleep state, such as one of the groups (1) to (4) above. In this case, the sleep stage method of the first sleep data, the second sleep data and the first comprehensive sleep data is the same, which is convenient for calculation and comparison. Different sleep state groupings can be used for different types of sensor data. In one embodiment, the first sleep state grouping can be used for ACC data. For example, when the first sensor data and the second sensor data are ACC data, the sleep states of the first sleep data, the second sleep data, and the first comprehensive sleep data in each time window can be awake, asleep, or uncertain. In one embodiment, the PPG data can be grouped into a second type of sleep state. For example, when the first sensor data and the second sensor data are PPG data, the sleep states of the first sleep data, the second sleep data, and the first comprehensive sleep data in each time window can be awake, REM sleep, sleep stage N1, sleep stage N2, sleep stage N3, or uncertain.

[0097] The processes in steps S601, S603, S605, and S607 will be described in detail later with reference to the accompanying drawings. Figure 6 In this embodiment, the execution order of steps S601 and S603 is not explicitly limited. Considering that the first sensor data and the second sensor data come from different ear-wearing devices, the order of steps S601 and S603 can be set based on their generation order or the convenience of data processing. Step S601 can be executed before step S603, can be executed simultaneously with or at least partially overlap with step S603, or can be executed after step S603. Step S605 is executed after steps S601 and S603, and step S607 is executed after step S605. In step S607, a sleep quality assessment result is determined based on the first comprehensive sleep data. The sleep quality assessment result may be, for example, difficulty falling asleep, normal sleep, insomnia, shallow sleep, etc. The criteria for judging sleep quality can be... Figure 1 The implementation examples are similar.

[0098] Figure 6 The sleep quality assessment method shown can be executed on an ear-worn device, or partly on an ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S601, S603, S605, and S607 are all executed on the first or second ear-worn device, wherein first sensor data is sent from the first ear-worn device to the second ear-worn device, or second sensor data is sent from the second ear-worn device to the first ear-worn device. In one embodiment, step S601 is executed on the first ear-worn device, step S603 is executed on the second ear-worn device, and steps S605 and S607 are executed on either the first or second ear-worn device. In one embodiment, step S601 is executed on the first ear-worn device, step S603 is executed on the second ear-worn device, and steps S605 and S607 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, step S601 is executed on the first ear-wearing device, step S603 is executed on the second ear-wearing device, step S605 is executed on the control module connected to the first ear-wearing device, the second ear-wearing device, or both, and step S607 is executed on a terminal device such as a mobile phone.

[0099] exist Figure 6 In the embodiments, the same as can be used. Figure 1 The sleep quality assessment results are played or displayed in the same manner as in the previous embodiments.

[0100] exist Figure 6In this embodiment, sleep quality is assessed based on two-channel sensor data (i.e., first and second sensor data) from two ear-wearing devices. If sensor data cannot be collected or the collected sensor data is inaccurate due to problems such as poor fit, looseness, falling off, lack of power, or malfunction of one ear-wearing device during wear, sensor data from the other ear-wearing device can be used to supplement it. In other words, the sensor data from the two ear-wearing devices can complement each other, thereby improving the accuracy of the sleep quality assessment results.

[0101] Figure 7 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Figure 7 As shown, in step S701, first sleep data is generated based on first sensor data from the first ear-wearing device. In step S703, second sleep data is generated based on second sensor data from the second ear-wearing device. In step S702, third sleep data is generated based on third sensor data from the first ear-wearing device. In step S704, fourth sleep data is generated based on fourth sensor data from the second ear-wearing device. In step S705, first comprehensive sleep data is generated based on the first and second sleep data. In step S706, second comprehensive sleep data is generated based on the third and fourth sleep data. In step S707, a sleep quality assessment result is determined based on the first and second comprehensive sleep data.

[0102] In one embodiment, the first sensor data and the second sensor data are first type data, and / or, the third sensor data and the fourth sensor data are second type data, different from the first type data. For example, the first type data is obtained by an accelerometer (i.e., ACC data), and the second type data is obtained by a photoplethysmography (PPG) sensor (i.e., PPG data); or, the first type data is obtained by a PPG sensor (i.e., PPG data), and the second type data is obtained by an accelerometer (i.e., ACC data). Figure 7 In the illustrated embodiment, different types of sensor data can be used, such as ACC data and PPG data. The ACC sensor can monitor key body movement information during sleep, especially around the ear, reflecting head movement information unaffected by limb movements; the PPG sensor can monitor core physiological data of sleep, reflecting heart activity and the state of the cardiovascular system, which is closely related to sleep state. In other words, ACC data is body movement data, and PPG data is physiological data. Combining body movement data with physiological data for sleep quality assessment can effectively improve the accuracy of sleep quality assessment results.

[0103] Sensor data from wearable ear devices is processed to generate sleep data. Figure 7 In the illustrated embodiment, the first sleep data is generated based on the first sensor data, the second sleep data is generated based on the second sensor data, and the first and second sleep data are combined to generate the first comprehensive sleep data; the third sleep data is generated based on the third sensor data, the fourth sleep data is generated based on the fourth sensor data, and the third and fourth sleep data are combined to generate the second comprehensive sleep data. The first, second, third, and fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state at each time window within a sleep interval. The first, second, third, and fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data can be a sleep graph, i.e., a graph including both time and sleep state information, which can indicate the user's sleep state at different times. The meaning and setting method of sleep intervals, time windows, and sleep states can be... Figure 1 The embodiments are the same and will not be repeated here. In one embodiment, the first sleep data, the second sleep data, and the first comprehensive sleep data can be grouped using the same sleep state. In this case, the sleep staging methods for the first sleep data, the second sleep data, and the first comprehensive sleep data are the same, which facilitates calculation and comparison. Similarly, the third sleep data, the fourth sleep data, and the second comprehensive sleep data can be grouped using the same sleep state. In this case, the sleep staging methods for the third sleep data, the fourth sleep data, and the second comprehensive sleep data are the same, which facilitates calculation and comparison.

[0104] and Figure 6 Similar to the embodiments, in Figure 7 In this embodiment, the execution order of steps S701, S702, S703, and S704 is not explicitly limited. Step S701 may be executed before step S703, may be executed simultaneously with or at least partially overlap with step S703, or may be executed after step S703; step S702 may be executed before step S704, may be executed simultaneously with or at least partially overlap with step S704, or may be executed after step S704. The order of steps S701 with steps S702 and S704, and the order of steps S702 with steps S701 and S703, is also not explicitly limited. The execution order of steps S705 and S706 is also not explicitly limited. However, step S705 may be executed after steps S701 and S703, step S706 after steps S702 and S704, and step S707 after steps S705 and S706. Figure 7 The sleep quality assessment results in step S707 can be used with... Figure 1 , Figure 2 or Figure 6 The same method.

[0105] and Figure 6 The implementation examples are similar, Figure 7 The sleep quality assessment method shown can be executed on an ear-worn device, or partly on the ear-worn device and partly on a terminal device such as a mobile phone. In one embodiment, steps S701 to S707 are all executed on the first or second ear-worn device, wherein the data from the first and third sensors are sent from the first ear-worn device to the second ear-worn device, or the data from the second and fourth sensors are sent from the second ear-worn device to the first ear-worn device. In one embodiment, steps S701 and S702 are executed on the first ear-worn device, steps S703 and S704 are executed on the second ear-worn device, and steps S705, S706, and S707 are executed on the first or second ear-worn device. In one embodiment, steps S701 and S702 are executed on the first ear-worn device, steps S703 and S704 are executed on the second ear-worn device, and steps S705, S706, and S707 are executed on a control module (e.g., a Bluetooth control module) connected to both the first and second ear-worn devices. In one embodiment, steps S701 and S702 are executed on the first ear-wearing device, steps S703 and S704 are executed on the second ear-wearing device, steps S705 and S706 are executed on the control module connected to the first ear-wearing device, the second ear-wearing device, or both, and step S707 is executed on a terminal device such as a mobile phone.

[0106] exist Figure 7 In the embodiments, the same as can be used. Figure 1 The sleep quality assessment results are played or displayed in the same manner as in the embodiments.

[0107] exist Figure 7 In this embodiment, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared with two-channel sensor data, in addition to the two ear-worn devices complementing each other's sensor data, using four-channel sensor data can further improve the accuracy of sleep quality assessment results. This is because four-channel sensor data is richer in information than two-channel sensor data. More accurate assessment results can be obtained based on data with richer information. Moreover, four-channel sensor data can be set to different types of data, such as including first type data (e.g., ACC data) and second type data (e.g., PPG data), which can make full use of the characteristics of these different types of data to obtain more accurate assessment results.

[0108] Figure 8This is a flowchart illustrating the process of generating sleep data based on sensor data according to an embodiment of the present invention. Figure 8 As shown, the process of generating sleep data based on sensor data includes: in step S801, determining the sleep interval and the sensor data within that sleep interval; in step S802, filtering the sensor data within the sleep interval; in step S803, dividing the filtered sensor data into time windows; and in step S804, determining the sleep state for the sensor data in each time window. At this point, sleep data is generated based on the sensor data.

[0109] Figure 8 The process can be applied to Figure 6 Steps S601, S603 or Figure 7 Steps S701, S702, S703, and S704. For clarity, when describing specific sensor data or sleep data, the first sensor data and first sleep data in step S601 or S701, and the first comprehensive sleep data in step S605 or S705 will be used as examples, and the differences in other steps will be supplemented with explanations. These explanations can also be applied to other sensor data and / or sleep data. Figure 8 The process generates sleep data based on sensor data, therefore it can be applied not only to embodiments that first generate sleep data from sensors and then generate comprehensive sleep data based on that sleep data (such as...). Figure 6 and Figure 7 In addition to the embodiments described above, it can also be applied to embodiments that directly generate comprehensive sleep data from sensor data (such as...). Figure 3 and Figure 4 (Examples), for example, applied to Figure 3 Step S303 or Figure 4 Steps S403, S405, S408, and S409. Of course, when applied to... Figure 4 In steps S408 and S409, since the determination process of step S801, the filtering process of step S802, and the partitioning process of step S803 may have been performed before, only the sleep state determination process of step S804 needs to be performed.

[0110] In step S801, the sleep interval and the sensor data within that sleep interval are determined. As mentioned earlier, the sleep interval can be the time range from when the user wearing the ear-worn device begins to fall asleep to when they end their sleep, and can be determined in various ways. The reason for determining the sleep interval is that the start and end times of sensor data collection do not necessarily coincide with the start and end times of the sleep interval. For example, the sleep interval may be set by the user to 22:00 to 7:00 the next day. However, the user may actually put on the ear-worn device at 23:00 and go to bed, and wake up and remove the device at 7:30 the next day. In this case, the sensor data collection start time is 23:00, meaning no data is collected between 22:00 and 23:00. Since the user sets the end time of sleep to 7:00, no data may be collected between 7:00 and 7:30. In this scenario, sensor data from 11:00 PM to 7:00 AM can be used as the sensor data within the sleep interval, and the period from 10:00 PM to 11:00 PM within this sleep interval can be marked as a data-free period. This sensor data can be from the first and second sensors, or from the first to the fourth sensors. For example, the sleep interval is set by the user to be from 10:00 PM to 7:00 AM the next day. The user puts on the ear-worn device at 10:00 PM and goes to bed, with the first ear-worn device always worn, the second ear-worn device falling off between 6:00 AM and 7:00 AM the next day, and the user waking up at 7:00 AM the next day. In this scenario, sensor data from the first ear-wearing device between 22:00 and 7:00 the next day can be used as the sensor data within the sleep interval. This sensor data can be the first sensor data and / or the third sensor data. Sensor data from the second ear-wearing device between 22:00 and 6:00 the next day can be used as the sensor data within the sleep interval, and the period from 6:00 to 7:00 the next day is marked as a data-free period. This sensor data can be the second sensor data and / or the fourth sensor data.

[0111] Sleep intervals can be set by the system, for example, by setting a fixed value (e.g., always set to 22:00 to 7:00 the next day), by setting different periods of time (e.g., different settings according to the season), by automatically determining them based on sleep data monitoring within a certain period, or even by automatically determining them based on sleep data after each sleep period. Sleep intervals can also be set by the user, for example, by setting a fixed value, by setting different periods of time, or by setting them before or after each sleep period. For example, setting them by the user after each sleep period or by automatic determination can cover the user's actual sleep time and ensure the integrity of the sensor-collected data as much as possible.

[0112] Sleep intervals can be set based on users' sleep habits, ensuring that these intervals cover the user's typical sleep duration. This makes it easy to determine the time intervals for data processing when assessing sleep quality. For ease of processing, the same sleep interval can be set for all sensor data that needs to be processed. As an example, in Figure 1 or Figure 6 In one embodiment, the same sleep interval can be set for the data from the first and second sensors, for example, 22:00 to 7:00 the next day. Figure 2 or Figure 7 In one embodiment, the same sleep interval can be set for the data from the first to the fourth sensors, or a sleep interval can be set for the first and second sensor intervals and another sleep interval can be set for the third and fourth sensor data. In the latter case, when determining the sleep quality assessment result in step S207 or S707, it may be necessary to consider the differences in the sleep intervals of the first comprehensive sleep data and the second comprehensive sleep data, for example, to unify them into the same interval for subsequent processing.

[0113] In step S802, the sensor data within the sleep interval is filtered. For example, a Butterworth filter can be used to filter the sensor data to remove noise interference. The sensor data here may include data from the first and second sensors, or data from the first to the fourth sensors; the same applies to subsequent steps, and will not be elaborated further. For the data-free periods within the sleep interval, since there is no data, they are not processed in step S802 and subsequent steps.

[0114] In step S803, the filtered sensor data is divided according to time windows. The time window can be the time granularity for processing the sensor data, for example, each time window can be 30 seconds or other durations, and the time windows usually do not overlap.

[0115] In step S804, the sleep state is determined based on the sensor data for each time window. The sleep state can be set to match... Figure 1 The embodiments are the same.

[0116] In one embodiment, when determining sleep state based on sensor data for each time window, time windows where sleep state cannot be determined can be identified in a relatively simple way. These time windows may be unable to determine sleep state through subsequent calculations due to reasons such as weak signals. For example, the approximate signal-to-noise ratio (SNR) can be calculated for the filtered sensor data of each time window. If the approximate SNR of a time window is less than a threshold t, the time window is marked as "uncertain," so that it is not calculated and judged again when determining sleep state in subsequent calculations, thus saving overall computational load. In another embodiment, this "uncertain" state judgment can be performed uniformly on all sensor data instead of separately.

[0117] Different calculation methods can be used to determine sleep states for different types of sensor data. The following explanation assumes that an "uncertain" state is determined first, meaning that time windows with an "uncertain" state are excluded from the calculations and determinations below. If an "uncertain" state is not determined first, the "uncertain" state needs to be added to the sleep state determination result.

[0118] For PPG data, endpoint detection can be performed on sensor data for each time window to calculate PRV (pulse rate variability) feature data. Then, a machine learning model (such as Gradient Boosting Decision Tree (GBDT)) can be used to determine sleep state. As mentioned earlier, sleep states in PPG data can include, for example, wakefulness, REM sleep, sleep stage N1, sleep stage N2, and sleep stage N3, and sometimes also include uncertain states. For ACC data, time-domain features (such as mean, variance, zero-crossing rate, etc.) and / or frequency-domain features (amplitude, frequency, mean, etc. after Fourier transform) can be extracted from sensor data for each time window. Then, a machine learning model (such as GBDT) can be used to determine sleep state. Sleep states in ACC data can include wakefulness and sleep, and sometimes also include uncertain states.

[0119] Sleep state is determined by sensor data for each time window, and corresponding sleep data is generated based on the sensor data. Specifically, first sleep data is generated based on first sensor data, second sleep data is generated based on second sensor data, third sleep data is generated based on third sensor data, and fourth sleep data is generated based on fourth sensor data.

[0120] In one embodiment, the process of generating sleep data based on sensor data may not include the filtering in step S802, or in addition to the filtering in step S802, it may include amplification, normalization and other processing before step S803 to facilitate subsequent processing.

[0121] exist Figure 8 The process is applied to Figure 6 In steps S601 and S603, the division of the first sensor data and the division of the second sensor data in step S803 typically requires aligning the time windows. This ensures that when the first and second sensor data are integrated in step S605, the data in their respective time windows are more comparable and complementary. If the time windows for the first and second sensor data are not aligned, the data in their respective time windows actually target different sleep time intervals of the user, making the data incomparable. Using one to supplement the other may lead to errors. This alignment can be achieved by setting the same sleep interval or the same time base for different sensor data. For example, when dividing the sensor data, it can start from the beginning of the sleep interval determined in step S801 and divide it sequentially according to the time windows. If the starting point of the sleep interval is the same, subsequent time windows can be aligned. For periods with no data in the sleep interval, their markers can be retained without dividing them, as dividing them may be meaningless. When a time window contains data only for a portion of its duration due to periods without data (e.g., a 30-second window with data for the first 20 seconds and none for the last 10), this time window can be simply marked as having no data (empty data) for easier processing. Assuming the start of the data-free period is actually at the 20th second of this time window, if the window is marked as data-free, the start of that data-free period can be moved forward by 20 seconds to align with the boundaries (start and end) of the time window. Conversely, when only a portion of a time window contains data (e.g., the duration of the data-containing portion is longer than the duration of the data-free portion, or the signal-to-noise ratio of the time window is greater than the average signal-to-noise ratio of the overall data), the data-containing portion can be used as the entire data for that time window. In this case, the start of the data-free period can be moved forward or backward based on its positional relationship with the time window to avoid the data-free period.

[0122] Figure 9 This is a flowchart illustrating the process of generating comprehensive sleep data based on sleep data according to an embodiment of the present invention. Figure 9 In China, with Figure 6 Step S605 or Figure 7 The following explanation uses step S705, which generates first comprehensive sleep data based on first sleep data and second sleep data, as an example. Those skilled in the art will understand that... Figure 9 The same process applies to Figure 7In step S706, simply replace the first sleep data, the second sleep data, and the first comprehensive sleep data with the third sleep data, the fourth sleep data, and the second comprehensive sleep data.

[0123] like Figure 9 As shown, the process of generating first comprehensive sleep data based on first sleep data and second sleep data includes: in step S901, determining a first ear-worn device as the main analysis device; in step S902, determining whether a time period is a data-free period for the first sleep data; if the determination result of step S902 is "yes", in step S903, using the second sleep data to determine the first comprehensive sleep data; if the determination result of step S902 is "no", in step S904, determining whether there is second sleep data in the time period; if the determination result of step S904 is "no", in step S905, using the first sleep data to determine the first comprehensive sleep data; if the determination result of step S904 is "yes", in step S906, determining whether the sleep state of the first sleep data in a time window is uncertain; if the determination result of step S906 is "yes", in step S907, using the second sleep data to determine the first comprehensive sleep data; if the determination result of step S906 is "no", in step S908, determining whether the sleep state of the first sleep data in a time window is uncertain; if the determination result of step S907 is "yes", in step S908, determining whether the sleep state of the first sleep data in a time window is uncertain; if the determination result of step S908 is "yes", in step S909, determining whether the sleep state of the first sleep data in a time window is uncertain; if the determination result of step S908 is "yes", in step S909, determining whether the sleep state of the first sleep data in a time window is uncertain; if the determination result of step S909 ... 908. Determine whether the sleep states of the first sleep data and the second sleep data are the same within this time window. If the determination result of step S908 is "yes", in step S909, use the first sleep data to determine the first comprehensive sleep data. If the determination result of step S908 is "no", in step S910, use the first sleep data to determine the first comprehensive sleep data, or use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within this time window to determine the first comprehensive sleep data. After steps S907, S909, and S910, in step S911, determine whether all time windows within this time period have been processed. If not, return to step S906 to continue processing; if already processed, continue to step S912. If the determination result of step S911 is "yes" or after steps S903 and S905, in step S912, determine whether all time periods have been processed. If not, return to step S902 to continue processing; if already processed, the process ends. The following provides a detailed explanation of each step.

[0124] In step S901, the first ear-worn device is determined as the primary analysis device. By determining the primary analysis device, sleep data from the primary analysis device can be used as the main source, supplemented by sleep data from secondary analysis devices, simplifying the processing and reducing the computational load. The first ear-wearing device is designated as the primary analysis device in the following situations: the average signal-to-noise ratio (SNR) of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device. In this case, the overall data quality of the first ear-wearing device is higher than that of the second ear-wearing device, thus it is designated as the primary analysis device. This applies to two-channel or four-channel sensor data. Alternatively, for the case of using four-channel sensor data, the average SNR of the first sensor data from the first ear-wearing device is higher than that of the second sensor data from the second ear-wearing device, and the average SNR of the third sensor data from the first ear-wearing device is higher than that of the fourth sensor data from the second ear-wearing device. Or, the length of the data-free period in the first sleep data (or the total length of multiple data-free periods if there are multiple data-free periods) is shorter than the length of the data-free period in the second sleep data. In this case, the normal monitoring time of the first ear-wearing device is longer than that of the second ear-wearing device. This applies to two-channel or four-channel sensor data. In other cases, the user may also set either the first or second ear-wearing device as the primary analysis device; this invention is not limited to these limitations. Figure 2 The main analytical devices for steps S205 and S206 may differ, similar to the situation where... Figure 9 The process applied to Figure 7 In steps S705 and S706, the main analysis device may be different; that is, different ear-worn devices may be used as the main analysis device for generating the first and second comprehensive sleep data. For example, the first and second sensor data are PPG data, and the third and fourth sensor data are ECG data. If the average signal-to-noise ratio of the first sensor data is higher than the average signal-to-noise ratio of the second sensor data, then when determining the first comprehensive sleep data, the first ear-worn device is determined as the main analysis device, and the first sensor data and the first sleep data are determined as the main analysis data. Simultaneously, if the average signal-to-noise ratio of the fourth sensor data is higher than the average signal-to-noise ratio of the third sensor data, then when determining the second comprehensive sleep data, the second ear-worn device is determined as the main analysis device, and the fourth sensor data and the fourth sleep data are determined as the main analysis data.

[0125] In step S902, it is determined whether a time period is a data-free period for the first sleep data. The determination in step S902 can be performed sequentially by time period within the sleep interval, or sequentially by time period type. When the first ear-worn device is designated as the primary analysis device, the first sleep data, as the primary analysis data, forms the basis of the analysis. Therefore, the sleep interval can be divided into data-free periods for the first sleep data, periods containing only the first sleep data, and periods where both the first and second sleep data exist. Then, processing can be performed sequentially by time period or by these time period types.

[0126] If step S902 determines that a time period is a data-free period for the first sleep data, then step S903 is executed, that is, the second sleep data is used to determine the first comprehensive sleep data during this time period. For example, if the data-free period for the first sleep data is from 22:00 to 23:00, then the data from the second sleep data between 22:00 and 23:00 is used as the data for the first comprehensive sleep data between 22:00 and 23:00.

[0127] Next, the time periods containing the first sleep data within the sleep interval (i.e., the normal monitoring time of the first ear-worn device) are processed. If, in step S902, it is determined that a time period is not a data-free period for the first sleep data, meaning that the first sleep data exists in that time period, then step S904 is executed to further determine whether the second sleep data exists in that time period. If the second sleep data does not exist in that time period, meaning that only the first sleep data exists, then step S905 is executed, meaning that the first sleep data is used in that time period to determine the first comprehensive sleep data.

[0128] Starting from step S906, the time period in which both first and second sleep data exist simultaneously within the sleep interval is processed. This processing can be performed sequentially, window by window, according to a certain order. This order could be, for example, the chronological order of the time windows within the processed time period. In step S906, it is determined whether the sleep state of the first sleep data within a time window is uncertain. If the determination result of step S906 is "yes," then step S907 is executed. In step S907, the second sleep data is used to determine the first comprehensive sleep data; that is, the sleep state of the second sleep data within that time window is used as the sleep state of the first comprehensive sleep data within that time window. If the determination result of step S906 is "no," then step S908 is executed. In step S908, it is determined whether the sleep states of the first and second sleep data are the same within that time window.

[0129] If the judgment result of step S908 is "yes", then step S909 is executed. In step S909, the first sleep data is used to determine the first comprehensive sleep data, that is, the sleep state of the first sleep data in this time window is used as the sleep state of the first comprehensive sleep data in this time window. Although the first sleep data is used to determine the first comprehensive sleep data in step S909, since the prerequisite for executing step S909 is that the first sleep data and the second sleep data are in the same judgment time window, it can also be said that the second sleep data is used here, which does not affect the result of step S909. The reason for using the first sleep data in step S909 is, on the one hand, for ease of practical operation, and on the other hand, to save computation. For example, in actual operation, the first sleep data, which is the main analysis data, is first copied directly to the temporary storage address of the first comprehensive sleep data, and then the second sleep data is used to make partial adjustments to the first sleep data to obtain the final first comprehensive sleep data.

[0130] If the judgment result of step S908 is "no", then step S910 is executed. In step S910, the first sleep data can be used to determine the first comprehensive sleep data, that is, the sleep state of the first sleep data in this time window is used as the sleep state of the first comprehensive total sleep data in this time window. The reason for using the first sleep data is that when the first ear wearable device is determined to be the main analysis device in step S901, the first sleep data from it can be used as the main analysis data because it may be more accurate overall.

[0131] In step S910, the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within the time window can also be used to determine the first comprehensive sleep data. For example, if the feature value of the first sensor data is higher within the time window, the sleep state of the first sleep data corresponding to the first sensor data within that time window is used as the sleep state of the first comprehensive sleep data within that time window; if the feature value of the second sensor data is higher within the time window, the sleep state of the second sleep data corresponding to the second sensor data within that time window is used as the sleep state of the first comprehensive sleep data within that time window. Here, the feature value can be, for example, the signal-to-noise ratio, or other indicators that can evaluate data quality.

[0132] When the first ear-worn device is determined to be the main analysis device, the corresponding first sleep data of the main analysis device is used as the basis. Missing data (such as data from periods without data in the first sleep data, processed in step S903) and / or inaccurate data (such as data indicating an uncertain sleep state, processed in step S907; data indicating inconsistent sleep states, processed in step S910) are supplemented using second sleep data. This makes the synthesized first sleep data more complete and accurate compared to either unsynthesized first sleep data or unsynthesized second sleep data. Similar to the previous description, periods without data in the first sleep data refer to those periods within the sleep interval where no first sleep data is available due to missing first sensor data. An uncertain sleep state refers to a situation where the sensor data signal is too weak within a time window (e.g., the approximate signal-to-noise ratio is less than a threshold t, where t can be a predetermined value) to determine the sleep state through calculation.

[0133] After steps S907, S908, and S910, step S911 is executed, which determines whether all time windows within the current time period have been processed. If not, the process returns to step S906 to continue processing; if processed, the process continues to step S912. If the result of step S911 is "yes" or after steps S903 and S905, step S912 is executed, which determines whether all time periods have been processed. If not, the process returns to step S902 to continue processing; if processed, the process ends.

[0134] Figure 9 The embodiment generates comprehensive sleep data based on two-channel sleep data, that is, by combining the sleep data from the two channels, the sleep data from the two channels can complement each other, thereby making the comprehensive sleep data more complete and accurate.

[0135] Figure 10 This is a schematic diagram of multi-channel sleep data within a sleep interval according to an embodiment of the present invention. In the following description, first and second sleep data are used as examples of multi-channel sleep data. Those skilled in the art will understand that third and fourth sleep data are also applicable in other embodiments. For clarity and simplicity, the term "time period" is sometimes omitted and directly represented by letters.

[0136] Figure 10 The data for the first and second sleep periods within sleep interval T are shown. For example... Figure 10As shown, for the first sleep data, within the sleep interval T, there is a data-containing time period T1 (T1 = A + B + C) and a data-free time period T2 (T2 = D + E); for the second sleep data, within the sleep interval T, there is a data-containing time period T1' (T1' = A), a data-free time period T2' (T2' = B), a data-containing time period T3' (T3' = C + D), and a data-free time period T4' (T4' = E). Although Figure 10 The diagram shows that both the first and second sleep data have periods of no data within the sleep interval T. However, those skilled in the art will understand that in other embodiments, the first and / or second sleep data may not have periods of no data within the sleep interval T. Figure 10 The rectangles representing data periods only indicate that data was available within those periods, not that all data values ​​were the same. Sleep data can have different values ​​at different times. Similarly, the two rectangles representing the second sleep data only indicate that they represent different time periods, not that the data values ​​are higher or lower.

[0137] Based on the simultaneous presence of first and second sleep data, the time periods within sleep interval T are as follows: Time periods A and C contain both first and second sleep data; time period B contains only first sleep data; time period D contains only second sleep data; and time period E contains neither first nor second sleep data. In specific processing, some of these time periods can be merged based on the data used as the basis for analysis.

[0138] Figure 10 Sleep data can be applied to Figure 9 Examples of implementations. Applied to... Figure 9 In the embodiment, for the data-free time period T2 (=D+E) of the first sleep data, in step S903, the data of the second sleep data in this time period (including the second sleep data in time period D and the empty data in time period E) is used as the first comprehensive sleep data. For the data-rich time period T1 (=A+B+C) of the first sleep data, it can be processed in two ways: if there is no second sleep data, that is, for the time period (B) where only the first sleep data exists, the first sleep data is used to determine the first comprehensive sleep data in step S905; if there is second sleep data, that is, for the time periods (A and C) where both the first and second sleep data exist, the first comprehensive sleep data is determined by processing the first and second sleep data one time window at a time (steps S906 to S911). Figure 10 The processing order of each time period and each time window can be adopted as follows: Figure 5 The same order can be used, but other orders can also be used.

[0139] Figure 10 The diagram shows four time windows, W1, W2, W3, and W4. In time window W1, the sleep state of the first sleep data is "uncertain." In time windows W2, W3, and W4, the sleep state of the first sleep data is not "uncertain." Specifically, in time window W2, the sleep state of the first sleep data is the same as that of the second sleep data; in time window W3, the feature value of the first sensor data is lower than that of the second sensor data; and in time window W4, the feature value of the first sensor data is higher than that of the second sensor data.

[0140] For time window W1, in step S906, the sleep state of the first sleep data is determined to be "uncertain". Therefore, step S907 is executed to use the second sleep data in time window W1 as the first comprehensive sleep time.

[0141] For time window W2, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are the same. Therefore, step S909 is executed, and the first sleep data in time window W2 is used as the first comprehensive sleep data.

[0142] For time window W3, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are different. Therefore, step S910 is executed to use the first sleep data to determine the first comprehensive sleep data, or to use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in time window W3 (i.e., the second sleep data) to determine the first comprehensive sleep data.

[0143] For time window W4, in step S906 it is determined that the sleep state of the first sleep data is not "uncertain" and in step S908 it is determined that the sleep states of the first sleep data and the second sleep data are different. Therefore, step S910 is executed to use the first sleep data to determine the first comprehensive sleep data, or to use the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data in time window W4 (i.e., the first sleep data) to determine the first comprehensive sleep data.

[0144] Figure 11 This is a flowchart illustrating the process of determining sleep quality assessment results according to an embodiment of the present invention. Figure 11As shown, the process of determining the sleep quality assessment result includes: in step S1101, normalizing the first comprehensive sleep data; in step S1102, modifying the normalized first comprehensive sleep data to a standard size; and in step S1103, obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data through deep learning modeling.

[0145] Figure 11 The process can be applied to Figure 1 Step S107 and Figure 6 Step S607 will be described in detail below.

[0146] In step S1101, the first comprehensive sleep data is normalized, for example, by the following method:

[0147]

[0148] Where, x new This represents the first comprehensive sleep data after normalization, x = [x1, x2, x3, ..., x n [This represents the first comprehensive sleep data, x1 to x] n denoted by , respectively, represents the sleep state of the first comprehensive sleep data in each time window, n represents the data length of the first comprehensive sleep data, μ represents the average value of the first comprehensive sleep data, and σ represents the variance of the first comprehensive sleep data.

[0149] In step S1102, the normalized first comprehensive sleep data is modified to a standard size. Since everyone's sleep time is different, in subsequent processing, for example considering model requirements, it is necessary to convert it to the same data length. This can be done, for example, by using an interpolation algorithm to modify the data to a standard size, the size of which is determined by the model.

[0150] In step S1103, sleep quality assessment results are obtained based on the first comprehensive sleep data of standard size through deep learning modeling. The sleep quality assessment results include difficulty falling asleep, normal sleep, insomnia, or light sleep. Through deep learning modeling, the data enters a deep learning clustering model to obtain the sleep type. The model can be an unsupervised clustering algorithm. During model training, sleep data type modeling can be completed end-to-end without labels, aiming to increase the distance between different categories of data and decrease the distance between data of the same category until convergence. The trained model can be directly used for application testing to judge the sleep quality assessment results of the data, such as difficulty falling asleep, normal sleep, insomnia, light sleep, etc. Figure 1As described in the embodiments, the sleep quality assessment results can be played to the user via voice on the ear-wearing device, or displayed to the user on a mobile phone or other terminal device, so that the user can understand their sleep quality.

[0151] Figure 12 This is a flowchart illustrating the process of determining sleep quality assessment results according to an embodiment of the present invention. Figure 12 As shown, the process of determining the sleep quality assessment result includes: in step S1201, normalizing the first comprehensive sleep data and the second comprehensive sleep data respectively; in step S1202, modifying the normalized first comprehensive sleep data and the second comprehensive sleep data to a standard size; in step S1203, obtaining the sleep quality assessment result based on the standard-sized first comprehensive sleep data and the second comprehensive sleep data through deep learning modeling.

[0152] Figure 12 The process can be applied to Figure 2 Step S207 and Figure 7 Step S707 will be described in detail below.

[0153] In step S1201, the first comprehensive sleep data and the second comprehensive sleep data are normalized respectively, for example, by the following method:

[0154]

[0155] Where, x new This represents the normalized first or second composite sleep data, x = [x1, x2, x3, ..., x...]. n [This represents the first comprehensive sleep data, x1 to x] n These represent the sleep states of the first comprehensive sleep data in each time window, respectively, where n represents the data length of the first comprehensive sleep data, and x = [x1, x2, x3, ..., x...]. m [This represents the second comprehensive sleep data, x1 to x] m The numbers represent the sleep states of the second comprehensive sleep data in each time window, m represents the data length of the second comprehensive sleep data, m and n can be the same or different, μ represents the average value of the first or second comprehensive sleep data, and σ represents the variance of the first or second comprehensive sleep data.

[0156] In step S1202, the normalized first and second comprehensive sleep data are modified to a standard size. Since each person's sleep time is different, in subsequent processing, for example considering model requirements, it is necessary to convert them to the same data length. This can be done, for example, by using an interpolation algorithm to modify the data to a standard size, the size of which is determined by the model.

[0157] In step S1203, sleep quality assessment results are obtained through deep learning modeling based on standard-sized first and second comprehensive sleep data. These results include difficulty falling asleep, normal sleep, insomnia, or light sleep. Through deep learning modeling, the data is fed into a deep learning clustering model to determine the sleep type. The model can be an unsupervised clustering algorithm. During model training, the first and second comprehensive sleep data can be fused without labels to complete end-to-end modeling of sleep data types. The goal is to increase the distance between different categories of data and decrease the distance between data of the same category until convergence. The trained model can then be directly used for application testing to determine the sleep quality assessment results of the data, such as difficulty falling asleep, normal sleep, insomnia, light sleep, etc. Figure 1 As described in the embodiments, the sleep quality assessment results can be played to the user via voice on the ear-wearing device, or displayed to the user on a mobile phone or other terminal device, so that the user can understand their sleep quality.

[0158] Figure 13 This is a flowchart of a sleep quality assessment method according to an embodiment of the present invention. Figure 13 As shown, in step S1301, first comprehensive sleep data is received. In step S1302, second comprehensive sleep data is received. In step S1303, a sleep quality assessment result is determined based on the first comprehensive sleep data and the second comprehensive sleep data. The first comprehensive sleep data may be generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, where the first and second sensor data are first-type data. The second comprehensive sleep data may be generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, where the third and fourth sensor data are second-type data.

[0159] Figure 13 The data in the embodiments can be the same as those in the foregoing embodiments. Figure 13 The embodiments can be executed on terminal devices such as mobile phones. In this case, the terminal device can receive first and second integrated sleep data from the first and / or second ear-wearing devices, for example, via Bluetooth, and process the received data. Execution on the terminal device Figure 13 The sleep quality assessment method can make full use of the processing power of terminal devices, which is significantly higher than that of ear-worn devices (for example, deep learning or artificial intelligence clustering algorithms can be used), to obtain more accurate sleep quality assessment results and facilitate the subsequent presentation of the assessment results to users.

[0160] and Figure 2 and Figure 7 Similar to the previous embodiment, sleep quality assessment is performed based on four-channel sensor data (i.e., first to fourth sensor data) from two ear-worn devices. Compared with two-channel sensor data, the accuracy of sleep quality assessment results can be further improved. Moreover, the four-channel sensor data can be set to different types of data, which can make full use of the characteristics of these different types of data to obtain more accurate assessment results.

[0161] The sleep quality assessment method and its specific process of the present invention have been described above. Those skilled in the art will understand that some or all of these methods and processes can be fixed to ear wearable devices, mobile phones or other devices by means of software, firmware or hardware, and can be run when needed to perform the methods and / or processes.

[0162] In one embodiment, the present invention discloses a sleep quality assessment device, which includes a module for performing the above-described method steps. Similar to the method described above, this sleep quality assessment device can improve the accuracy of sleep quality assessment results.

[0163] Although the technical solutions of the present invention have been described in detail above with two-channel data, four-channel data and two types of data, those skilled in the art will understand that other numbers of multi-channel data (e.g., six-channel data, eight-channel data, etc.) and more types of data (e.g., three types of data, four types of data, etc.) can also be used.

[0164] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

[0165] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be interpreted as indicating or implying relative importance.

Claims

1. A method for assessing sleep quality, comprising: First comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device; and The sleep quality assessment result is determined based on the first comprehensive sleep data.

2. The sleep quality assessment method according to claim 1, wherein, The generation of the first comprehensive sleep data based on first sensor data from the first ear-wearing device and second sensor data from the second ear-wearing device includes: First sleep data is generated based on the first sensor data from the first ear-wearing device; Second sleep data is generated based on the second sensor data from the second ear-wearing device; and First comprehensive sleep data is generated based on the first sleep data and the second sleep data.

3. The sleep quality assessment method according to claim 1 or 2 further includes: A second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, wherein... Determining the sleep quality assessment result based on the first comprehensive sleep data includes: determining the sleep quality assessment result based on the first comprehensive sleep data and the second comprehensive sleep data.

4. The sleep quality assessment method according to claim 3, wherein, The second comprehensive sleep data is generated based on third sensor data from the first ear-wearing device and fourth sensor data from the second ear-wearing device, including: Third sleep data is generated based on third sensor data from the first ear-wearing device; Fourth sleep data is generated based on fourth sensor data from the second ear-wearing device; and A second comprehensive sleep data is generated based on the third sleep data and the fourth sleep data.

5. The sleep quality assessment method according to claim 3, wherein, The first sensor data and the second sensor data are first type data, and / or the third sensor data and the fourth sensor data are second type data.

6. The sleep quality assessment method according to claim 4, wherein, The first sleep data, the second sleep data, the third sleep data, the fourth sleep data, the first comprehensive sleep data, and the second comprehensive sleep data each indicate the sleep state of each time window in the sleep interval.

7. The sleep quality assessment method according to claim 6, wherein, The sleep state is selected from at least one of the following groups or from at least one of the following groups after adding an uncertain state: wakefulness and sleep; wakefulness, REM sleep, sleep stage N1, sleep stage N2 and sleep stage N3; wakefulness, REM sleep, light sleep and deep sleep; or wakefulness, REM sleep and non-REM sleep.

8. The sleep quality assessment method according to claim 1, wherein, The first ear-wearing device and the second ear-wearing device are a pair of earphones, one of which is the left earphone and the other is the right earphone.

9. The sleep quality assessment method according to claim 5, wherein, The first type of data is obtained through an accelerometer, and the second type of data is obtained through a photoplethysmography (PPG) sensor; or, the first type of data is obtained through a PPG sensor, and the second type of data is obtained through an accelerometer.

10. The sleep quality assessment method according to claim 2, wherein, Generating first sleep data based on the first sensor data from the first ear-wearing device includes: Determine the sleep interval and acquire the first sensor data within the sleep interval. The first sensor data within the sleep interval is filtered. The filtered first sensor data is divided according to time windows, and Sleep state is determined using the first sensor data for each time window; and Generating second sleep data based on the second sensor data from the second ear-wearing device includes: Determine the sleep interval and acquire the second sensor data within the sleep interval. The data from the second sensor within the sleep interval is filtered. The filtered second sensor data is divided according to the aforementioned time window, and Sleep state is determined using second sensor data for each time window.

11. The sleep quality assessment method according to claim 4, wherein, The third sleep data generated based on the third sensor data from the first ear-wearing device includes: Determine the sleep interval and acquire third sensor data within the sleep interval. The data from the third sensor within the sleep interval is filtered. The filtered third sensor data is divided according to time windows, and Sleep state is determined using third sensor data for each time window; and The fourth sleep data generated based on the fourth sensor data from the second ear-wearing device includes: Determine the sleep interval and acquire the fourth sensor data within the sleep interval. The data from the fourth sensor within the sleep interval is filtered. The filtered fourth sensor data is divided according to time windows, and Sleep state is determined using fourth sensor data for each time window.

12. The sleep quality assessment method according to claim 2, wherein, The first sleep data includes time periods with no data and / or time windows where sleep states are uncertain. Generating the first comprehensive sleep data based on the first sleep data and the second sleep data includes: For periods of no data in the first sleep data, the second sleep data is used to determine the first comprehensive sleep data; and / or For time windows where the sleep state of the first sleep data is uncertain, the second sleep data is used to determine the first comprehensive sleep data.

13. The sleep quality assessment method according to claim 2, wherein, The first sleep data and the second sleep data each indicate the sleep state at each time window within the sleep interval. Generating the first comprehensive sleep data based on the first sleep data and the second sleep data includes: If the first sleep data and the second sleep data show the same sleep state within a time window, then the data from the first sleep data within that time window is used to determine the first comprehensive sleep data; and / or If the sleep states of the first sleep data and the second sleep data are different within a time window, the sleep data corresponding to the higher feature value of the first sensor data and the second sensor data within the time window is used to determine the first comprehensive sleep data, or the data of the first sleep data within the time window is used to determine the first comprehensive sleep data.

14. The sleep quality assessment method according to claim 1, wherein, The period of missing data from the first sensor within the sleep interval is shorter than the period of missing data from the second sensor within the same sleep interval. The generation of first comprehensive sleep data based on the first sensor data from the first ear-wearing device and the second sensor data from the second ear-wearing device includes: For time periods within the sleep interval where the first sensor data is missing, the second sensor data is extracted, and the first comprehensive sleep data is generated based on the first sensor data and the extracted second sensor data; or The first comprehensive sleep data is generated based on the data from the first sensor.

15. The sleep quality assessment method according to claim 1, wherein, The first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device, including: If the feature value of the first sensor data in a time window is higher than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the first sensor data in that time window is used to determine the first comprehensive sleep data. If the feature value of the first sensor data in a time window is lower than the feature value of the second sensor data in the same time window, then the sleep data corresponding to the second sensor data in that time window is used to determine the first comprehensive sleep data.

16. The sleep quality assessment method according to claim 12 or 13, wherein, The average signal-to-noise ratio of the first sensor data is higher than that of the second sensor data, or the length of the data-free period of the first sleep data is shorter than that of the data-free period of the second sleep data.

17. The sleep quality assessment method according to claim 13 or 15, wherein, The characteristic value is the signal-to-noise ratio.

18. The sleep quality assessment method according to claim 1, wherein, The sleep quality assessment results determined based on the first comprehensive sleep data include: The first comprehensive sleep data is normalized; The normalized first composite sleep data was modified to standard size; and The sleep quality assessment result is obtained by modeling with deep learning based on the first comprehensive sleep data of the standard size.

19. The sleep quality assessment method according to claim 3, wherein, The sleep quality assessment result determined based on the first comprehensive sleep data and the second comprehensive sleep data includes: The first comprehensive sleep data and the second comprehensive sleep data are normalized respectively; The normalized first and second composite sleep data were modified to standard size; and The sleep quality assessment result is obtained by modeling using deep learning, based on the first and second comprehensive sleep data of the standard size.

20. The sleep quality assessment method according to claim 1, wherein, The sleep quality assessment results include difficulty falling asleep, normal sleep, insomnia, or light sleep.

21. A method for assessing sleep quality, comprising: Receive the first comprehensive sleep data; Receive second comprehensive sleep data; and Based on the first comprehensive sleep data and the second comprehensive sleep data, a sleep quality assessment result is determined, wherein... The first comprehensive sleep data is generated based on first sensor data from a first ear-wearing device and second sensor data from a second ear-wearing device. The first sensor data and the second sensor data are of type first data. The second comprehensive sleep data is generated based on third sensor data from the first ear wearable device and fourth sensor data from the second ear wearable device, wherein the third sensor data and the fourth sensor data are second type data.

22. A sleep quality assessment device comprising a module for performing the method steps of any one of claims 1 to 21.

Images