Electronic device and sleep management system including same

Abstract

The present invention relates to an electronic device for managing a sleep state of a user and a sleep management system including same. The electronic device of the present invention comprises: a plurality of sensors for detecting biometric signals of a user; an output interface for outputting at least one of video and audio; and a processor for determining a sleep stage of the user on the basis of the biometric signals of the user detected by the plurality of sensors, and controlling the output interface such that the output amount of the output interface is reduced at a preconfigured speed on the basis of the determined sleep stage.

Description

Electronic devices and sleep management systems including them

[0001] The disclosed invention relates to an electronic device for managing a user's sleep state and a sleep management system including the same.

[0002] People read books or watch content like e-books, music, movies, or dramas before bed. They use electronic devices like television to watch content. Some people even fall asleep without turning off their electronic devices after watching content before bed.

[0003] When people fall asleep without turning off their electronic devices, their bodies become tense from the electromagnetic waves, heat, and light they generate. This causes problems, making it difficult for people to get a good night's sleep.

[0004] Additionally, when people fall asleep without turning off their electronic devices, they continue to operate while they sleep. This leads to unnecessary power consumption within the home.

[0005] One aspect of the disclosed invention provides an electronic device that recognizes a user's sleep stage and controls an output interface to create a sleep-inducing environment based on the recognized sleep stage, and a sleep management system including the same.

[0006] An electronic device according to one aspect includes a plurality of sensors for detecting a user's bio-signals; an output interface for outputting at least one of video and audio; and a processor for determining a user's sleep stage based on the user's bio-signals detected by the plurality of sensors and controlling the output interface such that an output amount of the output interface is reduced at a preset rate based on the determined sleep stage.

[0007] The preset speed of the electronic device according to one aspect includes a first reference speed and a second reference speed that is faster than the first reference speed. The processor of the electronic device according to one aspect controls the output interface so that the output amount is reduced to the first reference speed or the second reference speed based on whether the sleep stage is determined to be a light sleep stage.

[0008] A processor of an electronic device according to one aspect determines a rate of decrease in output as a first reference rate based on the sleep stage being determined as stage 1 of a light sleep stage, and determines a rate of decrease in output as a second reference rate based on the sleep stage being determined as stage 2 of a light sleep stage.

[0009] A processor of an electronic device according to one aspect determines a rate of decrease in output as a first reference rate based on a change in sleep stage from stage 2 to stage 1.

[0010] A processor of an electronic device according to one aspect determines whether there is a tendency to wake up based on a bio-signal when the sleep stage is stage 1, determines a rate of decrease in output at a rate slower than a first reference rate based on the presence of the tendency to wake up, and determines a rate of decrease in output at a rate faster than the first reference rate based on the absence of the tendency to wake up.

[0011] A processor of an electronic device according to one aspect determines whether there is a tendency to wake up based on a bio-signal when the sleep stage is stage 2, determines a rate of decrease in output at a rate slower than a second reference rate based on the presence of the tendency to wake up, and determines a rate of decrease in output at a rate faster than the second reference rate based on the absence of the tendency to wake up.

[0012] A processor of an electronic device according to one aspect controls a decrease in the output amount of an output interface based on a change in the sleep stage from stage 1 to the wake stage.

[0013] A processor of an electronic device according to one aspect controls an output interface to be turned off based on whether the sleep stage is determined to be a deep sleep stage.

[0014] According to one aspect, the processor of the electronic device controls the output interface so that the output amount of the output interface is lower than a preset output amount in response to the sleep stage being recognized as a light sleep stage. The preset output amount of the electronic device according to one aspect is set by the average value of the output amount of the output interface during the wake stage.

[0015] An output interface of an electronic device according to one aspect includes at least one of a display unit for outputting video and a speaker for outputting audio. An output amount of the output interface of the electronic device according to one aspect includes at least one of a video brightness amount and an audio volume amount.

[0016] An electronic device according to one aspect further includes a communication unit that communicates with at least one external sensor that detects a user's biosignal. A processor of the electronic device according to one aspect determines the user's sleep stage based on the biosignal detected by the at least one external sensor and the biosignals detected by a plurality of sensors.

[0017] The plurality of sensors of the electronic device according to one aspect include at least two of a microphone, a camera, and a wireless radar sensor. The at least one external sensor of the electronic device according to one aspect includes at least one of a microphone, a camera, a heart rate sensor, an electrocardiogram sensor, a temperature sensor, a pressure sensor, an acceleration sensor, a wireless radar sensor, and an oxygen saturation sensor.

[0018] An electronic device according to one aspect further comprises an input interface for receiving user input; and a memory for storing information about a preset speed for each user.

[0019] A processor of an electronic device according to one aspect recognizes a user based on user input and obtains a preset speed corresponding to the recognized user based on bio-signals.

[0020] A sleep management system according to another aspect includes a plurality of sensors for detecting a user's vital signs; an electronic device for outputting at least one of video and audio; and a hub device for determining a user's sleep stage based on the user's vital signs detected by the plurality of sensors and transmitting a control command to the electronic device for reducing the output of the electronic device at a preset rate based on the determined sleep stage.

[0021] The preset speed of the sleep management system according to another aspect includes a first reference speed and a second reference speed that is faster than the first reference speed. The hub device of the sleep management system according to another aspect controls the electronic device so that the output is reduced to the first reference speed or the second reference speed based on whether the sleep stage is determined to be a light sleep stage.

[0022] The hub device of the sleep management system according to another aspect determines the rate of decrease in output as a first reference rate based on the sleep stage being determined as stage 1 of the light sleep stage, and determines the rate of decrease in output as a twelfth reference rate based on the sleep stage being determined as stage 2 of the light sleep stage.

[0023] The hub device of the sleep management system according to another aspect determines the rate of decrease in output as the first reference rate based on the change in sleep stage from stage 2 to stage 1.

[0024] The hub device of the sleep management system according to another aspect transmits a control command to the electronic device to stop the reduction in output based on the change in sleep stage from stage 1 to the wake stage.

[0025] The hub device of the sleep management system according to another aspect transmits a control command to the electronic device to turn off the electronic device based on whether the sleep stage is determined to be a deep sleep stage.

[0026] The hub device of the sleep management system according to another aspect transmits a control command to the electronic device to control the output of the electronic device to a preset output amount or less in response to the sleep stage being determined to be a light sleep stage.

[0027] According to one aspect of the disclosed invention, one aspect learns data detected by a plurality of contact sensors and a plurality of non-contact sensors to recognize a user's sleep stage, and if the recognized sleep stage is a light sleep stage, the brightness of the video and the volume of the audio are reduced so as not to disturb the user's sleep induction, thereby inducing the user to sleep deeply.

[0028] The present invention prevents sleep disruption caused by electronic devices by reducing the brightness and volume of the device when the user falls asleep while watching content on the device. The present invention can improve the quality of a user's sleep by changing an unsuitable sleep environment into a sleep-inducing environment.

[0029] The present invention can reduce power consumption by an electronic device by turning off the electronic device when the user is in a deep sleep stage.

[0030] The present invention can control the brightness of lighting or turn off lighting based on the user's setting information when the user is in a light sleep stage, adjust the temperature and humidity of the room, and reduce noise caused by home appliances in the room, thereby providing an optimized sleeping environment for the user.

[0031] The present invention can reduce the time it takes for a user to wake up and improve the user's condition by controlling video, audio, lighting, temperature and humidity when the user wakes up.

[0032] FIG. 1 illustrates an example of a network of a sleep management system according to one embodiment.

[0033] FIG. 2 schematically illustrates an example of the structure of a sleep management system according to one embodiment.

[0034] FIG. 3 illustrates an example of a control block diagram of a sleep management system according to one embodiment.

[0035] FIGS. 4 and 5 illustrate examples of multiple sensors of a sleep management system according to one embodiment.

[0036] FIG. 6 illustrates an example of a flowchart of a sleep management method according to one embodiment.

[0037] FIG. 7 is a diagram illustrating a process for processing data collected from multiple sensors of a sleep management system according to one embodiment.

[0038] Figure 8 is a diagram for explaining sleep stages.

[0039] FIG. 9 illustrates an example of sleep state information derived by a sleep management system according to one embodiment.

[0040] FIG. 10 illustrates an example of sleep summary information derived by a sleep management system according to one embodiment.

[0041] FIG. 11 and FIG. 12 illustrate an example of a flowchart of a sleep management method according to one embodiment.

[0042] Figure 13 illustrates an example of a predetermined operation to induce sleep in a user.

[0043] Figure 14 illustrates an example of a certain action to relieve user stress.

[0044] Figure 15 illustrates an example of a predetermined operation to alleviate a user's sleep disturbance.

[0045] Figure 16 illustrates an example of a predetermined action to induce a user's wake-up.

[0046] Fig. 17 is a control block diagram of a sleep management system according to another embodiment.

[0047] Fig. 18 is a reference speed graph for reducing the output of an electronic device of a sleep management system according to another embodiment.

[0048] Figures 19, 20 and 21 are exemplary diagrams of adjusting the reference speed for reducing the output amount of an electronic device of a sleep management system according to another embodiment.

[0049] Fig. 22 is a control flowchart of an electronic device of a sleep management system according to another embodiment.

[0050] FIGS. 23a and 23b are control flowcharts for adjusting the reference speed of an electronic device of a sleep management system according to another embodiment.

[0051] Fig. 24 is a control block diagram of a sleep management system according to another embodiment.

[0052] Fig. 25 is a control flowchart of a sleep management system according to another embodiment.

[0053] It should be understood that the various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but rather to encompass various modifications, equivalents, or alternatives of the embodiments.

[0054] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0055] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0056] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0057] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0058] For example, a phrase such as "A, B, and / or C" may include any one of the items listed in that phrase, or all possible combinations of them.

[0059] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0060] When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0061] The terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0062] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0063] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0064] Below, a sleep management system according to various embodiments is specifically described with reference to the attached drawings.

[0065] FIG. 1 illustrates an example of a network of a sleep management system according to one embodiment.

[0066] Referring to FIG. 1, a sleep management system according to one embodiment may include a hub device (1), a user device (2), a server device (3), and / or a home appliance (4).

[0067] The hub device (1) may include a communication module capable of communicating with a user device (2), a server device (3), and / or a home appliance (4), at least one processor for processing data, and at least one memory storing a program for controlling the operation of the hub device (1).

[0068] The hub device (1) can obtain processed data based on processing data collected from multiple sensors. In one embodiment, the hub device (1) can process data collected from multiple sensors using a machine learning model.

[0069] In one embodiment, the hub device (1) can transmit processing data to the user device (2). For example, the hub device (1) can transmit processing data to the user device (2) through direct communication without going through the server device (3).

[0070] The home appliance (4) may include various types of electronic products. For example, the home appliance (4) may include at least one of a display device (41), a furniture control device (42), a lighting device (43), an automatic curtain opening / closing device (44), an air conditioner (45), a speaker (46), and an air purifier (47). The aforementioned home appliances are merely examples, and therefore, in addition to the aforementioned home appliances, various types of electronic products, such as a clothing manager, may also be included in the home appliance (4).

[0071] The home appliance (4) can be remotely controlled by the server device (3).

[0072] The furniture control device (42) may include an actuator that can change the user's posture by changing the structure of the furniture and / or a vibration element that can transmit vibration to a user lying or sitting on the furniture. For example, the furniture control device (42) may include an actuator that can control the reclining angle of a recliner bed, a recliner chair, and / or a recliner sofa.

[0073] The lighting device (43) may include a light source whose light intensity and / or light color can be controlled.

[0074] The automatic curtain opening / closing device (44) may include an actuator for automatically opening or closing the curtain.

[0075] The server device (3) may include a communication module capable of communicating with the hub device (1), the user device (2), and / or the home appliance (4).

[0076] The server device (3) may include at least one processor capable of processing data received from the hub device (1), the user device (2), and / or the home appliance (4), and at least one memory capable of storing a program for processing the data or the processed data. The server device (3) may be implemented as various computing devices such as a workstation, a cloud, a data drive, a data station, etc. The server device (3) may be implemented as one or more servers that are physically or logically separated based on functions, detailed configurations of functions, or data, and may transmit and receive data through communication between each server and process the transmitted and received data.

[0077] The server device (3) can store and / or manage user accounts, register hub devices (1), user devices (2), and / or home appliances (4) by linking them to user accounts, and perform functions of managing or controlling the registered hub devices (1) and home appliances (4). For example, a user can access the server device (3) through the user device (2) and create a user account. The user account can be identified by an ID and password set by the user. The user can access the server device (3) through the user device (2) and manage the user account. The server device (3) can register hub devices (1), user devices (2), and / or home appliances (4) to the user account according to a set procedure. For example, the server device (3) can link identification information (e.g., serial number or MAC address, etc.) of the hub device (1) to the user account, thereby registering, managing, and controlling the hub device (1). Likewise, the server device (3) can register and control the user device (2) and home appliance (4) to the user account.

[0078] The server device (3) can receive various information from the hub device (1), user device (2), and / or home appliance (4) registered to the user account.

[0079] For example, the server device (3) may include a first server and a second server. The first server may create and / or manage user account information, and register and / or manage information of the hub device (1), the user device (2), and / or the home appliance (4) in the user account information. The second server may receive registration information of the user device (2) and the home appliance (4) from the first server, and control the user device (2) and / or the home appliance (4).

[0080] As another example, the second server may perform the management function of the hub device (1) and home appliance (4) registered in the first server on behalf of the first server.

[0081] The number of server devices (3) is not limited thereto, and the server devices (3) may include multiple servers for performing the same operation and / or different operations.

[0082] The user device (2) may include a communication module capable of communicating with a hub device (1), a server device (3), and / or a home appliance (4). The user device (2) may include a user interface for receiving user input or outputting information to the user. The user device (2) may include at least one processor for controlling the operation of the user device (2) and at least one memory for storing a program for controlling the operation of the user device (2).

[0083] The user device (2) may be carried by the user or placed in the user's home or office, etc. The user device (2) may include, but is not limited to, a personal computer, a terminal, a mobile phone, a smart phone, a handheld device, a wearable device, a display device, etc.

[0084] The memory of the user device (2) may store a program, i.e., an application, that can process data received from the hub device (1). The application may be sold installed on the user device (2) or downloaded and installed from an external server.

[0085] A user can access a server device (3) by executing an application installed on a user device (2), create a user account, and perform communication with the server device (3) based on the logged-in user account to register a hub device (1) and / or a home appliance (4).

[0086] For example, if the home appliance (4) is operated so that it can be connected to the server device (3) according to the procedure guided by the application installed on the user device (2), the server device (3) can register the home appliance (4) to the user account by registering the identification information (e.g., serial number or MAC address) of the home appliance (4) to the corresponding user account, thereby registering the home appliance (4). The home appliance (4) can also be registered to the user account in the same manner. It goes without saying that the information required to register a device such as a home appliance (4), home appliance (4), etc. to a user account may include other information that can identify the device in addition to the serial number or MAC address of the device.

[0087] The user device (2) can receive various information from the server device (3) or from the hub device (1) or home appliance (4) registered directly to the user account.

[0088] A network can include both wired and wireless networks. Wired networks include cable networks or telephone networks, while wireless networks can include any network that transmits and receives signals via radio waves. Wired and wireless networks can be interconnected.

[0089] A network may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an access point (AP), and a short-range wireless network that does not use an access point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth™ (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

[0090] An access point (AP) can connect a hub device (1), a user device (2), and / or a home appliance (4) to a wide area network (WAN) to which a server device (3) is connected. The hub device (1), the user device (2), and / or the home appliance (4) can be connected to the server device (3) via the wide area network (WAN).

[0091] The access point (AP) communicates with a hub device (1), a user device (2), and / or a home appliance (4) using wireless communication such as Wi-Fi (Wi-Fi™, IEEE 802.11), Bluetooth (Bluetooth™, IEEE 802.15.1), or Zigbee (IEEE 802.15.4), and can connect to a wide area network (WAN) using wired communication, but the wireless communication method of the access point (AP) is not limited thereto.

[0092] In one embodiment, the hub device (1) can communicate with the user device (2) via a short-range wireless network that does not go through an access point (AP).

[0093] For example, the hub device (1) may be connected to the user device (2) via a short-range wireless network (e.g., Wi-Fi Direct, Bluetooth, NFC). As another example, the hub device (1) may be connected to the user device (2) via a wide area network (WAN) using a long-range wireless network (e.g., a cellular communication module).

[0094] FIG. 2 schematically illustrates an example of the structure of a sleep management system according to one embodiment. FIG. 3 illustrates an example of a control block diagram of a sleep management system according to one embodiment.

[0095] Referring to FIGS. 2 and 3, the hub device (1) can receive data collected from a plurality of sensors (5).

[0096] The plurality of sensors (5) may include sensors for collecting user data (e.g., a first sensor (51), a second sensor (52), a third sensor (53), a fourth sensor (54), and / or a fifth sensor (55)).

[0097] Each of the plurality of sensors (5) can collect user data and transmit the collected user data to the hub device (1).

[0098] In one embodiment, the user's data may include data related to the user's sleep.

[0099] Data related to the user's sleep may include pressure data for measuring pressure changes corresponding to changes in the user's posture, displacement data corresponding to body displacement that changes according to the user's breathing, oxygen saturation data corresponding to the user's oxygen saturation, electrocardiogram data corresponding to the user's electrocardiogram, acceleration data corresponding to acceleration that changes according to the user's movement, and / or eye-movement data corresponding to the movement of the user's eyes.

[0100] The plurality of sensors (5) may include at least two of a pressure sensor for collecting pressure data for measuring pressure changes corresponding to changes in the user's posture, a UWB sensor for measuring displacement data corresponding to displacement of the body that changes according to the user's breathing, an oxygen saturation sensor for collecting oxygen saturation data corresponding to the user's oxygen saturation, an electrocardiogram sensor for collecting electrocardiogram data corresponding to the user's electrocardiogram, an acceleration sensor for collecting acceleration data corresponding to acceleration that changes according to the user's movement, and / or a radar sensor for collecting eye-movement data corresponding to the movement of the user's eyes.

[0101] In the terms "first, second, third, fourth, and fifth" in the first sensor (51), second sensor (52), third sensor (53), fourth sensor (54), and fifth sensor (55), the terms "first," "second," "third," "fourth," and "fifth" are intended to indicate that each sensor is a different sensor.

[0102] Each of the first sensor (51), second sensor (52), third sensor (53), fourth sensor (54), and fifth sensor (55) may be any one of a pressure sensor, an ultra-wideband (UWB) sensor, a radar sensor, an oxygen saturation sensor, an electrocardiogram sensor, or an acceleration sensor.

[0103] For convenience of explanation, the first sensor (51) is defined as a pressure sensor, the second sensor (52) as a UWB sensor, the third sensor (53) as a radar sensor, the fourth sensor (54) as an oxygen saturation sensor and / or an electrocardiogram sensor, and the fifth sensor (55) as an acceleration sensor.

[0104] The plurality of sensors (5) may further include additional sensors (e.g., microphones, cameras) in addition to the first sensor (51) to the fifth sensor (55), or may not include at least one of the first sensor (51) to the fifth sensor (55).

[0105] Data collected from multiple sensors (5) can be transmitted to the hub device (1).

[0106] In one embodiment, data collected by at least one sensor among a plurality of sensors (5) can be transmitted to the hub device (1) via wired communication, and data collected by the remaining sensors can be transmitted to the hub device (1) via wireless communication.

[0107] Accordingly, the hub device (1) can be connected to at least one of the plurality of sensors (5) by wire and connected to the remaining sensors wirelessly.

[0108] In one embodiment, the fourth sensor (54) may be included in a smart sensor device (e.g., a wearable device). The smart sensor device may include a wireless communication module and the fourth sensor (54). For example, the smart sensor device may include a smart watch in the form of a watch, and / or a smart ring in the form of a ring, but the form of the smart sensor device is not limited thereto.

[0109] The smart sensor device can establish wireless communication with the hub device (1) and transmit data collected from the fourth sensor (54) to the hub device (1) via wireless communication.

[0110] In one embodiment, the smart sensor device may include a fourth sensor (54) and a fifth sensor (55). The smart sensor device may establish wireless communication with the hub device (1) and transmit data collected from the fourth sensor (54) and the fifth sensor (55) to the hub device (1) via wireless communication.

[0111] In one embodiment, data collected from multiple sensors (5) can be transmitted to a hub device (1) via wired communication.

[0112] According to various embodiments, the plurality of sensors (5) may include imaging sensors (e.g., cameras). However, in one embodiment, the plurality of sensors (5) may all be non-imaging sensors.

[0113] According to one embodiment of the present disclosure, all of the plurality of sensors (5) correspond to non-imaging sensors, thereby preventing invasion of the user's privacy.

[0114] In one embodiment, the hub device (1) may include at least one processor (120) storing a program for processing data collected from a plurality of sensors (5), and at least one processor (110) capable of processing data collected from a plurality of sensors (5) based on the program stored in the at least one processor (120).

[0115] At least one processor (120) can store a machine learning model for processing data collected from multiple sensors (5).

[0116] In one embodiment, the machine learning model may be a machine learning model for feature extraction that extracts features of data when data collected from a plurality of sensors (5) is input, and outputs processed data including the extracted features.

[0117] Features of data may include elements that a machine learning model extracts from the data to perform classification or prediction.

[0118] In one embodiment, the machine learning model may be a machine learning model for determining sleep stages, which outputs processed data including data on the user's sleep stage when data collected from a plurality of sensors (5) is input.

[0119] For example, at least one processor (120) may include a first machine learning model (11) for processing first data collected from a first sensor (51), a second machine learning model (12) for processing second data collected from a second sensor (52), a third machine learning model (13) for processing third data collected from a third sensor (53), a fourth machine learning model (14) for processing fourth data collected from a fourth sensor (54), and / or a fifth machine learning model (15) for processing fifth data collected from a fifth sensor (55).

[0120] At least one processor (110) can obtain first processed data based on processing first data collected from the first sensor (51).

[0121] The first processed data may include feature data extracted from the first data and / or data on sleep stages extracted from the first data. The capacity of the first processed data may be smaller than the capacity of the first data.

[0122] At least one processor (110) can obtain second processed data based on processing second data collected from a second sensor (52).

[0123] The second processed data may include feature data extracted from the second data and / or data on sleep stages extracted from the second data. The capacity of the second processed data may be smaller than the capacity of the second data.

[0124] At least one processor (110) can obtain third processing data based on processing third data collected from a third sensor (53).

[0125] The third processing data may include feature data extracted from the third data and / or data on sleep stages extracted from the third data. The capacity of the third processing data may be smaller than the capacity of the third data.

[0126] At least one processor (110) can obtain fourth processed data based on processing fourth data collected from the fourth sensor (54).

[0127] The fourth processing data may include feature data extracted from the fourth data and / or data on sleep stages extracted from the fourth data. The capacity of the fourth processing data may be smaller than the capacity of the fourth data.

[0128] At least one processor (110) can obtain fifth processed data based on processing fifth data collected from the fifth sensor (55).

[0129] The fifth processed data may include feature data extracted from the fifth data and / or data on sleep stages extracted from the fifth data. The capacity of the fifth processed data may be smaller than the capacity of the fifth data.

[0130] According to the present disclosure, by first processing data in a hub device (1) and then transmitting it to a user device (2), the amount of data processing that the user device (2) must bear can be reduced.

[0131] The hub device (1) may include a communication unit (130) including a wired communication module for performing wired communication with a plurality of sensors (5), and / or a wireless communication module for performing wireless communication with a user device (2), a server device (3), and / or a home appliance.

[0132] The hub device (1) may include a printed circuit board including at least one processor (110), at least one processor (120), and a communication unit (130). At least some of the plurality of sensors (5) may be connected to the printed circuit board by wires.

[0133] The hub device (1) may include a housing covering a printed circuit board.

[0134] The hub device (1) is likely to be installed in a location where it is difficult for the user to operate. Accordingly, in one embodiment, the hub device (1) may not include a user interface device (input / output device).

[0135] In order to prepare for the case where the hub device (1) is installed in a place where the user can easily operate it, in one embodiment, the hub device (1) may include a user interface device (input / output device).

[0136] In one embodiment, a user can operate a user interface device provided on the hub device (1) to connect the hub device (1) to an access point (AP).

[0137] In one embodiment, a user can activate the communication unit (130) of the hub device (1) by operating a user interface device provided on the hub device (1).

[0138] In one embodiment, a user can turn on the power of the hub device (1) by operating a user interface device provided on the hub device (1).

[0139] At least one processor (110) can control multiple sensors (5).

[0140] For example, at least one processor (110) can control at least one wired sensor among a plurality of sensors (5).

[0141] At least one processor (110) can wake up at least one wired sensor among a plurality of sensors (5) based on the satisfaction of a sensor wake-up condition.

[0142] Waking up a sensor may include activating the sensor.

[0143] At least one processor (110) may switch at least one wired sensor among the plurality of sensors (5) to a standby state based on the satisfaction of a sensor standby condition. Switching the sensor to a standby state may include deactivating the sensor or operating it in a low power mode.

[0144] The user device (2) can receive data acquired by the hub device (1) from the hub device (1) via wireless communication.

[0145] The data acquired by the hub device (1) may include processed data obtained by processing data collected from a plurality of sensors (5).

[0146] In one embodiment, the user device (2) may include at least one memory (220) storing a program for processing data received from the hub device (1), and at least one processor (210) capable of processing data received from the hub device (1) based on the program stored in the at least one memory (220).

[0147] At least one memory (220) can store a machine learning model for processing data received from the hub device (1).

[0148] At least one memory (220) can store a sleep management application that can be downloaded from an external server. The sleep-related application can be a downloadable app, and at least some of it can be temporarily stored or temporarily created on a device-readable recording medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0149] A sleep management application may include a machine learning model. The machine learning model included in the sleep management application may be updated by an external server.

[0150] In one embodiment, the user device (2) may include a communication unit (230) including at least one communication module for establishing communication with a smart sensor device including at least some of a hub device (1), a server device (3), a home appliance (4), and / or a plurality of sensors (5).

[0151] The user device (2) can receive processing data from the hub device (1) through the communication unit (230).

[0152] In one embodiment, at least one processor (210) may establish communication between the communication unit (130) of the hub device (1) and the communication unit (230) of the user device (2) in response to the communication unit (230) (e.g., a short-range wireless communication module) being activated.

[0153] In one embodiment, at least one processor (210) can control the user interface (240) to provide feedback requesting activation of the communication unit (230) in response to the sleep management application being executed while the communication unit (230) (e.g., a short-range wireless communication module) is not activated.

[0154] At least one processor (210) can wirelessly receive data from the hub device (1) via a communication unit (230) (e.g., a short-range wireless communication module).

[0155] At least one processor (210) can process data received from the hub device (1) using a machine learning model stored in at least one memory (220).

[0156] The data received from the hub device (1) may include first processing data, second processing data, third processing data, fourth processing data and / or fifth processing data.

[0157] At least one processor (210) can input first processing data, second processing data, third processing data, fourth processing data, and / or fifth processing data into a machine learning model stored in at least one memory (220) to obtain sleep state information related to the user's sleep state.

[0158] Sleep state information related to the user's sleep state may include at least one of information about the user's sleep stage, information about the user's stress index, or information about the user's sleep disorder.

[0159] The machine learning model stored in at least one memory (220) may be a machine learning model for determining a user's sleep state, which outputs sleep state information related to the user's sleep state when first processing data, second processing data, third processing data, fourth processing data, and / or fifth processing data are input.

[0160] At least one processor (210) can input the first processing data, the second processing data, the third processing data, the fourth processing data, and / or the fifth processing data into the machine learning model and store the acquired sleep state information in at least one memory (220).

[0161] In one embodiment, at least one processor (210) may generate sleep summary information based on sleep state information accumulated and stored in at least one memory (220). At least one processor (210) may control a user interface (240) to output sensory information corresponding to the sleep summary information based on a predetermined condition being satisfied (e.g., receiving a user input for confirming the sleep summary information).

[0162] At least one processor (210) can control the communication unit (230) to transmit the acquired sleep state information by inputting the first processing data, the second processing data, the third processing data, the fourth processing data, and / or the fifth processing data into the machine learning model to the server device (3).

[0163] The communication unit (230) may include a first communication module for establishing communication with the hub device (1) and a second communication module for establishing communication with the server device (3). Accordingly, the communication unit (230) may perform communication with the hub device (1) through the first communication method while simultaneously performing communication with the server device (3) through the second communication method.

[0164] According to one embodiment of the present disclosure, data collected from a plurality of sensors (5) is primarily processed by a hub device (1) and secondarily processed by a user device (2), and then sleep state information is obtained, and the sleep state information can be transmitted to a server device (3) in real time.

[0165] According to one embodiment of the present disclosure, data collected from a plurality of sensors (5) related to the user's private life may not be directly transmitted to the server device (3).

[0166] The user device (2) may include a user interface (240) for communication with the user.

[0167] According to various embodiments, it is also possible for the hub device (1) to primarily process data collected from multiple sensors (5), and then secondarily process the primarily processed data to obtain sleep state information.

[0168] According to various embodiments, it is also possible for the hub device (1) to primarily process data collected from a plurality of sensors (5), transmit the primarily processed data to a server, and then acquire sleep state information after the server secondarily processes the primarily processed data.

[0169] According to various embodiments, it is also possible for the hub device (1) to transmit data collected from a plurality of sensors (5) to the user device (2), the user device (2) to primarily process the data received from the hub device (1), and to secondarily process the primarily processed data and then obtain sleep state information.

[0170] According to various embodiments, it is also possible for the hub device (1) to transmit data collected from a plurality of sensors (5) to the server device (3), the server device (3) to primarily process the data received from the hub device (1), and to secondarily process the primarily processed data and then obtain sleep state information.

[0171] The user interface (240) can obtain user input. The user interface (240) can provide various information regarding the operation of the user device (2). The user interface (240) can include an input interface and an output interface.

[0172] The input interface can convert sensory information received from a user into an electrical signal. The electrical signal can correspond to a user input. The user input can include various commands. The input interface can transmit an electrical signal (voltage or current) corresponding to the user input to at least one processor (210).

[0173] The input interface may include various input devices capable of converting tactile information into electrical signals. For example, the input interface may be provided as a physical button or a touch screen. The input interface may include a microphone capable of converting auditory information into electrical signals.

[0174] The input interface can receive user input to run a sleep management application.

[0175] The output interface can output information related to the operation of the user device (2). The output interface can display information input by the user or information provided to the user on various screens. The output interface can display information related to the operation of the user device (2) in the form of at least an image or text. For example, the output interface can output the interface of a sleep management application. In addition, the output interface can display a graphical user interface (GUI) that enables control of the user device (2). That is, the output interface can display a user interface element (UI element), such as an icon.

[0176] The output interface can output an interface corresponding to a sleep management application.

[0177] For example, the output interface may include a liquid crystal display panel (LCD Panel), a light emitting diode panel (LED Panel), an organic light emitting diode panel (OLED Panel), or a micro LED panel. The output interface may include a touch display that also functions as an input device.

[0178] The output interface and input interface may be provided as separate devices or as a single device (e.g., a touch display).

[0179] The server device (3) can receive data acquired by the user device (2) from the user device (2) via wireless communication.

[0180] The data acquired by the user device (2) may include the sleep state information described above.

[0181] In one embodiment, the server device (3) may include at least one memory (320) storing a program for processing sleep state information received from the user device (2), and at least one processor (310) capable of processing sleep state information received from the user device (2) based on the program stored in the at least one memory (320).

[0182] At least one memory (320) can store sleep state information received from the user device (2).

[0183] At least one memory (320) can store a program for generating sleep summary information based on sleep state information received from the user device (2).

[0184] At least one processor (310) can generate sleep summary information based on sleep state information accumulated and stored in at least one memory (320).

[0185] At least one memory (320) can store a program for controlling a home appliance (4) based on sleep state information received from a user device (2).

[0186] At least one processor (310) can control the home appliance (4) based on sleep state information received from the user device (2).

[0187] According to various embodiments, a program for generating sleep summary information based on sleep state information and a program for controlling a home appliance (4) based on sleep state information may be stored on different servers.

[0188] For example, a first server included in the server device (3) can store a program for generating sleep summary information based on sleep state information, and a second server included in the server device (3) can store a program for controlling a home appliance (4) based on sleep state information.

[0189] At least one memory (320) can store sleep summary information generated based on sleep state information.

[0190] In one embodiment, the server device (3) may include a communication unit (330) including at least one communication module for establishing communication with a smart sensor device including at least some of a hub device (1), a user device (2), a home appliance (4), and / or a plurality of sensors (5).

[0191] The server device (3) can receive sleep status information from the user device (2) through the communication unit (330).

[0192] The server device (3) can transmit a control command to the home appliance (4) through the communication unit (330).

[0193] The server device (3) can transmit sleep summary information to the user device (2) through the communication unit (330).

[0194] According to one embodiment of the present disclosure, the user's sleep can be managed in various ways as the server device (3) performs various operations based on sleep state information.

[0195] FIGS. 4 and 5 illustrate examples of multiple sensors of a sleep management system according to one embodiment.

[0196] Referring to FIGS. 4 and 5, in one embodiment, some of the plurality of sensors (5) (e.g., the first sensor (51), the second sensor (52), and / or the third sensor (53)) may be installed in furniture (10), and other some of the plurality of sensors (5) (e.g., the fourth sensor (54), and / or the fifth sensor (55)) may be provided in a smart sensor device (e.g., a smart watch, a smart ring, etc.).

[0197] At least some of the plurality of sensors (5) may be provided on furniture (10) on which a user can sit or lie.

[0198] Furniture (10) on which a user can sit or lie down may include, for example, a bed, a chair, and / or a sofa, but of course, any type of furniture on which a user can sit or lie down may be adopted as furniture (10) according to one embodiment without limitation.

[0199] In one embodiment, furniture (10), such as a bed, chair, and / or sofa, may include an actuator that can change the user's posture by changing its structure and / or a vibration element that can transmit vibration to the user.

[0200] The first sensor (51) may include a pressure sensor. The pressure sensor may include a piezoelectric element that generates an electrical signal corresponding to a displacement caused by pressure.

[0201] The first sensor (51) can be installed at a location where it can measure the pressure generated by the user's body (e.g., whole body) depending on whether the user is lying down or sitting.

[0202] For example, if the furniture (10) corresponds to a bed, the first sensor (51) may be installed on the mattress, which is the part where pressure is generated by the user's body. The mattress may have a flat polygonal or circular shape, and may include a cover that forms the exterior and has a receiving space, and a pad that is installed in the receiving space of the cover and on which the first sensor (51) is placed. The mattress may be placed on the floor, a chair, a sofa, or a bed.

[0203] The mattress may further include springs and / or sponges. The springs and / or sponges may be provided in the accommodation space of the cover.

[0204] The structure (e.g., length, arrangement, etc.) of the first sensor (51) may be changed depending on the size of the mattress.

[0205] As another example, if the furniture (10) corresponds to a chair, the first sensor (51) may be provided in the seating area, backrest area, headrest, and / or leg area where pressure is generated by the user's body.

[0206] The seating portion may include a portion that the user's buttocks close to, the backrest portion may include a portion that the user's back closes to, the headrest may include a portion that the user's head closes to, and the leg portion may include a portion that the user's legs close to.

[0207] The location of the first sensor (51) is not limited to the examples shown in FIGS. 4 and 5, and the first sensor (51) may be installed in various locations capable of measuring pressure generated by the body of a user lying or sitting on the furniture (10).

[0208] The first sensor (51) can measure the pressure generated by a user lying or sitting on the furniture (10). For example, the first sensor (51) can measure the pressure distribution generated by a user lying or sitting on the furniture (10). The first sensor (51) can obtain pressure data corresponding to the pressure generated by a user lying or sitting on the furniture (10).

[0209] The second sensor (52) may include a UWB sensor. The UWB sensor may include a UWB signal emitting unit that transmits an ultra-wideband signal (UWB signal) and a UWB signal receiving unit that receives a UWB signal reflected by the user's body.

[0210] The second sensor (52) may have a detection area facing the body (e.g., torso) of a user lying or sitting on the furniture (10). The second sensor (52) may have a detection area capable of detecting body displacement according to the user's breathing. The second sensor (52) may be provided on a frame of the furniture (10) to have a detection area facing the body (e.g., torso) of the user, but the location of the second sensor (52) is not limited thereto.

[0211] For example, the second sensor (52) may have a detection area directed toward a part of the body of a user lying or sitting on the furniture (10).

[0212] For example, if the furniture (10) corresponds to a bed, the second sensor (52) may have a detection area facing the central area of ​​the bed.

[0213] As another example, if the furniture (10) corresponds to a chair, the second sensor (52) may have a detection area facing the backrest of the chair.

[0214] The second sensor (52) can transmit an ultra-wideband signal (UWB signal) toward the user's body and receive a UWB signal reflected from the user's body.

[0215] The second sensor (52) can measure the displacement of the user's body based on the UWB signal reflected from the user's body. For example, the second sensor (52) can measure the displacement of the user's body based on the time of flight (ToF) of the UWB signal. As another example, the second sensor (52) can measure the displacement of the user's body based on the change in the wavelength (and frequency) of the UWB signal by utilizing the Doppler effect.

[0216] That is, the second sensor (52) can obtain displacement data corresponding to the displacement of the body that changes according to the user's breathing.

[0217] The third sensor (53) may include a radar sensor. The third sensor (53) may include a radar signal irradiation unit that transmits a radar signal (e.g., mmWave signal, millimeter wave wave) and a radar signal reception unit that receives a radar signal (e.g., mmWave signal) reflected from the user's body.

[0218] The frequency band (e.g., 28 GHz) of the radar signal output from the third sensor (53) may be higher than the frequency band (e.g., 6.0 to 8.8 GHz) of the UWB signal output from the second sensor (52).

[0219] The frequency bandwidth of the radar signal output from the third sensor (53) may be narrower than the frequency bandwidth of the UWB signal output from the second sensor (52).

[0220] The third sensor may have a detection area facing the body (e.g., face) of a user lying or sitting on the furniture (10). The third sensor (53) may have a detection area capable of detecting the movement of the user's eyes. The third sensor (53) may be provided on a frame of the furniture (10) to have a detection area facing the body (e.g., torso) of the user, but the location of the third sensor (53) is not limited thereto.

[0221] The third sensor (53) may have a detection area directed toward a part of the body of a user lying or sitting on the furniture (10).

[0222] For example, if the furniture (10) corresponds to a bed, the third sensor (53) may have a detection area facing the head area of ​​the bed.

[0223] As another example, if the furniture (10) corresponds to a chair, the third sensor (53) may have a detection area directed toward the headrest of the chair.

[0224] The third sensor (53) can transmit a radar signal (mmWave signal, millimeter wave wave) toward the user's body and receive a mmWave signal reflected from the user's body.

[0225] The third sensor (53) can measure the movement of the user's eyes based on mmWave reflected from the user's eyes.

[0226] That is, the third sensor (53) can obtain eye-movement data corresponding to the movement of the user's eyes.

[0227] The fourth sensor (54) may include an oxygen saturation sensor and / or an electrocardiogram sensor. The oxygen saturation sensor and / or the electrocardiogram sensor may include a light source that irradiates light and a light receiving unit that receives light reflected from the user's body.

[0228] The fourth sensor may have a detection area facing the body of a user lying or sitting on the furniture (10).

[0229] The fourth sensor (54) may be provided in a smart sensor device (e.g., a smart watch, a smart ring, etc.) wearable by a user.

[0230] The fourth sensor (54) operates in a non-invasive manner, and can irradiate light toward a part of the user's body (e.g., wrist) and receive light reflected from the user's body.

[0231] The fourth sensor (54) can measure the oxygen saturation in the user's blood or the user's electrocardiogram based on the intensity of light reflected from the user's body.

[0232] Some of the light directed at a part of the body is absorbed by blood vessels, and depending on the absorption rate of the light or the pattern of the absorbed light, the oxygen saturation in the user's blood or the user's electrocardiogram can be measured.

[0233] The fifth sensor (55) may include an acceleration sensor. The acceleration sensor may include a MEMS sensor, a 3-axis acceleration sensor, and / or a 6-axis acceleration sensor.

[0234] According to various embodiments, the fifth sensor (55) may be provided in a wearable smart sensor device (e.g., a smart watch, a smart ring, etc.) by a user, similar to the fourth sensor (54).

[0235] In one embodiment, the fifth sensor (55) may be installed on the furniture (10).

[0236] The fifth sensor (55) can obtain acceleration data corresponding to the user's body movement.

[0237] The hub device (1) may be installed on furniture (10) or installed in a location adjacent to the furniture (10) and may be connected to some of the plurality of sensors (5) by wires. In addition, the hub device (1) may perform wireless communication with a smart sensor device (e.g., a wearable device) worn by a user.

[0238] Fig. 6 illustrates an example of a flowchart of a sleep management method according to one embodiment. Fig. 7 is a diagram illustrating a process for processing data collected from multiple sensors of a sleep management system according to one embodiment.

[0239] Referring to FIGS. 6 and 7, the hub device (1) can collect data from a plurality of sensors (5) (S1).

[0240] When multiple sensors (5) are maintained in an activated state, the power consumption supplied to the multiple sensors (5) may be large. Maintaining the multiple sensors (5) in an activated state may include maintaining a state in which the multiple sensors (5) are supplied with power and thus capable of acquiring sensor data.

[0241] In one embodiment, the hub device (1) can keep at least some of the plurality of sensors (5) in a deactivated state, but can switch the plurality of sensors (5) to an activated state based on the satisfaction of a predetermined condition.

[0242] For example, the hub device (1) can determine whether a user is present in the furniture (10) based on processing data collected from multiple sensors (5). The presence of a user in the furniture (10) may include the user lying down or sitting on the furniture (10).

[0243] The hub device (1) can switch multiple sensors (5) to a standby state based on the determination that there is no user in the furniture (10).

[0244] For example, the hub device (1) can deactivate the remaining sensors except for the first sensor (51) and operate the first sensor (51) in a low power mode. Deactivating the sensor may include cutting off power supplied to the sensor.

[0245] Operating the sensor in a low power mode may involve setting the sensor's operating cycle (e.g., data collection cycle) to a longer period.

[0246] The hub device (1) can determine whether a user is present in the furniture (10) based on processing data collected by a sensor (e.g., the first sensor (51)) operating in a low power mode among the plurality of sensors (5).

[0247] The hub device (1) can wake up multiple sensors (5) based on the determination that a user is present in the furniture (10).

[0248] A plurality of sensors (5) can transmit collected data to the hub device (1) based on activation.

[0249] The first sensor (51) can transmit first data to the hub device (1), the second sensor (52) can transmit second data to the hub device (1), the third sensor (53) can transmit third data to the hub device (1), the fourth sensor (54) can transmit fourth data to the hub device (1), and the fifth sensor (55) can transmit fifth data to the hub device (1).

[0250] In one embodiment, at least some of the plurality of sensors (5) can transmit sensor data to the hub device (1) via wired communication, and some of the remaining sensors (5) can transmit sensor data to the hub device (1) via wireless communication.

[0251] The hub device (1) can primarily process data collected from multiple sensors (5) (S2). To this end, the hub device (1) can be equipped with a machine learning model.

[0252] Data collected from multiple sensors (5) can be processed by a machine learning model (11, 12, 13, 14, 15) mounted on the hub device (1).

[0253] The hub device (1) may be equipped with a first machine learning model (11) for extracting features from first data collected from a first sensor (51), a second machine learning model (12) for extracting features from second data collected from a first sensor, a third machine learning model (13) for extracting features from third data collected from a third sensor (53), a fourth machine learning model (14) for extracting features from fourth data collected from a fourth sensor (54), and / or a fifth machine learning model (15) for extracting features from first data collected from a fifth sensor (55).

[0254] The first machine learning model (11) can be pre-trained to extract features from pressure data collected by a pressure sensor. The second machine learning model (12) can be pre-trained to extract features from displacement data collected by a UWB sensor. The third machine learning model (13) can be pre-trained to extract features from eye-movement data collected by a radar sensor. The fourth machine learning model (14) can be pre-trained to extract features from oxygen saturation data and / or electrocardiogram data collected by an oxygen saturation sensor and / or an electrocardiogram sensor. The fifth machine learning model (15) can be pre-trained to extract features from acceleration data collected by an acceleration sensor.

[0255] The first machine learning model (11) can use the first data collected by the first sensor (51) as input data and output first processed data as output data. The first processed data may include, for example, information about the user's posture, respiration rate, and heart rate inferred from the first data. As another example, the first processed data may include information about the user's sleep stage inferred from the first data, information about the user's stress index, or information about the user's sleep disorder.

[0256] The second machine learning model (12) can use the second data collected by the second sensor (52) as input data and output second processed data as output data. The second processed data may include, for example, information regarding respiration rate and heart rate inferred from the second data. As another example, the second processed data may include information regarding the user's sleep stage, information regarding the user's stress index, or information regarding the user's sleep disorder inferred from the second data.

[0257] The third machine learning model (13) can use third data collected by the third sensor (53) as input data and output third processed data as output data. The third processed data may include, for example, information about eye movements inferred from the third data. As another example, the third processed data may include information about the user's sleep stage, information about the user's stress index, or information about the user's sleep disorder inferred from the third data.

[0258] The fourth machine learning model (14) can use the fourth data collected by the fourth sensor (54) as input data and output fourth processed data as output data. The fourth processed data may include, for example, information regarding oxygen saturation and / or electrocardiogram inferred from the fourth data. As another example, the fourth processed data may include information regarding the user's sleep stage, information regarding the user's stress index, or information regarding the user's sleep disorder inferred from the fourth data.

[0259] The fifth machine learning model (15) can use the fifth data collected by the fifth sensor (55) as input data and output fifth processed data as output data. The fifth processed data may include, for example, information about movement inferred from the fifth data. As another example, the fifth processed data may include information about the user's sleep stage inferred from the fifth data, information about the user's stress index, or information about the user's sleep disorder.

[0260] According to the present disclosure, the hub device (1) primarily processes data collected from a plurality of sensors (5), thereby enabling data collected from a plurality of sensors (5) to be transmitted to a user device (2) even in a communication method with a low data transmission capacity.

[0261] The hub device (1) can process data collected from multiple sensors (5) and transmit the processed data to the user device (2) (S3).

[0262] The hub device (1) can transmit processing data to the user device (2) via wireless communication. In one embodiment, the communication unit (130) of the hub device (1) can include a first communication module for receiving data collected from some sensors (e.g., the fourth sensor (54)) among the plurality of sensors (5) via a first wireless communication method, and a second communication module for transmitting the processing data to the user device (2) via a second wireless communication method.

[0263] The first wireless communication method and the second wireless communication method may be the same or different from each other.

[0264] According to one embodiment of the present disclosure, a hub device (1) has both a first communication module for receiving data from some of a plurality of sensors (5) and a second communication module for communicating with a user device (2), so that it can communicate with a plurality of sensors (5) and a user device (2) at the same time.

[0265] The user device (2) can process the processing data received from the hub device (1) (S4). For this purpose, the user device (2) can be equipped with a machine learning model.

[0266] Processing data received from the hub device (1) can be processed by a machine learning model (21) installed in the user device (2).

[0267] The machine learning model (21) mounted on the user device (2) may include an artificial neural network (deep neural network) model having multiple layers (e.g., an input layer, a hidden layer, and an output layer). The machine learning model (21) mounted on the user device (2) may be configured as a perceptron structure having a structure that receives multiple signals as input and outputs one signal. The machine learning model (21) mounted on the user device (2) may be trained for the purpose of estimating the user's sleep state based on processing data processed by the hub device (1).

[0268] The machine learning model (21) mounted on the user device (2) can use the processed data output by the machine learning model of the hub device (1) as input data and output sleep state information related to the user's sleep state as output data.

[0269] The machine learning model (21) mounted on the user device (2) can output sleep state information by using the first processing data, the second processing data, the third processing data, the fourth processing data, and / or the fifth processing data as input data.

[0270] The sleep state information output by the machine learning model (21) installed in the user device (2) may include at least one of information about the user's sleep stage, information about the user's stress index, or information about the user's sleep disorder.

[0271] According to the present disclosure, data output by a machine learning model (11, 12, 13, 14, 15) primarily mounted on a hub device (1) is secondarily input into a machine learning model (21) mounted on a user device (2) to output sleep state information, so that the user's sleep state can be accurately estimated. That is, according to the present disclosure, a large amount of data can be processed in stages to accurately estimate the user's sleep state.

[0272] The user device (2) can transmit sleep state information to the server device (3) (S5).

[0273] According to the present disclosure, instead of direct data related to the user's private life being transmitted to the server device (3), only sleep state information related to the user's sleep state is transmitted to the server device (3), thereby making it easier to obtain the user's consent for data collection.

[0274] The server device (3) can generate a control command for controlling the home appliance (4) based on the sleep state information (S6). At this time, the home appliance (4) can include at least one home appliance (4) registered in conjunction with the user device (2). The server device (3) can store and / or manage a user account and register the user device (2) and the home appliance (4) by linking them to the user account.

[0275] At least one home appliance (4) registered in conjunction with the user device (2) may include a home appliance (4) registered to the user account to which the user device (2) is registered.

[0276] The process (S6) in which the server device (3) generates a control command for controlling the home appliance (4) based on the sleep state information is described in detail later with reference to FIGS. 11 to 16. The server device (3) can transmit a control command for controlling the home appliance (4) to the home appliance (4) (S7).

[0277] According to various embodiments, the process (S6) of generating a control command for controlling the home appliance (4) based on sleep state information and the process of transmitting the control command for controlling the home appliance (4) to the home appliance (4) can of course also be performed by the user device (2).

[0278] The home appliance (4) can perform a predetermined operation corresponding to a control command received from the server device (3) (S8).

[0279] The machine learning model stored in the user device (2) can be updated by an external server. To this end, according to various embodiments, data output by the machine learning model (11, 12, 13, 14, 15) mounted on the hub device (1) can be transmitted to the server device (3).

[0280] Figure 8 is a diagram for explaining sleep stages.

[0281] Human sleep stages can be divided into wakefulness stages, rapid eye movement (REM) stages, and non-rapid eye movement (NREM) stages.

[0282] The awakening stage corresponds to the stage in which a person is awake.

[0283] REM sleep is a rapid eye movement sleep stage, a shallow sleep similar to being awake, and is a stage of sleep characterized by rapid eye movement.

[0284] In adults, REM sleep typically accounts for approximately 20–25% of total sleep, occurring approximately 90–120 minutes per night.

[0285] During REM sleep, the brain's neural activity is quite similar to that of awake states. However, because the body is in a relaxed state, it is involuntary. For this reason, REM sleep is also called paradoxical sleep, meaning that brain waves are not suppressed during REM sleep.

[0286] Non-REM sleep is a non-rapid eye movement sleep stage, and unlike REM sleep, there is almost no eye movement during non-REM sleep.

[0287] During non-REM sleep, dreams are rare, and muscle movements are not inhibited as in REM sleep. Those who do not properly progress through the sleep stages become trapped in non-REM sleep, and the lack of muscle inhibition can lead to sleep disorders (e.g., sleepwalking).

[0288] Non-REM sleep stages can be divided into stage 1 (stage N1), stage 2 (stage N2), stage 3 (stage N3), and stage 4 (stage N4). Within non-REM sleep stages, stages 1 (stage N1) and 2 (stage N2) can be divided into light sleep stages, while stages 3 (stage N3) and 4 (stage N4) can be divided into deep sleep stages.

[0289] As another example, non-REM sleep stages may be divided into stage 1 (stage N1), stage 2 (stage N2), and stage 3 (stage N3).

[0290] During non-REM sleep, brain wave activity gradually slows down and physiological functions decline. During non-REM sleep, brain tissue cells and epithelial cells regenerate, body energy is restored, and hormones for skeletal growth, protein synthesis, and tissue regeneration are secreted.

[0291] Stage N1 is the most liminal stage of sleep, marking the body's gradual transition into sleep. Stage N1 occurs before deep sleep and represents a state of non-deep sleep.

[0292] Stage N2 is a transition from borderline sleep to a deeper sleep. Brainwave patterns shift from slightly slower beta waves to slower theta waves, and eye movements cease, but intermittent rapid eye movements are absent.

[0293] Stage N3 is the initial stage of deep sleep, where brain wave patterns change to delta waves, muscle tone is relaxed, and physical movement is minimal. Snoring may occur in stage N3.

[0294] Stage N4 is a deep sleep from which it is very difficult to wake up. In stage N4, the secretion of growth hormones for skeletal growth, protein synthesis, and tissue regeneration increases, and symptoms such as sleepwalking and enuresis may appear.

[0295] Typically, when a person falls asleep, the first sleep cycle begins. In this first sleep cycle, the person enters non-REM sleep from the wakeful stage, then moves through stages N1, N2, N3, and N4, and then returns to N3, N2, N1, and REM sleep.

[0296] Afterwards, in the second sleep cycle, we go through stages N1, N2, N3, N4, and then return to stages N3 and N2.

[0297] Afterwards, in the third sleep cycle, we go through stage N3 and then return to stage N2, stage N1, and REM sleep.

[0298] Afterwards, in the fourth sleep cycle, we go through stage N1 and N2 sleep, and then enter stage N1 and REM sleep again.

[0299] After that, the person goes through the N1 stage, N2 stage, and REM sleep stage and wakes up naturally.

[0300] If a person wakes up in stage N4 sleep, they may feel groggy and not feel like they slept at all. Therefore, it is recommended that a person wake up in stage N1 or REM sleep.

[0301] FIG. 9 illustrates an example of sleep state information derived by a sleep management system according to one embodiment.

[0302] Sleep status information can be derived in real time by a sleep management system.

[0303] Referring to FIG. 9, sleep state information derived by a sleep management system according to one embodiment may include information on sleep stages, information on oxygen saturation, information on sleep disorders, information on stress index, information on respiration rate, information on movement, information on heart rate, and / or information on body pressure.

[0304] According to various embodiments, the user device (2) (or display device (41)) may output sleep state information through an output interface (e.g., a display). For example, the user device (2) may output sleep state information based on the acquired sleep state information.

[0305] Information about sleep stages may include information about the user's current sleep stage and / or information about the user's sleep stages over time.

[0306] Information about the user's current sleep stage may include information about whether the user's current sleep stage corresponds to a wake stage, a REM sleep stage, or stage 1, stage 2, stage 3, or stage 3 of non-REM sleep.

[0307] Information about the user's sleep stages over time may include information about changes in sleep stages from a time point preset by the user using the user device (2), a time point when the user lies down or sits on furniture (10) to fall asleep, a time point when the user falls asleep, and / or a time point when the user runs the sleep management application to the present time point.

[0308] When information about the user's sleep stage over time is output by the user device (2) (or display device (41)), the information about the user's sleep stage over time can be displayed in the form of a graph, and the x-axis of the graph can represent time and the y-axis can represent the user's sleep stage.

[0309] Information about oxygen saturation may include information about the user's current oxygen saturation and / or information about the user's oxygen saturation over time.

[0310] When information about oxygen saturation is output by the user device (2) (or display device (41)), the numerical value of oxygen saturation may be output in the form of a percentage. In addition, when information about oxygen saturation is output by the user device (2) (or display device (41)), whether the user's oxygen saturation is normal according to medical standards may be displayed.

[0311] Information about sleep disorders may include information about the current user's sleep disorders and / or information about sleep disorders that have developed over time.

[0312] Sleep disorders may include information about breathing-related sleep disorders such as apnea and hypopnea. Information about breathing-related sleep disorders such as apnea and hypopnea may include the Apnea Hypopnea Index (AHI).

[0313] When information on sleep disorders related to breathing is output by the user device (2) (or display device (41)), the Apnea Hypopnea Index (AHI) may be output, and whether the user's Apnea Hypopnea Index is normal according to medical standards may be displayed.

[0314] Information about the user's stress level may include information about the user's stress level. When information about the user's stress level is output by the user device (2) (or display device (41)), the user's stress level may be output numerically or in the form of comparative terms (e.g., "high," "medium," "low"), and whether the user's stress level is normal according to medical standards may be indicated.

[0315] Information about respiration rate may include information about the user's current respiration rate and / or information about the user's respiration rate over time.

[0316] When information about the current respiratory rate is output by the user device (2) (or display device (41)), the user's respiratory rate can be output numerically, and whether the user's respiratory rate is normal according to medical standards can be displayed.

[0317] When information about the user's breathing rate over time is output by the user device (2) (or display device (41)), the information about the user's breathing rate over time can be displayed in the form of a graph, and the x-axis of the graph can represent time and the y-axis can represent the user's breathing rate.

[0318] Information about movement may include information about the degree of movement of the user, and / or information about the degree of movement of the user over time.

[0319] When information about the current degree of movement is output by the user device (2) (or display device (41)), the degree of the user's movement may be output numerically or in the form of comparative words (e.g., high, medium, low).

[0320] When information about the degree of movement over time is output by the user device (2) (or display device (41)), the information about the degree of movement of the user over time can be displayed in the form of a graph, and the x-axis of the graph can represent time and the y-axis can represent the degree of movement of the user.

[0321] Information about heart rate may include information about the user's current heart rate and / or information about the user's heart rate over time.

[0322] When information about the current heart rate is output by the user device (2) (or display device (41)), the user's heart rate can be output numerically, and whether the user's heart rate is normal according to medical standards can be displayed.

[0323] When information about the user's heart rate over time is output by the user device (2) (or display device (41)), the information about the user's heart rate over time can be displayed in the form of a graph, and the x-axis of the graph can represent time and the y-axis can represent a pulse corresponding to the user's heartbeat.

[0324] Information about body pressure may include information about the user's body posture. Information about body posture may include information about whether the user's posture is lying upright, lying on the left side, lying on the right side, lying face down, or sitting.

[0325] When information about body pressure is output by the user device (2) (or display device (41)), a term indicating the user's posture may be used, and a pressure distribution table based on the furniture (10) on which the user is lying or sitting may be output.

[0326] FIG. 10 illustrates an example of sleep summary information derived by a sleep management system according to one embodiment.

[0327] Sleep summary information may include information compiled from sleep status information acquired in real time by a sleep management system.

[0328] For example, sleep summary information may include sleep score information corresponding to sleep quality, sleep time information, sleep efficiency information, sleep latency information, sleep continuity information, and / or sleep stage information.

[0329] A sleep score corresponding to the quality of sleep can be determined based on sleep state information acquired in real time during the user's sleep. For example, the user device (2) and / or the server device (3) can calculate a sleep score based on sleep state information acquired in real time during the user's sleep. Information on the sleep score can be calculated based on the user's sleep stage, oxygen saturation, sleep disorder, stress index, respiratory rate, movement, heart rate, and / or body pressure during sleep.

[0330] Sleep time information may include information about the time from when the user transitions from an awake state to a sleep state to when the user finally awakens.

[0331] As another example, sleep time information may include information about the time from the time of lying down on the furniture (10) to the time of finally waking up.

[0332] Information about sleep efficiency can be determined based on the proportion of time occupied by a given stage (e.g., non-REM sleep stage) among sleep stages. The user device (2) and / or the server device (3) can calculate sleep efficiency based on sleep state information acquired in real time during the user's sleep. In one embodiment, the user device (2) and / or the server device (3) can determine sleep efficiency based on the proportion of time occupied by a given stage (e.g., non-REM sleep stage) among the user's sleep stages.

[0333] Information about sleep onset latency (SOL) may include information about the time it takes for a sleep stage to change from a wakeful stage to a non-REM sleep stage.

[0334] Information about sleep continuity can be determined based on the time that the sleep stage remains in the non-REM sleep stage without changing to the wake stage (and / or REM sleep stage). The user device (2) and / or the server device (3) can determine sleep continuity based on sleep state information acquired in real time during the user's sleep. In one embodiment, the user device (2) and / or the server device (3) can determine sleep continuity based on the time that a predetermined stage (e.g., non-REM sleep stage) among the user's sleep stages is maintained.

[0335] Information about sleep stages may include information about changes in sleep stages over time. Information about sleep stages may include information about the percentage of time each sleep stage occupies during the user's sleep, information about whether the user's sleep cycle falls within a normal range, and information about the degree of physical recovery, mental recovery, and / or periodic awakening determined based on the percentage of time each sleep stage occupies during the user's sleep.

[0336] The sleep summary information is not limited to the examples described above and may include various information related to the user's sleep. For example, the sleep summary information may include information about the user's snoring time and information about the number of times the user's posture changed.

[0337] The sleep state information and sleep summary information described with reference to FIGS. 9 and 10 may be provided to the user by the user device (2) or may be provided to the user by a home appliance (4) (e.g., a display device (41), a speaker (46), etc.).

[0338] For example, a user can run a sleep management application using a user device (2) and check sleep status information and sleep summary information through an interface provided by the sleep management application.

[0339] As another example, the server device (3) can control a home appliance (4) (e.g., a display device (41), a speaker (46), etc.) to provide sleep summary information when the user wakes up.

[0340] According to various embodiments, a user may share sleep state information and / or sleep summary information with other users via the user device (2). For example, a user may run a sleep management application and transmit sleep state information and / or sleep summary information to another user's device via an interface provided by the sleep management application.

[0341] FIG. 11 and FIG. 12 illustrate an example of a flowchart of a sleep management method according to one embodiment.

[0342] As described with reference to FIG. 6, in one embodiment, the server device (3) can control the home appliance (4) based on sleep state information.

[0343] Referring to FIGS. 11 and 12, the hub device (1) can wake up a plurality of sensors (5) based on the satisfaction of a predetermined condition (1000).

[0344] For example, the hub device (1) can wake up multiple sensors (5) based on determining that a user is present in the furniture (10).

[0345] As another example, the user device (2) may transmit a sleep management start request to the hub device (1) based on receiving a user input corresponding to the start of sleep, and the hub device (1) may wake up a plurality of sensors (5) based on receiving the sleep management start request through the user device (2).

[0346] As another example, the user device (2) can receive information about the expected sleep time from the user and transmit it to the hub device (1). The hub device (1) can wake up a plurality of sensors (5) based on reaching the expected sleep time.

[0347] The server device (3) can determine whether the current time corresponds to sleep time (1100).

[0348] For example, a user can preset a sleep time through the user device (2). The sleep time can be set as a range of hours or as a time. For example, the sleep time can be set by the user to "10:00 PM to 11:00 PM" or "10:00 PM."

[0349] The server device (3) can determine that the current time corresponds to the sleep time if the current time corresponds to the sleep time set by the user.

[0350] As another example, when a user inputs a command corresponding to the start of sleep through the user device (2), the server device (3) can determine that the current time corresponds to the sleep time.

[0351] A command corresponding to the start of sleep can be received via the user interface (240) of the user device (2). For example, the command corresponding to the start of sleep can be received by the user device (2) via touch input and / or voice input.

[0352] According to various embodiments, when a user inputs an automatic sleep management request through a user device (2), the server device (3) may determine that the current time corresponds to sleep time based on satisfaction of a predetermined condition. In one embodiment, the server device (3) may determine whether the current time corresponds to sleep time based on result information. For example, the server device (3) may determine that the current time corresponds to sleep time when the user maintains a predetermined posture (e.g., lying down or prone posture) for a predetermined period of time.

[0353] If the current time corresponds to sleep time (example of 1100), the server device (3) can control the home appliance (4) based on sleep status information received from the user device (2).

[0354] In one embodiment, the server device (3) can control the home appliance (4) to perform a predetermined operation for inducing sleep (1300) when the current time corresponds to a sleep time (example of 1100) and the user's sleep stage corresponds to a predetermined stage (e.g., an awake stage) (example of 1200).

[0355] For example, the user device (2) can transmit a sleep start request to the server device (3) based on receiving a command corresponding to sleep start from the user, and the server device (3) can receive the sleep start request and control the home appliance (4) to perform a predetermined operation for sleep induction when the user's sleep stage corresponds to a predetermined stage (e.g., an awake stage).

[0356] The predetermined action for sleep induction may include a predetermined action that each of at least one home appliance (4) can perform.

[0357] Certain actions for inducing sleep may be preset and may be changed by the user through the user device (2).

[0358] Figure 13 illustrates an example of a predetermined operation to induce sleep in a user.

[0359] Referring to FIG. 13, the server device (3) can control the display device (41) to perform a predetermined operation for inducing sleep.

[0360] For example, the server device (3) can control the display device (41) so that the brightness of the image output from the display device gradually darkens. As another example, the server device (3) can control the display device (41) so that the display device (41) plays predetermined music (e.g., music for inducing sleep).

[0361] According to various embodiments, the server device (3) may also control the speaker (46) to play predetermined music (e.g., music for inducing sleep).

[0362] The server device (3) can control the lighting device (43) to perform a predetermined operation for inducing sleep. For example, the server device (3) can control the lighting device so that the brightness of the light output from the lighting device (43) gradually darkens. As another example, the server device (3) can control the lighting device so that the color of the light output from the lighting device (43) changes to a predetermined color (e.g., a color having a color temperature of 2000 K or less).

[0363] The server device (3) can control the automatic curtain opening / closing device (44) to perform a predetermined operation for inducing sleep. For example, the server device (3) can control the automatic curtain opening / closing device (44) to close the curtain.

[0364] The furniture control device (42) may include a vibration element (42a) that can transmit vibration to a user lying or sitting on the furniture (10) and / or an actuator (42b) that can change the user's posture by changing the structure of the furniture (10) (see FIGS. 14 and 15).

[0365] The server device (3) can control the furniture control device (42) to perform a predetermined operation for inducing sleep.

[0366] For example, the server device (3) can control the vibration element (42a) to output vibration at a predetermined cycle that can relieve the user's tension.

[0367] As another example, the server device (3) can control the actuator (42b) to change the structure of the furniture so that the user's posture changes to a predetermined posture (lying upright) that is comfortable for sleeping.

[0368] The server device (3) can control the air conditioner (45) and / or the air purifier (47) to perform a predetermined operation for inducing sleep. For example, the server device (3) can control the air conditioner and / or the air purifier (47) to operate in sleep mode.

[0369] When the air conditioner (45) and / or air purifier (47) is operated in sleep mode, noise can be minimized, for example, by controlling the fan speed to slow.

[0370] The specific operations for sleep induction that can be performed by the home appliance (4) are not limited to the examples described above, and can of course be changed depending on the user's settings and the type of home appliance (4).

[0371] Referring again to FIG. 11, the server device (3) can determine whether the user's sleep stage corresponds to an awake stage (1400).

[0372] If the user's sleep stage does not correspond to the awakening stage (example of 1400), the server device (3) can control the home appliance (4) to perform a predetermined action to alleviate the user's stress index or alleviate sleep disorders.

[0373] In one embodiment, the server device (3) can control the home appliance (4) to perform a predetermined operation to alleviate the user's stress index or alleviate sleep disorders when the user's sleep stage corresponds to the REM sleep stage or the non-REM sleep stage.

[0374] More preferably, the server device (3) can control the home appliance (4) to perform a predetermined operation to alleviate the user's stress index or alleviate sleep disorders only when the user's sleep stage corresponds to the non-REM sleep stage.

[0375] According to one embodiment of the present disclosure, by controlling the home appliance (4) to perform a predetermined operation to alleviate the user's stress index or alleviate sleep disorders only when the user's sleep stage corresponds to the non-REM sleep stage, the user can be prevented from being woken up by the operation of the home appliance (4) during the REM sleep stage.

[0376] Certain actions for stress relief that can be performed by the home appliance (4) are not limited to the examples described above, and may of course be changed depending on the user's settings and the type of home appliance (4).

[0377] The server device (3) can control the home appliance (4) to perform a predetermined action to relieve the user's stress in response to the user's stress index exceeding a predetermined value (example of 1500) (1550).

[0378] Figure 14 illustrates an example of a certain action to relieve user stress.

[0379] Referring to FIG. 14, the server device (3) can control the lighting device (43) to perform a predetermined operation to relieve the user's stress.

[0380] For example, the server device (3) can control the lighting device (43) to output light of a predetermined color (e.g., a color having a color temperature of 1000 K or less).

[0381] The server device (3) can control the air conditioner (45) to perform a predetermined operation to relieve user stress.

[0382] For example, the server device (3) can control the air conditioner (45) to slightly increase the speed of the fan so that the wind output from the air conditioner (45) can be delivered to the user.

[0383] The server device (3) can control the furniture control device (42) to perform a predetermined operation to relieve the user's stress. The furniture control device (42) can include a vibration element (42a) that can transmit vibration to a user lying or sitting on the furniture (10).

[0384] For example, the server device (3) can control the vibration element (42a) to output vibration corresponding to the user's heart rate.

[0385] According to one embodiment of the present disclosure, a user's stress index can be alleviated during sleep, thereby inducing a user to have good quality sleep.

[0386] Referring again to FIG. 11, the server device (3) can control the home appliance (4) to perform a predetermined action to alleviate the developed sleep disorder in response to the user experiencing a predetermined sleep disorder (example of 1600) (1650).

[0387] Certain operations for alleviating sleep disorders that can be performed by home appliances (4) are not limited to the examples described above, and may of course be changed depending on the user's settings and the type of home appliance (4).

[0388] Figure 15 illustrates an example of a predetermined operation to alleviate a user's sleep disturbance.

[0389] Referring to FIG. 15, the server device (3) can control the air conditioner (45) to perform a predetermined operation to alleviate sleep disturbance.

[0390] For example, the server device (3) can turn off the air conditioner (45).

[0391] The server device (3) can control the furniture control device (42) to perform a predetermined operation to alleviate sleep disorders. The furniture control device (42) can include an actuator (42b) that can change the user's posture by changing the structure of the furniture (10).

[0392] For example, the server device (3) can control the actuator (42b) to change the structure of the furniture so that the user's posture changes to a predetermined posture that is comfortable for breathing (a posture in which the user's head is higher than the torso and legs).

[0393] According to one embodiment of the present disclosure, a user can be induced to have good quality sleep by alleviating sleep disorders that occur during sleep.

[0394] Referring again to FIG. 12, the server device (3) can control the home appliance (4) to perform a predetermined operation for waking up based on the sleep stage not corresponding to the wake-up stage (No at 1700) and the current time corresponding to the wake-up time (Yes at 1800) (1950).

[0395] For example, a user can preset a wake-up time through the user device (2). The wake-up time can be set as a range of hours or as a time. For example, the wake-up time can be set by the user to "7:00 AM to 8:00 AM" or "7:00 AM."

[0396] The server device (3) can determine that the current time corresponds to the wake-up time if the current time corresponds to the wake-up time set by the user.

[0397] As another example, when a user inputs a command corresponding to the start of the wake-up through the user device (2), the server device (3) can determine that the current time corresponds to the wake-up time.

[0398] A command corresponding to the start of the wake-up can be received through the user interface (240) of the user device (2). For example, a command corresponding to the start of the wake-up can be received by the user device (2) by touch input and / or voice input.

[0399] According to various embodiments, when a user inputs an automatic sleep management request through the user device (2), the server device (3) may determine that the current time corresponds to the wake-up time based on satisfaction of a predetermined condition. In one embodiment, the server device (3) may determine whether the current time corresponds to the wake-up time based on the result information. For example, the server device (3) may determine that the current time corresponds to the wake-up time if the user's sleep stage maintains the wake-up stage for a predetermined period of time (e.g., 10 minutes).

[0400] As explained earlier, it is desirable for a person to wake up during stage N1 or REM sleep.

[0401] The server device (3) can determine the possibility of entering the N1 stage of a predetermined sleep stage (e.g., REM sleep stage and / or non-REM sleep stage) within the wake-up time based on the sleep state information (1900).

[0402] When the wake-up time is set to a range of time, determining the likelihood of entering a predetermined sleep stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep) within the wake-up time may include determining whether the user's sleep stage enters a predetermined sleep stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep) within the set range of time.

[0403] If the wake-up time is set to a time, determining the likelihood of entering a predetermined sleep stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep) may include determining whether the user's sleep stage is likely to enter a predetermined sleep stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep) within a predetermined time (e.g., 15 minutes) before or after the time corresponding to the wake-up time.

[0404] The server device (3) can control the home appliance (4) to perform a predetermined operation to wake up the user based on the user's sleep stage entering the predetermined stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep stage) within the wake-up time (example of 1900) (1960).

[0405] That is, the server device (3) can control the home appliance (4) to perform a predetermined operation to wake up the user when the current time corresponds to the wake-up time and the user's sleep stage corresponds to a predetermined stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep stage).

[0406] According to the present disclosure, by waking up a user in the REM sleep stage and / or the N1 stage of the non-REM sleep stage, the user can be induced to feel refreshed upon waking up.

[0407] On the other hand, if the server device (3) is unlikely to enter a predetermined stage (e.g., REM sleep stage and / or N1 stage of non-REM sleep stage) of the user's sleep stage within the wake-up time (No of 1900), the server device (3) can control the home appliance (4) to perform a predetermined action to wake up the user if the current time corresponds to the wake-up time (1950).

[0408] In one embodiment, the server device (3) can control the home appliance (4) to perform a predetermined operation to wake up the user (1950) when the user's sleep stage corresponds to the wake-up stage and the current time corresponds to the wake-up time (example of 1750).

[0409] In one embodiment, if the user's sleep stage corresponds to the wake-up stage and the current time does not correspond to the wake-up time (No of 1750), the server device (3) can perform an action for waking up or an action for inducing sleep again according to the user's input.

[0410] For example, the server device (3) may perform an action for waking up based on receiving a user input requesting the user device (2) to start waking up, if the user's sleep stage does not correspond to the current waking up time (1950).

[0411] As another example, the server device (3) may perform an operation for inducing sleep based on the user device (2) not receiving a user input requesting to start waking up for a predetermined period of time when the user's sleep stage does not correspond to the current time for waking up (1300).

[0412] As another example, the server device (3) may perform an operation for sleep induction based on receiving a user input requesting the user device (2) to start re-sleep when the user's sleep stage does not correspond to the current wake-up time (1300).

[0413] Figure 16 illustrates an example of a predetermined action to induce a user's wake-up.

[0414] Referring to FIG. 16, the server device (3) can control the display device (41) to perform a predetermined operation for inducing sleep.

[0415] For example, the server device (3) can control the display device (41) so that the brightness of the image output from the display device (41) gradually increases. As another example, the server device (3) can control the display device (41) so that the display device (41) plays predetermined music (e.g., music for waking up).

[0416] According to various embodiments, the server device (3) may also control the speaker (46) to play a predetermined music (e.g., music for waking up).

[0417] The server device (3) can control the lighting device (43) to perform a predetermined operation for weather induction. For example, the server device (3) can control the lighting device (43) so that the brightness of the light output from the lighting device (43) gradually increases. As another example, the server device (3) can control the lighting device (43) so that it outputs light similar to natural light.

[0418] The server device (3) can control the automatic curtain opening / closing device (44) to perform a predetermined operation for weather induction. For example, the server device (3) can control the automatic curtain opening / closing device (44) to open the curtain.

[0419] The server device (3) can control the furniture control device (42) to perform a predetermined operation for waking up. The furniture control device (42) can include a vibration element (42a) that can transmit vibration to a user lying or sitting on the furniture (10) and / or an actuator (42b) that can change the user's posture by changing the structure of the furniture (10).

[0420] For example, the server device (3) can control the vibration element (42a) to output vibration at a predetermined cycle corresponding to the user's heart rate.

[0421] As another example, the server device (3) can control the actuator (42b) to change the structure of the furniture so that the user's posture changes to a comfortable predetermined posture (a posture in which the upper body is higher than the lower body).

[0422] The server device (3) can control the air conditioner (45) and / or the air purifier (47) to perform a predetermined operation for weather induction. For example, the server device (3) can control the air conditioner and / or the air purifier (47) to operate in a weather mode.

[0423] When the air conditioner (45) and / or the air purifier (47) operates in the weather mode, it can provide a refreshing feeling to the user, for example, by quickly controlling the speed of the fan.

[0424] The predetermined operation for waking up that can be performed by the home appliance (4) is not limited to the above-described example, and can of course be changed depending on the user's settings and the type of home appliance (4).

[0425] According to various embodiments, the operations described with reference to FIGS. 11 to 16 may be performed in the user device (2). For example, the user device (2) may control the home appliance (4) based on sleep state information.

[0426] According to one embodiment of the present disclosure, a user can be helped to have good quality sleep.

[0427] A sleep management system according to one embodiment of the present disclosure comprises: a plurality of sensors (5) including a first sensor (51, 52, 53, 54, or 55) and a second sensor (51, 52, 53, 54, or 55) for collecting user data; a hub device (1) for receiving first data collected by the first sensor (51, 52, 53, 54, or 55) and second data collected by the second sensor (51, 52, 53, 54, or 55), obtaining first processed data based on processing the first data, and obtaining second processed data based on processing the second data; a user device (2) for receiving the first processed data and the second processed data from the hub device (1) via wireless communication, and inputting the first processed data and the second processed data into a machine learning model to obtain sleep state information related to the sleep state of the user; And it may include a server device (3) that receives the sleep state information from the user device (2) via wireless communication.

[0428] The above hub device (1) can input the first data into a first machine learning model (11, 12, 13, 14, or 15) to obtain the first processing data, and input the second data into a second machine learning model (11, 12, 13, 14, or 15) to obtain the second processing data.

[0429] The above hub device (1) can receive the user's data from the plurality of sensors (5) through wired communication.

[0430] The above hub device (1) can receive the first data from the first sensor (51, 52 or 53) through wired communication and can receive the second data from the second sensor (54 or 55) through wireless communication.

[0431] The above plurality of sensors (5) may include at least two of a pressure sensor, a UWB sensor, a radar sensor, an oxygen saturation sensor, an electrocardiogram sensor, or an acceleration sensor.

[0432] The sleep state information related to the sleep state of the user may include at least one of information about the sleep stage of the user, information about the user's stress index, or information about the user's sleep disorder.

[0433] The user device (2) can store a sleep management application including the machine learning model (21).

[0434] The above sleep management application may be downloadable from an external server.

[0435] The above server device (3) can control at least one home appliance (4) registered in conjunction with the user device (2) based on the sleep state information.

[0436] The server device (3) can control at least one home appliance (4) to perform a predetermined operation to wake up the user in response to the current time corresponding to the wake-up time and the user's sleep stage corresponding to a predetermined stage.

[0437] The server device (3) can control at least one home appliance (4) to perform a predetermined operation for inducing sleep of the user in response to the current time corresponding to sleep time and the sleep stage of the user corresponding to a predetermined stage.

[0438] The server device (3) can control at least one home appliance (4) to perform a predetermined operation to relieve the user's stress in response to the user's stress index exceeding a predetermined value.

[0439] The server device (3) can control at least one home appliance (4) to perform a predetermined operation to alleviate the developed sleep disorder in response to the development of a predetermined sleep disorder in the user.

[0440] The above hub device (1) may include a printed circuit board connected by wires to the plurality of sensors (5).

[0441] A sleep management method according to one embodiment of the present disclosure may include: a hub device (1) receiving first data collected by a first sensor (51, 52, 53, 54, or 55) and second data collected by a second sensor (51, 52, 53, 54, or 55); the hub device (1) obtaining first processed data based on processing the first data and obtaining second processed data based on processing the second data; the hub device (1) transmitting the first processed data and the second processed data to a user device (2) via wireless communication; the user device (2) inputting the first processed data and the second processed data received from the hub device (1) via wireless communication into a machine learning model to obtain sleep state information related to a sleep state of the user; and the user device (2) transmitting the sleep state information to a server device (3) via wireless communication.

[0442] The above hub device (1) may obtain the first processing data and the second processing data by inputting the first data into a first machine learning model (11, 12, 13, 14, or 15) to obtain the first processing data, and inputting the second data into a second machine learning model (11, 12, 13, 14, or 15) to obtain the second processing data.

[0443] The hub device (1) receiving the first data and the second data may include receiving the first data from the first sensor (51, 52 or 53) through wired communication and receiving the second data from the second sensor (51, 52 or 53) through wired communication.

[0444] The hub device (1) receiving the first data and the second data may include receiving the first data from the first sensor (51, 52 or 53) through wired communication and receiving the second data from the second sensor (54 or 55) through wireless communication.

[0445] The above sleep management method may further include the server device (3) controlling at least one home appliance (4) registered in conjunction with the user device (2) based on the sleep state information.

[0446] The controlling of the at least one home appliance (4) by the server device (3) may include at least one of: controlling the at least one home appliance (4) to perform a first predetermined operation for waking up the user in response to the current time corresponding to a wake-up time and the user's sleep stage corresponding to a first predetermined stage; controlling the at least one home appliance (4) to perform a second predetermined operation for inducing sleep of the user in response to the current time corresponding to a sleep time and the user's sleep stage corresponding to a second predetermined stage; controlling the at least one home appliance (4) to perform a third predetermined operation for alleviating stress of the user in response to the user's stress index exceeding a predetermined value; or controlling the at least one home appliance (4) to perform a fourth predetermined operation for alleviating a sleep disorder that has been expressed in the user in response to the expression of a sleep disorder that has been expressed.

[0447] Fig. 17 is a configuration diagram of a sleep management system according to another embodiment, which is described with reference to Figs. 18 to 21.

[0448] A sleep management system (2000) according to another embodiment may include an electronic device (2100) and an external sensor (2200).

[0449] The electronic device (2100) can obtain information on the user's sleep state based on bio-signals detected by a plurality of sensors, recognize the user's sleep stage based on the obtained information on the user's sleep state, and control the output based on the recognized sleep stage.

[0450] The electronic device (2100) may be a device that outputs at least one of video and audio.

[0451] The electronic device (2100) may be a multimedia device.

[0452] The electronic device (2100) may be a display device or a television.

[0453] The electronic device (2100) may be an audio device such as a sound bar.

[0454] The electronic device (2100) may be a user device such as a personal computer, a terminal, a mobile phone, a smart phone, a handheld device, or a wearable device.

[0455] An external sensor (2200) may be provided outside the electronic device (2100) and may communicate with the electronic device (2100).

[0456] The external sensor (2200) may also be detachably connected to the electronic device (2100).

[0457] An external sensor (2200) can detect a user's biosignal and transmit the detected biosignal to an electronic device (2100).

[0458] The external sensor (2200) can also obtain the user's image information and audio information to recognize the user's biosignal and transmit the obtained image information and audio information to the electronic device (2100).

[0459] The external sensor (2200) may include at least one of an image sensor for acquiring an image of the user and a microphone for receiving sound caused by the user's breathing.

[0460] The image sensor may include a CCD or CMOS image sensor.

[0461] The image sensor may include a camera. The image sensor may also include a 3D spatial recognition camera, such as a time-of-flight (TOF) camera or a stereo camera.

[0462] The image sensor can be installed in the room where the user sleeps.

[0463] There may be at least one or two or more external sensors (2200).

[0464] The external sensor (2200) may include at least one of a pressure sensor for detecting pressure applied by a user, a temperature sensor for detecting the user's body temperature, a heart rate sensor for detecting the user's heart rate, an electrocardiogram sensor for detecting the user's electrocardiogram, an oxygen saturation sensor for detecting the user's oxygen saturation, and a radar sensor and an acceleration sensor for detecting the user's movement.

[0465] The pressure sensor may include a single strip force sensitive resistor (FSR).

[0466] A single-strip force-sensitive resistor (FSR) is a sensor that detects pressure, weight, or touch by utilizing the principle that resistance value decreases when pressure is applied.

[0467] The pressure sensor may also include a liquid metal based pressure sensor.

[0468] The pressure sensor may also include a piezoelectric pressure sensor.

[0469] The pressure sensor can be placed on a mattress within the room where the user sleeps.

[0470] The radar sensor may include a transmitting antenna (or transmitting antenna array) that radiates radio waves toward a sleeping area within the room, and a receiving antenna (or receiving antenna array) that receives reflected radio waves reflected from obstacles or users in the sleeping area.

[0471] A radar sensor can obtain radar information from radio waves transmitted by a transmitting antenna and reflected radio waves received by a receiving antenna.

[0472] The radar sensor may include at least one of an ultra-wideband (UWB) radar sensor for detecting the presence and movement of a user using high-frequency radio waves, and a millimeter wave (mmWave) radar sensor for detecting the presence and movement of a user using millimeter radio waves.

[0473] Image sensors, microphones, and radar sensors may be non-contact sensors. Non-contact sensors may be installed within the room where the user sleeps.

[0474] Pressure sensors, temperature sensors, electrocardiogram sensors, acceleration sensors and heart rate sensors may be contact-type sensors.

[0475] Contact sensors may be installed on the mat on which the user lies.

[0476] Contact sensors can be worn on the user's body. Contact sensors can be installed in the sleeping area of ​​the room where the user sleeps.

[0477] As illustrated in FIG. 17, the electronic device (2100) may include a sensor unit (2110), a communication unit (2120), a processor (2130), a memory (2140), an output interface (2150), and an input interface (2160).

[0478] At least one component may be added or deleted in accordance with the performance of the components of the electronic device (2100) illustrated in FIG. 17. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be changed in accordance with the performance or structure of the electronic device (2100).

[0479] The sensor unit (2110) may include a plurality of sensors. The sensor unit (2110) may include a non-contact sensor.

[0480] For example, the sensor unit (2110) may include an image sensor as a first sensor (2111), a microphone as a second sensor (2112), and a radar sensor as a third sensor (2113).

[0481] Image sensors capture images of users using electronic devices.

[0482] The image sensor may include a CCD or CMOS image sensor.

[0483] The image sensor may include a camera. The image sensor may also include a 3D spatial recognition camera, such as a time-of-flight (TOF) camera or a stereo camera.

[0484] The microphone picks up sound from the user's breathing.

[0485] The radar sensor may include at least one of an ultra-wideband (UWB) radar sensor for detecting the presence and movement of a user using high-frequency radio waves, and a millimeter wave (mmWave) radar sensor for detecting the presence and movement of a user using millimeter radio waves.

[0486] The communication unit (2120) may include one or more components that enable communication between components within the electronic device (2100).

[0487] The communication unit (2120) may include one or more components that enable communication with an external device, such as at least one of a short-range communication module, a wired communication module, and a wireless communication module. Here, the external device may include an external sensor, a home appliance, a server device, and a user device.

[0488] The communication unit (2120) may include both wired and wireless networks.

[0489] The processor (2130) oversees control related to the operation of the electronic device (2100). There may be one or more processors (2130). That is, there may be at least one processor (2130).

[0490] The processor (2130) performs control related to sleep management using data stored in the memory (2140). The processor (2130) can also perform control related to the operation of the electronic device (2100) using data stored in the memory (2140).

[0491] The processor (2130) can receive a power on command or a power off command from the input interface (2160).

[0492] The processor (2130) can receive a content playback command, a content pause command, a content selection command, and a content termination command.

[0493] The processor (2130) can control audio output or video output in response to selected content.

[0494] The processor (2130) can control the output of audio and video in response to selected content.

[0495] The processor (2130) can control the activation of the sensor unit (2110) in response to the playback of content.

[0496] For example, content could be a program broadcast on television. Content could be multimedia being displayed on a user device.

[0497] The processor (2130) can also receive a sleep management command from the input interface (2160).

[0498] The processor (2130) can also control the deactivation of the sensor unit (2110) when a power-off command is received from the input interface (2160).

[0499] The processor (2130) can also receive a power off command from a remote controller (not shown).

[0500] If the electronic device is a television, the power off command may be a television power off command. In this case, the television may enter sleep mode.

[0501] The processor (2130) can also control the deactivation of the sensor unit (2110) when a content termination command is received from the input interface (2160).

[0502] If the electronic device is a user device, the content termination command may be a command to terminate content running on the user device. In this case, the user device may enter sleep mode and turn off the display.

[0503] The processor (2130) can recognize one or more biosignals based on information received from multiple sensors during playback of content, fuse the recognized one or more biosignals, and recognize the user's sleep state information based on the fused information.

[0504] The configuration of the processor (2130) that recognizes biosignals will be described in more detail.

[0505] The processor (2130) can recognize the user's movement based on the image received from the first sensor (2111) and recognize the user's breathing volume based on the recognized user's movement. The processor (2130) can recognize the breathing volume during a preset first reference time.

[0506] The processor (2130) can recognize the user's upper body image from the received image, recognize the movement of the upper body based on the recognized upper body image, and recognize the user's breathing volume based on the movement of the recognized upper body.

[0507] The processor (2130) can recognize the user's upper body image in real time and check the image acquisition time for the recognized upper body image.

[0508] The processor (2130) recognizes a first upper body image in which inhalation is performed and a second upper body image in which exhalation is performed based on the upper body image, confirms a first time at which the first upper body image is acquired and a second time at which the second upper body image is acquired, and can recognize the user's breathing volume based on the confirmed first and second times.

[0509] The processor (2130) can also recognize the user's heart rate based on the image. The processor (2130) can recognize the heart rate during a preset second reference time.

[0510] The processor (2130) can also recognize the breathing sound corresponding to the user's breathing among the sounds received from the second sensor (2112) and recognize the user's breathing volume based on the recognized breathing sound.

[0511] The processor (2130) can recognize the user's breathing sound in real time and check the time at which the breathing sound was received.

[0512] The processor (2130) can identify the recognized breathing sound as an inhalation sound and an exhalation sound, confirm the third time at which the inhalation sound was acquired and the fourth time at which the exhalation sound was acquired, and recognize the user's breathing volume based on the confirmed first and second times.

[0513] The processor (2130) can recognize the user's movement based on radio wave information received from the third sensor (2113) and recognize the user's heart rate and respiration rate based on the recognized user's movement.

[0514] When the third sensor (2113) is a radar sensor, the processor (2130) can obtain distance information and direction information from the radar sensor to the user's body based on the phase difference (or time difference) between the transmitted radio wave transmitted from the radar sensor and the reflected radio wave received by the radar sensor, and can recognize the user's body based on the obtained distance information and direction information.

[0515] The processor (2130) can acquire a body image by imaging the user's body based on distance information to each area of ​​the user's body when recognizing the user's body through a radar sensor.

[0516] The processor (2130) can identify an upper body area and a lower body area in the user's body image based on the body image.

[0517] The processor (2130) can identify the upper body region as a head region, a chest region, an abdomen region, and an arm region.

[0518] The processor (2130) can recognize the user's breathing volume based on distance information of the chest area or the abdomen area of ​​the body image, and can recognize the user's heart rate based on distance information of the chest area or the head area of ​​the body image.

[0519] The processor (2130) can recognize a biosignal based on information detected by at least one of an image sensor, a microphone, and a radar sensor.

[0520] The processor (2130) can also recognize a biosignal based on detection information received from an external sensor (2200) and recognize the user's sleep status information based on the recognized biosignal. This will be described in more detail below.

[0521] When the external sensor is a pressure sensor, the processor (2130) can recognize the user's movement based on pressure information received from the pressure sensor and recognize the user's breathing volume based on the recognized user's movement. The breathing volume may be the breathing volume during a preset first reference time.

[0522] Here, the user's movement may be the movement of the upper body caused by the user's breathing.

[0523] The processor (2130) can recognize pressure changes based on pressure information received from a pressure sensor, identify a fifth time point where the pressure increased and a sixth time point where the pressure decreased based on the recognized pressure changes, and recognize the respiratory volume based on the identified fifth and sixth times. Here, the pressure information can be a pressure value or a resistance value corresponding to pressure applied by a user.

[0524] If the external sensor is a pressure sensor, the processor (2130) can also recognize the user's heart rate based on pressure information received from the pressure sensor. The heart rate may be the heart rate during a preset second reference time.

[0525] If the external sensor is a temperature sensor, the processor (2130) can recognize the user's body temperature based on temperature information received from the temperature sensor.

[0526] When the external sensor is an oxygen saturation sensor, the processor (2130) can recognize the amount of oxygen in the user's blood (i.e., blood oxygen or oxygen saturation) based on the oxygen saturation information received from the oxygen saturation sensor.

[0527] If the external sensor is a heart rate sensor, the processor (2130) can recognize the heart rate based on the heart rate information detected by the heart rate sensor.

[0528] The processor (2130) can recognize the user's heart rate, the user's body temperature, and the user's oxygen saturation as the user's bio-signals.

[0529] The processor (2130) can recognize a biosignal based on at least one of image information of an image sensor, sound information of a microphone, pressure information of a pressure sensor, radar information of a radar sensor, body temperature detected by a temperature sensor, heart rate detected by a heart rate sensor, and oxygen saturation detected by an oxygen saturation sensor, and recognize the user's sleep state information based on the recognized biosignal.

[0530] The processor (2130) can recognize the user's biosignal by learning at least two or more pieces of detection information from the sensor unit (2110).

[0531] The processor (2130) can recognize the user's biosignal by learning the detection information of the sensor unit (2110) and the detection information of the external sensor (2200).

[0532] The processor (2130) can recognize the user's sleep state information by learning the user's bio-signal recognition.

[0533] For example, the processor (2130) can recognize biosignals by learning pre-stored biosignals, image information from an image sensor, sound information from a microphone, pressure information from a pressure sensor, radar information from a radar sensor, body temperature detected by a temperature sensor, heart rate detected by a heart rate sensor, and oxygen saturation detected by an oxygen saturation sensor, and can recognize the user's sleep state information by learning the recognized biosignals.

[0534] The processor (2130) can identify whether the user is in a sleeping state or a non-sleeping state based on the recognized sleep state information of the user, and if it is determined that the user is in a sleeping state, the processor can determine the user's sleep stage based on the recognized sleep state information.

[0535] The processor (2130) can recognize the user's breathing volume by applying a first weight to the breathing volume acquired by the image sensor and the breathing volume acquired by the microphone, and by applying a second weight to the breathing volume acquired by the radar sensor.

[0536] When receiving pressure information from a pressure sensor, the processor (2130) can recognize the user's breathing volume by applying a first weight to the breathing volume acquired by the image sensor and the breathing volume acquired by the microphone, applying a second weight to the breathing volume acquired by the radar sensor, and applying a second weight to the breathing volume acquired by the pressure sensor.

[0537] The processor (2130) determines the user's sleep stage as an arousal stage when the recognized user's breathing volume exceeds a first reference breathing volume, determines the user's sleep stage as stage 1 of a non-REM sleep stage when the recognized user's breathing volume is less than or equal to the first reference breathing volume and exceeds a second reference breathing volume, determines the user's sleep stage as stage 2 of a non-REM sleep stage when the recognized user's breathing volume is less than or equal to the second reference breathing volume and exceeds a third reference breathing volume, and determines the user's sleep stage as stage 3 of a non-REM sleep stage when the recognized user's breathing volume is less than or equal to the third reference breathing volume.

[0538] The second reference tidal volume may be less than the first reference tidal volume and more than the third reference tidal volume.

[0539] The processor (2130) can recognize the user's heart rate by applying a first weight to the heart rate acquired by the image sensor and a second weight to the heart rate acquired by the radar sensor.

[0540] When receiving pressure information from a pressure sensor, the processor (2130) can recognize the user's heart rate by applying a first weight to the heart rate acquired by the image sensor, applying a second weight to the heart rate acquired by the radar sensor, and applying a second weight to the respiratory volume acquired by the pressure sensor.

[0541] The processor (2130) determines the user's sleep stage as an arousal stage when the recognized user's heart rate exceeds a first reference heart rate, determines the user's sleep stage as stage 1 of the non-REM sleep stage when the recognized user's heart rate is lower than or equal to the first reference heart rate and higher than a second reference heart rate, determines the user's sleep stage as stage 2 of the non-REM sleep stage when the recognized user's heart rate is lower than or equal to the second reference heart rate and higher than a third reference heart rate, and determines the user's sleep stage as stage 3 of the non-REM sleep stage when the recognized user's heart rate is lower than or equal to the third reference heart rate.

[0542] The third reference heart rate may be lower than the first reference heart rate or higher than the third reference heart rate.

[0543] The processor (2130) can determine the user's sleep stage by applying a first weight to the biosignals acquired by the image sensor and the microphone, and applying a second weight to the biosignals acquired by the radar sensor and the pressure sensor.

[0544] When receiving temperature information from a temperature sensor, the processor (2130) may recognize the user's body temperature based on the temperature information received from the temperature sensor, and if the recognized body temperature is lower than the reference body temperature, determine the user's sleep stage as stage 2 of the non-REM sleep stage.

[0545] The processor (2130) can determine the user's sleep stage by applying a third weight to the recognized body temperature. That is, the processor (2130) can determine the user's sleep stage by applying a first weight to the biosignals acquired by the image sensor and the microphone, a second weight to the biosignals acquired by the radar sensor and the pressure sensor, and a third weight to the biosignals acquired by the temperature sensor.

[0546] The processor (2130) can improve the accuracy of determining the sleep stage by fusing bio-signals detected by multiple sensors and determining the user's sleep stage in real time based on the fusing bio-signals.

[0547] The processor (2130) can identify a user based on an image acquired by an image sensor and a pre-stored user image, and can identify a user based on a voice acquired by a microphone and a pre-stored voice.

[0548] The processor (2130) can identify a user based on user identification information received through the input interface (2160) and pre-stored user identification information.

[0549] The processor (2130) can also obtain the first, second, and third reference heart rates and the first, second, and third reference respiratory volumes corresponding to the identified user from information stored in the memory (2140).

[0550] The processor (2130) can also obtain the first and second reference speeds and the first, second, third, and fourth constant speeds corresponding to the identified user from the information stored in the memory (2140).

[0551] When the sleep stage is stage 1, the processor (2130) can determine whether the user has a tendency to wake up based on the bio-signals detected by the sensor unit (2110).

[0552] When the sleep stage is stage 1, the processor (2130) can determine whether the user has a tendency to wake up based on the biosignals detected by the sensor unit (2110) and the biosignals detected by the external sensor (2200).

[0553] Arousal tendency is not determined by the stage of arousal, but means that biosignals indicating a state of arousal are detected in stage 1.

[0554] For example, when the processor (2130) determines that the user's breathing volume exceeds the first reference breathing volume when the user's sleep stage is determined to be stage 1, the processor (2130) can check the number of times the user's breathing volume exceeds the first reference breathing volume, and if the number of times checked is determined to be greater than the reference number, it can determine that there is a tendency to be aroused.

[0555] When the processor (2130) determines that the user's heart rate exceeds the first reference heart rate while the user's sleep stage is determined to be stage 1, the processor can determine that there is a tendency to wake up if the number of times the user's heart rate exceeds the first reference heart rate is determined to be greater than the reference number.

[0556] When the sleep stage is stage 2, the processor (2130) can determine whether the user has a tendency to wake up based on the bio-signals detected by the sensor unit (2110).

[0557] When the sleep stage is stage 2, the processor (2130) can determine whether the user has a tendency to wake up based on the biosignals detected by the sensor unit (2110) and the biosignals detected by the external sensor (2200).

[0558] Arousal tendency means that biosignals indicating a state of arousal are detected at stage 2, although this is not determined by the stage of arousal.

[0559] For example, when the processor (2130) determines that the user's breathing volume exceeds the first reference breathing volume when the user's sleep stage is determined to be stage 2, the processor (2130) can check the number of times the user's breathing volume exceeds the first reference breathing volume, and if the number of times checked is determined to be greater than the reference number, it can determine that there is a tendency to be aroused.

[0560] When the processor (2130) determines that the user's heart rate exceeds the first reference heart rate while the user's sleep stage is determined to be stage 2, the processor can determine that there is a tendency to wake up if the number of times the user's heart rate exceeds the first reference heart rate is determined to be greater than the reference number.

[0561] The processor (2130) can also determine whether a user has a tendency to be aroused based on eye movements (i.e., eye movements) or brain waves. For example, if brain waves in the arousal stage are detected for a preset amount of time or less while the sleep stage is determined to be stage 1 or stage 2, the processor (2130) can determine that a tendency to be aroused exists.

[0562] The processor (2130) can acquire the user's sleep habits based on bio-signals for each sleep stage while the user is asleep, and store information about the acquired sleep habits. The processor (2130) can also learn sleep habits based on bio-signals whenever the user is asleep, and acquire sleep habits based on the learned results.

[0563] The processor (2130) can set the first and second reference speeds based on sleep habits of each sleep stage.

[0564] The processor (2130) can also determine whether there is a tendency to wake up in each sleep stage based on sleep habits in each sleep stage.

[0565] The processor (2130) can determine whether to adjust the first and second reference speeds for each sleep stage based on the presence or absence of an arousal tendency for each sleep stage.

[0566] The processor (2130) can also set the first, second, third, and fourth constant speeds to adjust the first and second reference speeds based on the presence or absence of a tendency to wake up in each sleep stage.

[0567] The processor (2130) can also determine in real time whether there is a tendency to wake up in each sleep stage based on bio-signals detected when the user is in a sleeping state.

[0568] The processor (2130) can also determine whether there is a tendency to wake up in each sleep stage based on the user's sleeping habits acquired through learning.

[0569] When the processor (2130) is in the awakening stage, it can check the output amount of the output interface (2150), obtain an average value of the checked output amount, and set the maximum output amount of the output interface controlled in the sleep state based on the obtained average value.

[0570] For example, the processor (2130) can obtain an average value of the output amount of the output interface for approximately 1 to 2 minutes during the awakening phase.

[0571] The processor (2130) can set the average value of the output amount of the output interface to the maximum output amount in the non-REM sleep stage and the REM sleep stage.

[0572] The average value of the output amount of the output interface may include the average value of the brightness amount of the display unit (2151).

[0573] The average value of the output volume of the output interface may include the average value of the volume of the speaker (2152).

[0574] The maximum output amount may include the maximum output amount of the brightness of the display unit (2151).

[0575] The maximum output may include the maximum output of the volume of the speaker (2152).

[0576] The processor (2130) may control the output interface so that the output amount of the output interface is reduced to a preset reference speed when the user's sleep stage is determined to be a light sleep stage. The preset reference speed may include a first reference speed and a second reference speed. The second reference speed may be faster than the first reference speed.

[0577] Light sleep stages may include stage 1 of non-REM sleep and stage 2 of non-REM sleep.

[0578] As illustrated in FIG. 18, the processor (2130) can control the output interface so that the output amount of the output interface is reduced to a first reference speed when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage.

[0579] The processor (2130) can control the output interface so that the output amount of the output interface is reduced to a second reference speed when the user's sleep stage is determined to be stage 2 of the non-REM sleep stage.

[0580] More specifically, when the processor (2130) determines that the output mode of the output interface is a video mode, it checks the current brightness of the display unit, and when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage, it controls the display unit (2151) so that the brightness of the display unit decreases at a first reference speed, and when the user's sleep stage changes to stage 2 of the non-REM sleep stage, it controls the display unit (2151) so that the brightness of the display unit (2151) decreases at a second reference speed.

[0581] For example, if the sleep stage is stage 1, the brightness of the display is 70%, and the change corresponding to the first reference speed is 3%, the processor can reduce the brightness of the display by 5% at 1-minute intervals.

[0582] If the user's sleep stage changes to stage 2 6 minutes after entering stage 1 and the change corresponding to the second reference speed is 5%, the processor can reduce the brightness of the display by 5% at 1-minute intervals. In other words, the processor can reduce the brightness of the display, which has been reduced to 52% in stage 1, by 5% at 1-minute intervals.

[0583] The processor (2130) can control the display unit so that the brightness of the display unit (2151) is gradually reduced from the current brightness level to a first reference speed or a second reference speed in response to the video mode.

[0584] When the processor (2130) determines that the output mode of the output interface is an audio mode, it checks the current volume of the speaker, and when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage, it controls the speaker (2152) so that the volume of the speaker (2152) decreases at a first reference speed, and when the user's sleep stage changes to stage 2 of the non-REM sleep stage, it controls the speaker (2152) so that the volume of the speaker decreases at a second reference speed.

[0585] For example, if the sleep stage is stage 1, the speaker volume is level 50, and the change corresponding to the first reference speed is level 1, the processor can decrease the speaker volume by level 1 at one-minute intervals.

[0586] If the user's sleep stage changes to stage 2 6 minutes after entering stage 1 and the change corresponding to the second reference speed is level 3, the processor can reduce the speaker volume by level 3 at one-minute intervals. In other words, the processor can reduce the volume of a speaker whose volume has decreased to level 44 in stage 1 by level 3 at one-minute intervals.

[0587] The processor (2130) can control the speaker so that the volume of the speaker gradually decreases from the current volume to a first reference speed or a second reference speed in response to the audio mode.

[0588] When the processor (2130) determines that the output mode of the output interface is the simultaneous mode, the processor (2130) checks the current brightness of the display and the current volume of the speaker, and when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage, the processor controls the display so that the brightness of the display decreases at a first reference speed and controls the speaker so that the volume of the speaker decreases at the first reference speed, and when the user's sleep stage changes to stage 2 of the non-REM sleep stage, the processor controls the display so that the brightness of the display decreases at a second reference speed and controls the speaker (2152) so that the volume of the speaker decreases at the second reference speed.

[0589] Simultaneous mode can be a mode where video and audio are output together.

[0590] The processor (2130) can control the display unit (2151) to gradually decrease from the current brightness level to the first reference speed in response to the simultaneous mode, and can control the speaker (2152) to gradually decrease from the current volume level to the first reference speed.

[0591]

[0592] As illustrated in FIG. 19, the processor (2130) can control the output interface so that the output amount of the output interface is reduced to a first reference speed when the user's sleep stage changes from stage 2 to stage 1.

[0593] Controlling the output interface so that the output amount of the output interface is reduced at a first reference speed may include controlling at least one of the brightness of the display and the volume of the speaker.

[0594] More specifically, the processor (2130) can control the display unit (2151) so that the brightness of the display unit decreases at a first reference speed when the user's sleep stage changes from stage 2 to stage 1 during the performance of the video mode.

[0595] For example, if the user's sleep stage changes from stage 2 to stage 1 while the brightness of the display is reduced to 30%, the processor can reduce the brightness of the display by 3% at one-minute intervals. In other words, the processor can reduce the brightness of the display by 3% at one-minute intervals starting from 30% brightness.

[0596] The processor (2130) can control the speaker (2152) so that the volume of the speaker is reduced at a first reference speed when the user's sleep stage changes from stage 2 to stage 1 during the performance of the audio mode.

[0597] For example, if the sleep stage changes from stage 1 to stage 2 while the volume is reduced to level 35, the processor can reduce the speaker volume by level 1 every minute. In other words, the processor can reduce the volume by level 3 every minute starting from level 35.

[0598] The processor (2130) can control the display unit (2151) so that the brightness of the display unit decreases at a first reference speed when the user's sleep stage changes from stage 2 to stage 1 during the execution of the simultaneous mode, and can control the speaker (2152) so that the volume of the speaker decreases at the first reference speed.

[0599] As illustrated in FIG. 19, the processor (2130) can control the reduction of the output amount of the output interface to be stopped when the user's sleep stage changes from stage 1 to the awake stage. The processor (2130) can check the output amount at the time of change when the user's sleep stage changes from stage 1 to the awake stage and control the output interface to maintain the checked output amount.

[0600] Controlling the decrease in the output amount of the output interface may include controlling at least one of the brightness of the display and the volume of the speaker.

[0601] More specifically, the processor (2130) can control the decrease in brightness of the display to be stopped when the user's sleep stage changes from stage 1 to the wake-up stage during the performance of the video mode.

[0602] For example, if the user's sleep stage changes from stage 1 to the awake stage while the brightness of the display is reduced to 20%, the processor can maintain the brightness of the display at 20% by stopping the control of reducing the brightness of the display.

[0603] The processor (2130) can control the reduction of the speaker volume to stop when the user's sleep stage changes from stage 1 to the wake-up stage during the audio mode.

[0604] For example, if the sleep stage changes from stage 1 to awake while the volume is reduced to level 25, the processor can stop the speaker from decreasing the volume so that the volume is maintained at level 25.

[0605] The processor (2130) can control to stop the decrease in brightness of the display and to stop the decrease in volume of the speaker when the user's sleep stage changes from stage 1 to the wake-up stage during the execution of the simultaneous mode.

[0606] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a first reference speed, it can determine whether the user has a tendency to be aroused based on at least one of the user's biosignal detected by the sensor unit (2110) and the user's biosignal detected by an external sensor.

[0607] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a first reference speed, if it is determined that there is no awakening tendency, the processor (2130) can increase the first reference speed by a certain amount and control the output interface so that the output amount of the output interface is reduced based on the increased first reference speed.

[0608] As illustrated in FIG. 20, the processor (2130) can control the output interface so that the output amount is reduced to the sum of the first reference speed (a) and the first constant speed (c1) and the summed speed (a+c1).

[0609] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a first reference speed, if it is determined that an awakening tendency exists, the processor (2130) can reduce the first reference speed by a certain amount and control the output interface so that the output amount of the output interface is reduced based on the reduced first reference speed.

[0610] As illustrated in FIG. 20, the processor (2130) can control the output interface so that the output amount is reduced to the subtracted speed (a-c2) by subtracting the second constant speed (c2) from the first reference speed (a).

[0611] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a second reference speed, it can determine whether the user has an arousal tendency based on at least one of the user's biosignal detected by the sensor unit (2110) and the user's biosignal detected by an external sensor.

[0612] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a second reference speed, if it is determined that there is no awakening tendency, the processor (2130) can increase the second reference speed by a certain amount and control the output interface so that the output amount of the output interface is reduced based on the increased second reference speed.

[0613] As illustrated in FIG. 21, the processor (2130) can control the output interface so that the output amount is reduced to the sum of the second reference speed (b) and the third constant speed (c3) and the summed speed (b+c3).

[0614] When the processor (2130) controls the output of the output interface so that the output amount of the output interface is reduced at a second reference speed, if it is determined that an awakening tendency exists, the processor (2130) can reduce the second reference speed by a certain amount and control the output interface so that the output amount of the output interface is reduced based on the reduced second reference speed.

[0615] As illustrated in FIG. 21, the processor (2130) can control the output interface so that the output amount is reduced to the subtracted speed (b-c4) by subtracting the fourth constant speed (c4) from the second reference speed (b).

[0616] Among the speeds for reducing the output amount of the output interface, the speed obtained by subtracting the second constant speed from the first reference speed may be the slowest, and the speed obtained by adding the third constant speed to the second reference speed may be the fastest.

[0617] The order of speed could be as follows:

[0618] (a-c2) < a <(a+c1) < (b-c4) < b < (b+c3)

[0619] When the sleep stage is stage 1 or stage 2, if there is a tendency to wake up, the rate at which the output decreases can be slowed down so that the user does not perceive the change in output.

[0620] In step 1, when the processor (2130) controls the reduction of the output amount at the first reference speed, it is also possible to control the output interface to be turned off when the reduced output amount becomes zero (0).

[0621] For example, when controlling the decrease in brightness of the display unit at the first reference speed in step 1, the processor (2130) can control the display unit to be turned off when the brightness becomes 0%.

[0622] In step 1, when the processor (2130) controls the decrease in the volume of the speaker at the first reference speed, the speaker can be turned off when the volume reaches level 0.

[0623] In step 2, when the processor (2130) controls the reduction of the output amount at the second reference speed, it is also possible to control the output interface to be turned off when the reduced output amount becomes zero (0).

[0624] For example, when controlling the decrease in brightness of the display unit at the second reference speed in step 2, the processor (2130) can control the display unit to be turned off when the brightness becomes 0%.

[0625] In step 2, when the processor (2130) controls the decrease in the volume of the speaker at the second reference speed, the processor can control the speaker to be turned off when the volume reaches level 0.

[0626]

[0627] The processor (2130) can identify a user and determine the user's sleep habits based on information stored in memory. The processor (2130) can also determine whether the user has a tendency to wake up in step 1 or step 2 by determining the user's sleep habits.

[0628] The processor (2130) can also adjust the first reference speed based on the awakening tendency confirmed from the information in the memory when the user's sleep stage is stage 1.

[0629] The processor (2130) can also adjust the second reference speed based on the awakening tendency confirmed from the information in the memory when the user's sleep stage is stage 2.

[0630] As illustrated in FIG. 18, the processor (2130) can control the output interface (2150) to turn off when the user's sleep stage changes to stage 3.

[0631] That is, the processor (2130) can turn off the display (2151) and speaker (2152) when the user's sleep stage changes to stage 3.

[0632] The processor (2130) can control the output interface so that the output amount of the output interface becomes 0 when the user's sleep stage changes to stage 3.

[0633] The processor (2130) can also control recording of content from the time the sleep stage is determined to be stage 1 when the user's sleep stage is determined to be stage 1.

[0634] The processor (2130) can also check the priorities of the multiple users when there are multiple recognized users, recognize the user with the highest priority among the confirmed priorities, and control a reduction in the output amount of the output interface based on the bio-signal of the recognized user.

[0635] The processor (2130) can control the reduction of the output amount based on the sleep stage so as not to interfere with the entry into the light sleep stage and can induce the entry into the deep sleep stage.

[0636] The processor (2130) can prevent power consumption from increasing by controlling the output interface to be turned off based on the sleep stage being determined to be stage 3. That is, power consumption in an electronic device according to the present embodiment can be reduced.

[0637] The memory (2140) can store weights for detection information for each sensor.

[0638] For example, the memory (2140) can store a first weight corresponding to detection information of an image sensor and a microphone, a second weight corresponding to a radar sensor and a pressure sensor, and a third weight corresponding to a temperature sensor.

[0639] The memory (2140) can store time information for a first reference time for checking the respiratory volume and a second reference time for checking the heart rate.

[0640] The memory (2140) can store reference information for determining sleep stages.

[0641] The baseline information may include first, second, and third baseline respiratory volumes, first, second, and third baseline heart rates, and baseline body temperature.

[0642] Baseline information may include baseline eye movement velocity, baseline oxygen saturation, and baseline electrocardiogram.

[0643] The memory (2140) can also store reference information for determining sleep stages for each user.

[0644] The memory (2140) can also store information about each user's sleep habits.

[0645] For example, sleep habits may include arousal tendencies.

[0646] The memory (2140) can store speed information for the first reference speed and the second reference speed for controlling the output amount.

[0647] The memory (2140) can store speed information for the first and second constant speeds for controlling the first reference speed, and can store speed information for the third and fourth constant speeds for controlling the second reference speed.

[0648] The speeds of the first, second, third and fourth schedules can be the same.

[0649] The speeds of the 1st, 2nd, 3rd and 4th schedules may be different from each other.

[0650] The first and third constant speeds may be equal to each other, and may be different from the second and fourth constant speeds. The second and fourth constant speeds may be equal to each other.

[0651] The memory (2140) can store identification information of multiple users and store speed information for the first and second reference speeds and the first, second, third, and fourth constant speeds for each user.

[0652] The memory (2140) and the processor (2130) may be implemented as separate chips. Alternatively, the memory (2140) and the processor (2130) may be implemented as a single chip.

[0653] The memory (2140) may be implemented as at least one of a non-volatile memory device such as a cache, a ROM (Read Only Memory), a PROM (Programmable ROM), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a flash memory, a volatile memory device such as a RAM (Random Access Memory), or a storage medium such as a hard disk drive (HDD) or a CD-ROM, but is not limited thereto.

[0654] The output interface (2150) may include a display unit (1251) that displays a video.

[0655] The display unit (1251) can display a video corresponding to a control command of the processor (1230), and can change the displayed video or turn off the screen in response to the control command of the processor (1230).

[0656] The display unit (1251) can adjust the brightness of the screen in response to a control command of the processor (1230). The display unit (1251) can decrease or increase the brightness of the screen in response to a control command of the processor (1230), and can maintain the brightness of the screen in response to a control command of the processor (1230).

[0657] The display unit (1251) may be provided as a cathode ray tube (CRT), a digital light processing (DLP) panel, a plasma display panel, a liquid crystal display (LCD) panel, an electroluminescence (EL) panel, an electrophoretic display (EPD) panel, an electrochromic display (ECD) panel, a light emitting diode (LED) panel, or an organic light emitting diode (OLED) panel, but is not limited thereto.

[0658] The display unit (1251) may be composed of a touch screen panel (TSP) that forms a mutual layer structure with a touch pad.

[0659] The output interface (2150) may include a speaker (1252) that outputs audio.

[0660] The speaker (1252) can output audio corresponding to the control command of the processor (1230), and can change the audio output corresponding to the control command of the processor (1230) or turn off the audio output.

[0661] The speaker (1252) can adjust the volume of audio in response to a control command of the processor (1230). The speaker (1252) can decrease or increase the volume of audio in response to a control command of the processor (1230), and can maintain the volume of audio in response to a control command of the processor (1230).

[0662] The input interface (2160) receives user input and transmits the received user input to the processor (2130).

[0663] User input may include a power on command and a power off command for the electronic device (2100).

[0664] User input may contain user-identifying information.

[0665] If there are multiple users, the input interface (2160) can further receive user identification information and priority information of the users.

[0666] The input interface (2160) can receive any one of a content selection command, a content playback command, a content pause command, and a content end command.

[0667] The input interface (2160) can receive brightness information of the display and volume information of the speaker.

[0668] The input interface (2160) can receive a command to turn off the speaker output.

[0669] The input interface (2160) may include hardware devices such as various buttons, switches, pedals, keyboards, mice, trackballs, various levers, handles, or sticks.

[0670] Additionally, the input interface (2160) may include a GUI (Graphical User Interface), i.e., a software device, such as a touch pad. The touch pad may be implemented as a touch screen panel (TSP) and may form a mutual layer structure with the screen.

[0671] The artificial intelligence-related functions according to the present disclosure are operated through a processor (2130) and a memory (2140). The processor (2130) may be composed of one or more processors. In this case, the one or more processors (130) may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU.

[0672] One or more processors (2130) control input data to be processed according to predefined operating rules or artificial intelligence models stored in memory (2140). Alternatively, if one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0673] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0674] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0675] At least one component may be added or deleted in accordance with the performance of the components of the electronic device (2100) illustrated in FIG. 17. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be changed in accordance with the performance or structure of the terminal device (130).

[0676] Meanwhile, each component illustrated in FIG. 17 refers to software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0677] Fig. 22 is a control flowchart of an electronic device according to another embodiment.

[0678] The electronic device plays content in response to a content playback command received through the input interface.

[0679] The electronic device can output video, audio, or both video and audio, depending on the type of content.

[0680] The electronic device detects the user's biosignals using multiple sensors while playing content (2301).

[0681] More specifically, the electronic device can recognize one or more biosignals based on information received from a plurality of sensors, fuse the recognized one or more biosignals, and recognize the user's sleep state information based on the fused information.

[0682] The electronic device can also receive the user's biosignals from external sensors. In this case, the electronic device can fuse the biosignals received from the external sensor with information received from multiple sensors and determine the user's sleep stage based on the fused information (2302).

[0683] Vital signs may include the user's breathing rate, heart rate, body temperature, eye movement, oxygen saturation, and electrocardiogram.

[0684] Determining a user's sleep stages involves determining the light sleep stage and the deep sleep stage of non-REM sleep.

[0685] Determining the light sleep stage of non-REM sleep involves determining stages 1 and 2 of non-REM sleep.

[0686] Determining the deep sleep stage of non-REM sleep involves determining stage 3 of non-REM sleep.

[0687] Describes an example of determining a user's sleep stage.

[0688] The electronic device may determine the user's sleep stage as an arousal stage if the user's breathing volume exceeds the first reference breathing volume, determine the user's sleep stage as stage 1 of the non-REM sleep stage if the recognized user's breathing volume is less than or equal to the first reference breathing volume and exceeds the second reference breathing volume, determine the user's sleep stage as stage 2 of the non-REM sleep stage if the recognized user's breathing volume is less than or equal to the second reference breathing volume and exceeds the third reference breathing volume, and determine the user's sleep stage as stage 3 of the non-REM sleep stage if the recognized user's breathing volume is less than or equal to the third reference breathing volume.

[0689] The second reference tidal volume may be less than the first reference tidal volume and more than the third reference tidal volume.

[0690] The electronic device may determine the user's sleep stage as an awake stage if the user's heart rate exceeds a first reference heart rate, determine the user's sleep stage as stage 1 of the non-REM sleep stage if the recognized user's heart rate is lower than or equal to the first reference heart rate and higher than the second reference heart rate, determine the user's sleep stage as stage 2 of the non-REM sleep stage if the recognized user's heart rate is lower than or equal to the second reference heart rate and higher than the third reference heart rate, and determine the user's sleep stage as stage 3 of the non-REM sleep stage if the recognized user's heart rate is lower than or equal to the third reference heart rate.

[0691] The third reference heart rate may be lower than the first reference heart rate or higher than the third reference heart rate.

[0692] The electronic device determines whether the user's sleep stage is determined to be a light sleep stage (2303).

[0693] The electronic device maintains and controls the output amount of the output interface (2150) when the user's sleep stage is determined to be an awake stage rather than a light sleep stage (2304).

[0694] The electronic device can control the output interface (2150) so that the output amount of the output interface is reduced at a preset reference speed when the user's sleep stage is determined to be a light sleep stage (2035).

[0695] The preset reference speed may include a first reference speed and a second reference speed. The second reference speed may be a speed faster than the first reference speed.

[0696] The electronic device can control a reduction in the output amount of the output interface based on the first reference speed and the second reference speed when the user's sleep stage is determined to be a light sleep stage.

[0697] That is, the electronic device can control the output interface (2150) so that the output amount of the output interface is reduced at a first reference speed when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage, and can control the output interface (2150) so that the output amount of the output interface is reduced at a second reference speed when the user's sleep stage is determined to be stage 2 of the non-REM sleep stage.

[0698] The electronic device can control the output interface to be turned off (2307) when the user's sleep stage is determined to be a deep sleep stage (2306).

[0699] Turning off the output interface may include turning off the display of the electronic device or turning off the speaker.

[0700] FIG. 23a and FIG. 23b are detailed control flowcharts for adjusting the output amount of an electronic device according to another embodiment.

[0701] The electronic device can determine the user's sleep stage by fusing biosignals received from an external sensor and biosignals received from multiple sensors.

[0702] Determining a user's sleep stages involves determining stages 1, 2, and 3 of non-REM sleep.

[0703] The electronic device can determine whether the user's sleep stage is determined to be stage 1 (2401).

[0704] The electronic device can control the output amount of the output interface in response to determining that the user's sleep stage is not stage 1 (2402).

[0705] Determining that the user's sleep stage is not stage 1 may include determining that the user is in an awake state.

[0706] The electronic device can control the output interface so that the output amount of the output interface is reduced at a first reference speed when the sleep stage is determined to be stage 1 (2403).

[0707] Controlling the output amount of the output interface to decrease at a first reference speed may include controlling the display unit (2151) to decrease the brightness of the display unit at the first reference speed.

[0708] Controlling the output volume of the output interface to decrease at a first reference rate may include controlling the speaker so that the volume of the speaker decreases at the first reference rate.

[0709] The electronic device can determine whether the user has a tendency to be aroused while controlling the output amount of the output interface to be reduced at a first reference speed (2404).

[0710] Determining whether a user has a tendency to be aroused can be accomplished by acquiring sleep state information by fusing bio-signals received from an external sensor and bio-signals received from multiple sensors, and determining whether a user has a tendency to be aroused based on the acquired sleep state information.

[0711] Determining whether a user has a tendency to be aroused can be done based on the user's sleeping habits stored in memory.

[0712] If the electronic device determines that the user does not have an arousal tendency, the electronic device can control the output interface so that the output amount of the output interface is reduced at a rate faster than the first reference rate (2405).

[0713] If the electronic device determines that there is no arousal tendency in step 1, it can increase the first reference speed by adding the first constant speed to the first reference speed.

[0714] If the electronic device determines that the user has an arousal tendency, it determines whether the user is in an arousal stage (2406), and if it determines that the user is in an arousal stage, it stops the control of reducing the output amount of the output interface (2407).

[0715] Although the electronic device is not in an arousal stage, if it is determined that the user has a tendency to be aroused, the electronic device can control the output interface so that the output amount of the output interface is reduced at a speed slower than the first reference speed (2408).

[0716] If the electronic device determines that an arousal tendency exists in step 1, it can reduce the first reference speed by subtracting the second constant speed from the first reference speed.

[0717] In this way, the electronic device can adjust the first reference speed to be faster or slower based on the user's arousal tendency in step 1.

[0718] The electronic device can determine whether the user's sleep stage has changed to stage 2 based on the biosignals received from the external sensor and the biosignals received from the plurality of sensors (2409).

[0719] The electronic device can control the output interface so that the output amount of the output interface is reduced to a second reference speed when the user's sleep stage changes to stage 2 (2410).

[0720] Controlling the output amount of the output interface to decrease at a second reference speed may include controlling the display unit (2151) to decrease the brightness of the display unit at the second reference speed.

[0721] Controlling the output volume of the output interface to decrease at a second reference rate may include controlling the speaker so that the volume of the speaker decreases at the second reference rate.

[0722] The electronic device can determine whether the user has a tendency to be aroused while controlling the output amount of the output interface to be reduced at a second reference speed (2411).

[0723] The electronic device can control the output interface so that the output amount of the output interface is reduced at a rate faster than the second reference rate when it is determined that the user does not have an arousal tendency (2412).

[0724] If the electronic device determines that there is no tendency to awaken in step 2, it can increase the second reference speed by adding the third constant speed to the second reference speed.

[0725] If the electronic device determines that the user has a tendency to be aroused, the electronic device can control the output interface so that the output amount of the output interface is reduced at a speed slower than the second reference speed (2413).

[0726] If the electronic device determines that there is an arousal tendency in step 2, it can reduce the second reference speed by subtracting the fourth constant speed from the second reference speed.

[0727] In this way, the electronic device can adjust the second reference speed to be faster or slower based on the user's arousal tendency in step 2.

[0728] The electronic device can determine whether the user's sleep stage changes from stage 2 to stage 1 based on the biosignals received from the external sensor and the biosignals received from multiple sensors (2414).

[0729] The electronic device can control the output interface so that the output amount of the output interface is reduced to a first reference speed when the user's sleep stage changes from stage 2 to stage 1 (2415).

[0730] The electronic device can determine whether a change has occurred from step 2 to step 3 if it determines that a change has not occurred from step 2 to step 1 (2416).

[0731] The electronic device can control the output of the output interface to turn off when the user's sleep stage changes from stage 2 to stage 3 (2417).

[0732] Controlling the output of the output interface off includes controlling at least one of the display and speaker of the electronic device off.

[0733] Figure 24 is a configuration diagram of a sleep management system according to another embodiment.

[0734] A sleep management system (3000) according to another embodiment may include a plurality of sensors (3100), a hub device (3200), and electronic devices (3300, 3400).

[0735] A plurality of sensors (3100) can detect a user's biosignal and transmit the detected biosignal to a hub device (3200).

[0736] The plurality of sensors (3100) may include at least one of an image sensor for acquiring an image of the user and a microphone for receiving sound caused by the user's breathing.

[0737] The plurality of sensors (3100) may include at least one of a pressure sensor for detecting pressure applied by a user, a temperature sensor for detecting the body temperature of the user, a heart rate sensor for detecting the heart rate of the user, an electrocardiogram sensor for detecting the electrocardiogram of the user, an oxygen saturation sensor for detecting the oxygen saturation of the user, and a radar sensor and an acceleration sensor for detecting the movement of the user.

[0738] The radar sensor may include at least one of an ultra-wideband (UWB) radar sensor for detecting the presence and movement of a user using high-frequency radio waves, and a millimeter wave (mmWave) radar sensor for detecting the presence and movement of a user using millimeter radio waves.

[0739] Image sensors, microphones, and radar sensors may be non-contact sensors. Non-contact sensors may be installed within the room where the user sleeps.

[0740] Pressure sensors, temperature sensors, electrocardiogram sensors, acceleration sensors and heart rate sensors may be contact-type sensors.

[0741] Contact sensors may be installed on the mat on which the user lies.

[0742] Contact sensors can be worn on the user's body. Contact sensors can be installed in the sleeping area of ​​the room where the user sleeps.

[0743] Some of the plurality of sensors (3100) may be provided in a wearable device. For example, the sensors provided in the wearable device may include a heart rate sensor, a bioelectrical impedance analysis sensor, a body temperature sensor, and an acceleration sensor.

[0744] Some of the multiple sensors (3100) may also be provided in the electronic device (3300, 3400).

[0745] The hub device (3200) can execute an application of a sleep management platform that provides a service for managing the user's sleep. The application herein may be an application for providing a sleep management service.

[0746] The hub device (3200) can control the download, installation (setup) and execution of an application for providing sleep management services.

[0747] The hub device (3200) can communicate with a server for sleep management, a web server for a website, an application server providing a sleep monitoring service, and a database server.

[0748] The hub device (3200) may include a communication module capable of communicating with an electronic device (3300, 3400) and / or a server device (not shown), at least one processor for processing data, and at least one memory storing a program for controlling the operation of the hub device (3200).

[0749] The hub device (3200) can communicate with multiple sensors (3100) and electronic devices (3300, 3400).

[0750] It is also possible to receive bio-signals of at least one of the user's heart rate, blood pressure, electrocardiogram and brain waves from the user device (20).

[0751] The hub device (100) monitors the user's sleep state based on the detected user's bio-signals and recognizes the user's sleep stage based on the monitored user's sleep state information.

[0752] The hub device (3200) can communicate with an electronic device and transmit control commands to the electronic device.

[0753] The hub device (3200) recognizes the location information of electronic devices through communication with the electronic devices, and based on the location information of the recognized electronic devices, recognizes electronic devices provided in the same room as the room in which the user sleeps, and can also transmit information on the user's sleep status to the recognized electronic devices.

[0754] The hub device (3200) can recognize one or more biosignals based on information received from a plurality of sensors (3100), fuse the recognized one or more biosignals, and determine the user's sleep stage based on the fused information.

[0755] The configuration for determining the user's sleep stage based on the fusion information is the same as the sleep stage determination configuration in the processor of Fig. 17, and thus, a description thereof is omitted.

[0756] The hub device (3200) can identify a user based on an image acquired by an image sensor among a plurality of sensors (3100) and a pre-stored user image, and can identify a user based on a voice acquired by a microphone among a plurality of sensors (3100) and a pre-stored voice.

[0757] The hub device (3200) can also obtain the first and second reference speeds and the first, second, third, and fourth constant speeds corresponding to the user identified from pre-stored information.

[0758] The hub device (3200) can determine whether the user has a tendency to wake up based on bio-signals detected by multiple sensors (3100) when the sleep stage is stage 1.

[0759] The hub device (3200) can determine whether the user has a tendency to wake up based on bio-signals detected by multiple sensors (3100) when the sleep stage is stage 2.

[0760] Arousal tendency is not determined by the stage of arousal, but means that biosignals indicating a state of arousal are detected in stage 1 or stage 2.

[0761] The hub device (3200) can acquire the user's sleep habits based on bio-signals for each sleep stage while the user is asleep, and store information about the acquired sleep habits. The processor (2130) can also learn the user's sleep habits based on bio-signals whenever the user is asleep, and acquire the sleep habits based on the learned results.

[0762] The hub device (3200) can set the first and second reference speeds based on sleep habits of each sleep stage.

[0763] The hub device (3200) can also determine whether there is a tendency to wake up in each sleep stage based on sleep habits in each sleep stage.

[0764] The hub device (3200) can determine whether to adjust the first and second reference speeds for each sleep stage based on the presence or absence of an arousal tendency for each sleep stage.

[0765] The hub device (3200) can also set the first, second, third, and fourth constant speeds to adjust the first and second reference speeds based on the presence or absence of a tendency to wake up in each sleep stage.

[0766] The hub device (3200) can also determine in real time whether there is a tendency to wake up in each sleep stage based on bio-signals detected when the user is in a sleeping state.

[0767] The hub device (3200) can also determine whether there is a tendency to wake up in each sleep stage based on the user's sleeping habits acquired through learning.

[0768] The hub device (3200) can check the output amount of the electronic device (3300, 3400) when in the awakening stage, obtain an average value of the checked output amount, and set the maximum output amount of the electronic device (3300, 3400) controlled in the sleep state based on the obtained average value.

[0769] For example, the hub device (3200) can obtain an average value of the output of the electronic device (3300, 3400) for approximately 1 to 2 minutes when in the awakening stage.

[0770] The hub device (3200) can set the average value of the output of the electronic devices (3300, 3400) to the maximum output in the non-REM sleep stage and the REM sleep stage.

[0771] The average value of the output of the electronic device (3300, 3400) may include the average value of the brightness of the display unit.

[0772] The average value of the output of the electronic device (3300, 3400) may include the average value of the loudness of the speaker.

[0773] The maximum output may include the maximum output of the brightness of the display of the electronic device (3300, 3400).

[0774] The maximum output may include the maximum output volume of the speaker of the electronic device (3300, 3400).

[0775] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced to a preset reference speed when the user's sleep stage is determined to be a light sleep stage. The preset reference speed may include a first reference speed and a second reference speed. The second reference speed may be a speed faster than the first reference speed.

[0776] Light sleep stages may include stage 1 of non-REM sleep and stage 2 of non-REM sleep.

[0777] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced to a first reference speed when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage.

[0778] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced to a second reference speed when the user's sleep stage is determined to be stage 2 of the non-REM sleep stage.

[0779] The hub device (3200) can control the electronic device to turn off when the user's sleep stage is determined to be stage 3.

[0780] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced to a first reference speed when the user's sleep stage changes from stage 2 to stage 1.

[0781] Controlling the electronic device so that the output of the electronic device (3300, 3400) is reduced at a first reference speed may include controlling at least one of the brightness of the display and the volume of the speaker.

[0782] Controlling the electronic device (3300, 3400) so that the output amount of the electronic device is reduced at a first reference speed includes transmitting a control command to the electronic device (3300, 3400) to reduce the output amount at the first reference speed.

[0783] The hub device (3200) can control the reduction in output of the electronic device (3300, 3400) when the user's sleep stage changes from stage 1 to the wakefulness stage. In this case, the hub device (3200) can transmit a control command to the electronic device (3300, 3400) to stop the reduction in output.

[0784] When the hub device (3200) controls the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced at a first reference speed, the hub device (3200) can determine whether the user has a tendency to be aroused based on the user's bio-signals detected by a plurality of sensors.

[0785] When the hub device (3200) determines that there is no awakening tendency when the output of the electronic device is controlled at the first reference speed, the hub device (3200) can increase the first reference speed by a certain amount and transmit information about the increased first reference speed to the electronic device (3300, 3400).

[0786] The hub device (3200) can add the first constant speed (c1) to the first reference speed (a) and transmit information about the added speed (a+c1) to the electronic device (3300, 3400).

[0787] When the hub device (3200) controls the output of the electronic device so that the output of the electronic device is reduced to a first reference speed, if it is determined that an arousal tendency exists, the hub device (3200) can reduce the first reference speed by a certain amount and transmit information about the reduced first reference speed to the electronic device (3300, 3400).

[0788] The hub device (3200) can subtract the second constant speed (c2) from the first reference speed (a) and transmit information about the subtracted speed (a-c2) to the electronic device (3300, 3400).

[0789] The hub device (3200) can determine whether the user has a tendency to be aroused based on the user's bio-signals detected by a plurality of sensors when controlling the electronic device so that the output of the electronic device is reduced at a second reference speed.

[0790] When the hub device (3200) controls the electronic device so that the output of the electronic device is reduced to a second reference speed, if it is determined that there is no tendency to awaken, the hub device (3200) can increase the second reference speed by a certain amount and transmit information about the increased second reference speed to the electronic device (3300, 3400).

[0791] The hub device (3200) can add the third constant speed (c3) to the second reference speed (b) and transmit information about the added speed (b+c3) to the electronic device (3300, 3400).

[0792] When the hub device (3200) controls the electronic device so that the output of the electronic device is reduced to a second reference speed, if it is determined that an arousal tendency exists, the hub device (3200) can reduce the second reference speed by a certain amount and transmit information about the reduced second reference speed to the electronic device (3300, 3400).

[0793] The hub device (3200) can subtract the fourth constant speed (c4) from the second reference speed (b) and transmit information about the subtracted speed (b-c4) to the electronic device (3300, 3400).

[0794] In step 1, when the hub device (3200) controls the reduction of the output amount at the first reference speed, it is also possible to control the electronic device to turn off when the reduced output amount becomes zero (0).

[0795] In step 2, when the hub device (3200) controls the reduction of the output amount at the second reference speed, it is also possible to control the electronic device to turn off when the reduced output amount becomes zero (0).

[0796] When controlling an electronic device, the hub device (3200) can also transmit output information to the electronic device by controlling the output of the electronic device. The output information may include speed information for reducing the output amount.

[0797] When controlling an electronic device, the hub device (3200) can transmit a control command for output control of the electronic device to the electronic device.

[0798] For example, the hub device (3200) can transmit a first control command for output reduction control at a first reference speed to the electronic device, a second control command for output reduction control at a second reference speed to the electronic device, and a third control command for output off control to the electronic device.

[0799] The hub device (3200) can transmit a fourth control command for increasing the first reference speed, a fifth control command for decreasing the first reference speed, a sixth control command for increasing the second reference speed, and a seventh control command for decreasing the second reference speed to the electronic device in response to the fourth control command.

[0800] When controlling an electronic device, the hub device (3200) can identify an electronic device in an operating state and an electronic device in a stationary state, and transmit a control command to reduce the output amount to the electronic device in an operating state.

[0801] The hub device (3200) can also check the priorities of multiple users when there are multiple recognized users, recognize the user with the highest priority among the confirmed priorities, and control a reduction in the output of the electronic device based on the bio-signal of the recognized user.

[0802] The hub device (3200) can also check for an electronic device that generates noise when it determines that the user is in a sleeping state, and transmit a motion control command to the identified electronic device.

[0803] The electronic device may be a device that outputs at least one of audio and video.

[0804] The electronic device may be a multimedia device.

[0805] The electronic device may be a display device, such as a television or a projector.

[0806] The electronic device may include a home appliance (3300). For example, the home appliance (3300) may include at least one of an air conditioner, a dehumidifier, a humidifier, a heater, a refrigerator, a clothes manager, a dryer, an air purifier, and a washing machine, and may further include an audio device such as a sound bar.

[0807] The home appliance (3300) may include, but is not limited to, at least one of a dishwasher, an electric range, an electric oven, a microwave oven, and a cleaning robot.

[0808] The home appliance (3300) may be a lighting device that changes the indoor illuminance and an automatic curtain opening / closing device.

[0809] The electronic device may further include a user device (3400) such as a personal computer, a terminal, a mobile phone, a smart phone, a handheld device, or a wearable device.

[0810] The electronic device is a communication-capable device and can communicate with a hub device (3200).

[0811] The electronic device can control the output amount of at least one of audio and video in response to a control command received from the hub device (3200).

[0812] The electronic device can gradually reduce the output amount in response to a control command received from the hub device (3200).

[0813] For example, the electronic device may reduce the output amount at a first reference speed in response to a first control command, reduce the output amount at a second reference speed in response to a second control command, and turn off the output in response to a third control command.

[0814] The electronic device can increase the first reference speed in response to the fourth control command, decrease the first reference speed in response to the fifth control command, increase the second reference speed in response to the sixth control command, and decrease the second reference speed in response to the seventh control command.

[0815] The electronic device can gradually reduce the output based on the sleep stage received from the hub device (3200).

[0816] For example, the electronic device may reduce the output at a first reference rate when the sleep stage is stage 1 of non-REM sleep, reduce the output at a second reference rate when the sleep stage is stage 2 of non-REM sleep, and turn off the output when the sleep stage is stage 3 of non-REM sleep.

[0817] The electronic device can change the rate of decrease in output based on the sleep stage and presence of a wake-up tendency received from the hub device (3200).

[0818] The electronic device can increase the first reference speed when the sleep stage is stage 1 of the non-REM sleep stage and there is no tendency to wake up, and can decrease the first reference speed when the sleep stage is stage 1 of the non-REM sleep stage and there is a tendency to wake up.

[0819] The electronic device can increase the second reference speed when the sleep stage is stage 2 of the non-REM sleep stage and there is no tendency to wake up, and can decrease the second reference speed when the sleep stage is stage 2 of the non-REM sleep stage and there is a tendency to wake up.

[0820] Electronic devices can stop reducing output when the sleep stage is an awake stage.

[0821] If the home appliance is a television, the television can recognize the sleep stage based on the sleep state information received from the sleep monitoring device (100) and reduce the volume of the television or the brightness of the television screen based on the recognized sleep stage.

[0822] Additionally, the television can be turned off based on sleep stages.

[0823] If the home appliance is a lighting device, the lighting device can determine a sleep stage based on the user's sleep state information received from the hub device (3200), and if the determined sleep stage is stage 1 of the non-REM sleep stage, the brightness can be gradually reduced.

[0824] The lighting device can reduce the brightness at a first reference speed when the sleep stage is stage 1 of the non-REM sleep stage, reduce the brightness at a second reference speed when the sleep stage is stage 2 of the non-REM sleep stage, and turn off the power when the sleep stage is stage 3 of the non-REM sleep.

[0825] If the home appliance is an automatic curtain opening / closing device, the automatic curtain opening / closing device can recognize the sleep stage based on the user's sleep state information received from the hub device (3200) and open or close the curtain based on the recognized sleep stage.

[0826] The electronic device can stop, resume, or change the currently performing operation to another operation in response to a control command received from the hub device (3200).

[0827] For example, if the home appliance is a refrigerator, when sleep state information is received from the hub device (3200), the refrigerator can reduce the rotation speed of the compressor, reduce the rotation speed of the fan, or stop or delay the ice-making operation based on the received sleep state information.

[0828] When the home appliance is an air conditioner, the air conditioner can change the operation mode of the air conditioner to a sleep mode based on the user's sleep state information received from the hub device (3200) and control the operation of at least one of the compressor and the fan based on the changed sleep mode.

[0829] The air conditioner can adjust target temperature, target wind direction, and target wind speed based on the sleep mode, and control dehumidification. The target temperature, target wind direction, and target wind speed of the air conditioner corresponding to the sleep mode may be information acquired through testing and stored in advance.

[0830] Fig. 25 is a control flowchart of a sleep management system according to an embodiment.

[0831] The multiple sensors may include cameras and microphones.

[0832] The plurality of sensors may include at least one of a pressure sensor for detecting pressure applied by a user, a temperature sensor for detecting the body temperature of the user, a heart rate sensor for detecting the heart rate of the user, an electrocardiogram sensor for detecting the electrocardiogram of the user, an oxygen saturation sensor for detecting the oxygen saturation of the user, and a radar sensor and an acceleration sensor for detecting the movement of the user.

[0833] Multiple sensors can detect the user's biosignals (3501) and transmit the detected biosignals to the hub device.

[0834] The hub device can recognize the user's biosignals based on the user's video information and audio information.

[0835] The hub device (3200) can also directly receive biosignals from multiple sensors.

[0836] The hub device (100) can recognize one or more biosignals based on information received from a plurality of sensors (3100), fuse the recognized one or more biosignals, and determine the user's sleep stage based on the fused information (3502).

[0837] When the user's sleep stage is determined to be a light sleep stage (3503), the hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced to a preset reference speed.

[0838] In this case, the hub device (3200) can generate an output reduction control command for output reduction control and transmit the generated output reduction control command to the electronic device (3504).

[0839] The preset reference speed may include a first reference speed and a second reference speed. The second reference speed may be a speed faster than the first reference speed.

[0840] Light sleep stages may include stage 1 of non-REM sleep and stage 2 of non-REM sleep.

[0841] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced at a first reference speed when the user's sleep stage is determined to be stage 1 of the non-REM sleep stage. In this case, the hub device can transmit a first control command for output reduction control at the first reference speed to the electronic device.

[0842] The hub device (3200) can control the electronic device (3300, 3400) so that the output of the electronic device (3300, 3400) is reduced at a second reference speed when the user's sleep stage is determined to be stage 2 of the non-REM sleep stage. In this case, the hub device can transmit a second control command to the electronic device for output reduction control at the second reference speed.

[0843] The electronic device controls at least one output of video and audio in response to a user input (3505), and when a control command is received from a hub device during the at least one output control, the electronic device can control the output amount of at least one of video and audio to be reduced (3506).

[0844] For example, when a first control command is received from a hub device, the electronic device can control to reduce the output amount of at least one of audio and video based on a first reference speed, and when a second control command is received from the hub device, the electronic device can control to reduce the output amount of at least one of audio and video based on a second reference speed.

[0845] The electronic device can reduce the brightness of the display at a first reference speed when a first control command is received from the hub device while performing the video mode, and can reduce the volume of the speaker at the first reference speed when a first control command is received from the hub device while performing the audio mode.

[0846] The electronic device can reduce the brightness of the display at a second reference speed when a second control command is received from the hub device while performing the video mode, and can reduce the volume of the speaker at a second reference speed when a second control command is received from the hub device while performing the audio mode.

[0847] The electronic device can control the display or speaker to turn off when the output amount becomes 0 during the output amount reduction control by the first control command.

[0848] The electronic device can control the display or speaker to turn off when the output amount becomes 0 during the output amount reduction control by the second control command.

[0849] When the user's sleep stage is determined to be stage 3 (3507), the hub device (3200) can generate an output off control command for turning off the output of the electronic device and transmit the generated output off control command to the electronic device (3508). In this case, the hub device can transmit a third control command for output off control to the electronic device.

[0850] The electronic device can control off output of at least one of video and audio when a third control command is received from the hub device (3509).

[0851] The electronic device can turn off the display when a third control command is received from the hub device while performing in video mode, and can turn off the speaker when a third control command is received from the hub device while performing in audio mode.

[0852] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0853] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0854] Additionally, a computer-readable recording medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0855] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable recording medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated on a machine-readable recording medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0856] The above illustrates and describes specific embodiments. However, the invention is not limited to the above-described embodiments, and those skilled in the art will readily appreciate that various modifications and implementations can be made without departing from the spirit and scope of the invention as set forth in the claims below.

Claims

1. Multiple sensors that detect the user's vital signs; An output interface that outputs at least one of video and audio; and An electronic device including a processor that determines a user's sleep stage based on the user's bio-signals detected by the plurality of sensors and controls the output interface so that the output amount of the output interface is reduced at a preset rate based on the determined sleep stage.

2. In paragraph 1, The above preset speed includes a first reference speed and a second reference speed that is faster than the first reference speed, The processor is an electronic device that controls the output interface so that the output amount is reduced to the first reference speed or the second reference speed based on whether the sleep stage is determined to be a light sleep stage.

3. In the second paragraph, the processor, An electronic device that determines a rate of decrease in the output amount as the first reference rate based on the sleep stage being determined as stage 1 of the shallow sleep stage, determines a rate of decrease in the output amount as the second reference rate based on the sleep stage being determined as stage 2 of the shallow sleep stage, and determines a rate of decrease in the output amount as the first reference rate based on the sleep stage changing from stage 2 to stage 1.

4. In paragraph 2, The processor is an electronic device that determines a rate of decrease in the output amount as the first reference rate based on the change in the sleep stage from the stage 2 to the stage 1.

5. In paragraph 2, An electronic device in which the processor determines whether there is an arousal tendency based on the bio-signal when the sleep stage is the stage 1, determines a decrease rate of the output amount at a speed slower than the first reference speed based on the presence of the arousal tendency, and determines a decrease rate of the output amount at a speed faster than the first reference speed based on the non-existence of the arousal tendency.

6. In paragraph 2, An electronic device in which the processor determines whether there is an arousal tendency based on the bio-signal when the sleep stage is the stage 2, determines a decrease rate of the output amount at a speed slower than the second reference speed based on the presence of the arousal tendency, and determines a decrease rate of the output amount at a speed faster than the second reference speed based on the non-existence of the arousal tendency.

7. In paragraph 1, The above processor is an electronic device that controls the output interface to be turned off based on whether the sleep stage is determined to be a deep sleep stage.

8. In paragraph 1, The above processor controls the output interface so that the output amount of the output interface becomes less than a preset output amount in response to the sleep stage being recognized as a light sleep stage, An electronic device in which the above preset output amount is set by the average value of the output amount of the output interface at the awakening stage.

9. In paragraph 1, The above output interface includes at least one of a display unit for outputting the video; and a speaker for outputting the audio. An electronic device in which the output amount of the above output interface includes at least one of the video brightness amount and the audio volume amount.

10. In paragraph 1, Further comprising a communication unit that performs communication with at least one external sensor that detects the user's bio-signal, The processor determines the sleep stage of the user based on the biosignals detected from the at least one external sensor and the biosignals detected from the plurality of sensors, The above plurality of sensors include at least two of a microphone, a camera and a wireless radar sensor, An electronic device wherein said at least one external sensor comprises at least one of a microphone, a camera, a heart rate sensor, an electrocardiogram sensor, a temperature sensor, a pressure sensor, an acceleration sensor, a wireless radar sensor, and an oxygen saturation sensor.

11. In paragraph 1, an input interface for receiving user input; and It further includes memory for storing information about preset speeds for each user, An electronic device in which the processor recognizes a user based on the user input and obtains the preset speed corresponding to the recognized user based on the bio-signal.

12. Multiple sensors for detecting the user's vital signs; An electronic device that outputs at least one of video and audio; and A sleep management system including a hub device that determines a user's sleep stage based on the user's bio-signals detected by the plurality of sensors and transmits a control command to the electronic device to reduce the output amount of the electronic device at a preset rate based on the determined sleep stage.

13. In paragraph 12, The above preset speed includes a first reference speed and a second reference speed that is faster than the first reference speed, A sleep management system in which the hub device controls the electronic device so that the output amount is reduced to the first reference speed or the second reference speed based on whether the sleep stage is determined to be a light sleep stage.

14. In paragraph 13, An electronic device wherein the hub device determines a rate of decrease in the output amount as the first reference rate based on the sleep stage being determined as stage 1 of the shallow sleep stage, determines a rate of decrease in the output amount as the second reference rate based on the sleep stage being determined as stage 2 of the shallow sleep stage, determines a rate of decrease in the output amount as the first reference rate based on the sleep stage changing from stage 2 to stage 1, and transmits a control command to the electronic device to stop the decrease in the output amount based on the sleep stage changing from stage 1 to an arousal stage.

15. In paragraph 12, A sleep management system in which the hub device transmits a control command to the electronic device to turn off the electronic device based on the sleep stage being determined as the deep sleep stage, and transmits a control command to the electronic device to control the output amount of the electronic device to a preset output amount or less in response to the sleep stage being determined as the light sleep stage.

Images