Method, device and system for creating an environment through AI-based non-contact sleep analysis

Abstract

The present invention relates to a method for controlling an environment creation device, the method including: an acquisition step of acquiring environmental sensing information; a preprocessing step of performing preprocessing on the acquired environmental sensing information; a generation step of generating sleep state information based on the preprocessed environmental sensing information; and a control step of controlling the environment creation device based on the generated sleep state information.

Description

[Technical Field]

[0001] The present invention relates to a method, device, and system for creating an environment through AI-based non-contact sleep analysis. [Background technology]

[0002] True healthcare requires 24 / 7 monitoring and management, because health monitoring and management is not a simple one-to-one match, but rather a complex interrelationship of all factors.

[0003] There are various ways to maintain and improve health, such as exercise and diet, but the most important thing is to manage sleep well, which takes up more than 30% of the day.

[0004] However, despite the fact that modern people have been able to easily replace labor with machines and enjoy a more comfortable lifestyle, they are unable to get a good night's sleep due to irregular eating habits, lifestyle habits, and stress, and suffer from sleep disorders such as insomnia, excessive sleep, sleep apnea syndrome, nightmares, night terrors, and sleepwalking.

[0005] According to the National Health Insurance Service, the number of people with sleep disorders in Korea increased by an average of 8% per year from 2014 to 2018, and the number of people who received treatment for sleep disorders in Korea in 2018 reached approximately 570,000.

[0006] Deep sleep is recognized as an important factor affecting physical and mental health, and interest in deep sleep is increasing. However, in order to improve sleep disorders, patients must visit specialized medical institutions in person, which requires additional testing fees, and continuous management is difficult, so users are not making enough effort to seek treatment.

[0007] As sleep problems become more serious, the need for sleep health management is increasing, and the sleep tech market, which aims to solve sleep problems with technology, is also growing rapidly.

[0008] Korean Patent Publication No. 2003-0032529 discloses a sleep induction device and sleep induction method that receives input of a user's physical information and outputs vibrations and / or ultrasound waves in a frequency band detected through repetitive learning according to the user's physical condition during sleep, thereby enabling optimal sleep induction.

[0009] However, conventional technologies can reduce sleep quality due to the inconvenience caused by body-worn devices and require periodic maintenance of the devices (e.g., charging).

[0010] In addition, conventional sleep analysis methods using wearable devices have the problem that sleep analysis is impossible if the wearable device is not in proper contact with the user's body or if the user is not wearing the wearable device.

[0011] Furthermore, when multiple users sleep in the same space, the movements of those not wearing the wearable device can interfere with sleep analysis of those wearing the wearable device, and it is impossible to perform sleep analysis on those not wearing the wearable device.

[0012] In addition, conventional sleep analysis methods using wearable devices or non-contact sleep management research use the Heart Rate Variability (HRV) variance or EEG change values ​​between the asleep and awake states, but the difference between these is not large, meaning that they have the limitation of not being able to accurately time the awake state, which is the basis of all sleep treatments.

[0013] In particular, when using changes in brain waves to treat sleep disorders such as snoring, it is not possible to detect precursor symptoms of snoring from changes in brain waves at all, so it cannot be used to prevent snoring.In addition, since it detects changes in brain waves that occur after the patient snores, it has the limitation of only being used to diagnose snoring.

[0014] As a result, research has been progressing recently into a non-contact method of estimating sleep stages by monitoring breathing patterns and the degree of activation of the autonomic nervous system based on body movements during the night, and creating a sleep environment for the user based on the estimated sleep state.

[0015] In particular, many papers that have studied the relationship between sleep and the sleep environment, such as air quality, temperature, and humidity, have confirmed that the sleep environment, such as air quality, temperature, and humidity, has a decisive impact on sleep quality, which means that the sleep environment needs to be optimized in order to improve sleep quality. [Prior art documents] [Patent documents]

[0016] [Patent Document 1] Korean Patent Publication No. 2003-0032529 Summary of the Invention [Problem to be solved by the invention]

[0017] An object of the present invention is to provide a sleep analysis system and method that can conveniently and accurately analyze the sleep of various types of users in real time, without the need to purchase or wear a separate wearable device, and without being restricted by time or place.

[0018] Another object of the present invention is to provide a sleep analysis system and method that can in-depth analyze a user's sleep through artificial intelligence learning instead of conventional various biosignals based solely on the user's breathing, by simultaneously using a smart home appliance with a built-in microphone and a smartphone.

[0019] The present invention also provides various home appliances for providing an optimal sleeping environment related to various factors such as air quality, temperature and / or humidity of the sleeping environment based on sleep state information sensed in the user's sleeping environment.

[0020] The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0021] According to one embodiment of the present invention, there may be provided a method for creating an environment for an object, the method including: acquiring environmental sensing information; performing preprocessing on the acquired environmental sensing information; digitizing the preprocessed environmental sensing information; generating sleep state information based on the digitized environmental sensing information; and controlling an electronic device for creating the environment of the object based on the generated sleep state information.

[0022] Furthermore, according to an embodiment of the present invention, the step of controlling the electronic device may include generating information for controlling the environment of the object in real time based on the generated sleep state information.

[0023] In addition, in the method for creating an environment for an object according to an embodiment of the present invention, the environmental sensing information may include acoustic information.

[0024] In addition, in the method for creating an object environment according to an embodiment of the present invention, the acoustic information may include respiratory acoustic information.

[0025] In addition, in the method for creating an object environment according to an embodiment of the present invention, the sleep state information may include sleep stage information.

[0026] In addition, in the method for creating an environment of an object according to an embodiment of the present invention, the step of converting the environmental sensing information into data may further include a step of converting the preprocessed environmental sensing information into information including changes in frequency components of the preprocessed environmental sensing information over a time axis.

[0027] Here, in the method for creating an object environment according to an embodiment of the present invention, the information including the change in frequency components over time may be a spectrogram.

[0028] Meanwhile, according to one embodiment of the present invention, an electronic device for creating an environment for an object may be provided, including a sensor for acquiring environmental sensing information, means for performing preprocessing on the acquired environmental sensing information, means for digitizing the preprocessed environmental sensing information, means for generating sleep state information based on the digitized environmental sensing information, and means for controlling the electronic device so that the environment of the object is created based on the generated sleep state information.

[0029] Meanwhile, according to one embodiment of the present invention, an electronic device for creating an environment for an object may include a sensor for acquiring environmental sensing information, a means for performing preprocessing on the acquired environmental sensing information, a means for transmitting the digitized environmental sensing information to a server, a means for receiving, if the server generates sleep state information based on the transmitted environmental sensing information, the generated sleep state information, and a means for controlling the electronic device so that the environment of the object is created based on the received sleep state information.

[0030] Meanwhile, according to one embodiment of the present invention, an electronic device for creating an environment for an object may include a sensor for acquiring environmental sensing information, means for performing preprocessing on the acquired environmental sensing information, means for transmitting the preprocessed environmental sensing information to a server, means for the server to digitize the transmitted environmental sensing information and, if the server generates sleep state information based on the digitized environmental sensing information, means for receiving the generated sleep state information, and means for controlling the electronic device so that the environment of the object is created based on the received sleep state information.

[0031] Here, in the electronic device for creating an environment of an object according to an embodiment of the present invention, the means for controlling the electronic device may generate information for controlling the environment of the object in real time based on the generated sleep state information.

[0032] In addition, in the electronic device for creating an environment of an object according to an embodiment of the present invention, the environment sensing information may include acoustic information.

[0033] In addition, in the electronic device for creating an object environment according to an embodiment of the present invention, the acoustic information may include respiratory acoustic information.

[0034] In addition, in the electronic device for creating an object environment according to an embodiment of the present invention, the sleep state information may include sleep stage information.

[0035] In addition, in an electronic device for creating an environment of an object according to an embodiment of the present invention, the digitized environmental sensing information may be the pre-processed environmental sensing information converted into information including changes in frequency components of the pre-processed environmental sensing information over time.

[0036] Here, in the electronic device for creating an object environment according to an embodiment of the present invention, the information including the change in frequency components over time may be a spectrogram.

[0037] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, an electronic device for controlling a home appliance for creating an environment of an object may be provided, including a sensor for acquiring environmental sensing information, means for performing preprocessing on the acquired environmental sensing information, means for digitizing the preprocessed environmental sensing information, means for generating sleep state information based on the digitized environmental sensing information, and means for controlling the home appliance so that the environment of the object is created based on the generated sleep state information.

[0038] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, an electronic device for controlling a home appliance for creating an environment of an object may be provided, including a sensor for acquiring environmental sensing information, means for performing preprocessing on the acquired environmental sensing information, means for digitizing the preprocessed environmental sensing information, means for transmitting the digitized environmental sensing information to a server, means for receiving, if the server generates sleep state information based on the transmitted environmental sensing information, the generated sleep state information, and means for controlling the home appliance so that the environment of the object is created based on the received sleep state information.

[0039] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, an electronic device for controlling a home appliance for creating an environment of an object may be provided, including a sensor for acquiring environmental sensing information, means for performing preprocessing on the acquired environmental sensing information, means for transmitting the preprocessed environmental sensing information to a server, means for the server to digitize the transmitted environmental sensing information and, if the server generates sleep state information based on the digitized environmental sensing information, means for receiving the generated sleep state information, and means for controlling the home appliance so that the environment of the object is created based on the received sleep state information.

[0040] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, when another electronic device acquires environmental sensing information, digitizes the acquired environmental sensing information, and generates sleep state information based on the digitized environmental sensing information, the electronic device for controlling a home appliance for creating an environment of an object may be provided, the electronic device including: means for receiving the sleep state information generated in the other electronic device; and means for controlling the home appliance so that the environment of the object is created based on the received sleep state information.

[0041] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, when another electronic device acquires environmental sensing information, digitizes the acquired environmental sensing information, transmits the digitized environmental sensing information to a server, and the server generates sleep state information based on the transmitted environmental sensing information, the electronic device for controlling a home appliance for creating an environment of an object may include means for receiving the generated sleep state information from the server, and means for controlling the home appliance so that the environment of the object is created based on the received sleep state information.

[0042] Meanwhile, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, when another electronic device acquires environmental sensing information, transmits the acquired environmental sensing information to a server, and the server digitizes the transmitted environmental sensing information and generates sleeping state information based on the digitized environmental sensing information, the electronic device for controlling a home appliance for creating an environment of an object may include means for receiving the generated sleeping state information from the server, and means for controlling the home appliance so that the environment of the object is created based on the received sleeping state information.

[0043] Here, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, the means for controlling the home appliance may generate information for controlling the environment of the object in real time based on the generated sleep state information.

[0044] In addition, in the electronic device for controlling a home appliance for creating an object environment according to an embodiment of the present invention, the environment sensing information may include acoustic information.

[0045] In addition, in the electronic device for controlling a home appliance for creating an object environment according to an embodiment of the present invention, the acoustic information may include respiratory acoustic information.

[0046] In addition, in the electronic device for controlling a home appliance for creating an object-like environment according to an embodiment of the present invention, the sleep state information may include sleep stage information.

[0047] In addition, in an electronic device for controlling a home appliance for creating an environment of an object according to an embodiment of the present invention, the digitized environmental sensing information may be the pre-processed environmental sensing information converted into information including a change in frequency components of the pre-processed environmental sensing information over time.

[0048] Here, in the electronic device for controlling a home appliance for creating an object environment according to an embodiment of the present invention, the information including the change in frequency components over time may be a spectrogram.

[0049] The present invention relates to a method for controlling an environment creation device, the method including: an acquisition step of acquiring environmental sensing information; a preprocessing step of performing preprocessing on the acquired environmental sensing information; a generation step of generating sleep state information based on the preprocessed environmental sensing information; and a control step of controlling the environment creation device based on the generated sleep state information.

[0050] The present invention relates to a method for controlling an environment creating device, wherein the controlling step controls the environment creating device in real time based on the generated sleep state information.

[0051] The present invention relates to a method for controlling an environment-creating device, wherein the generating step further includes converting the environmental sensing information into information including a change in frequency components of the environmental sensing information along a time axis.

[0052] The present invention relates to a method for controlling an environment creation device, wherein the control step further includes the steps of generating first environment creation information based on the generated sleeping state information, causing the environment creation device to create an environment based on the first environment creation information, generating second environment creation information based on the user's sleeping state information generated after the environment creation device starts creating an environment based on the first environment creation information, and causing the environment creation device to create an environment based on the generated second environment creation information.

[0053] The method for controlling an environment creation device further includes generating the first environment creation information based on the sleep state information generated during a time period corresponding to one or more epochs, and generating the second environment creation information based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating an environment based on the first environment creation information.

[0054] The present invention relates to an electronic device for controlling an environment creation device, the electronic device including: a sensor for acquiring environment sensing information; and a control unit for performing an operation of preprocessing the acquired environment sensing information, an operation of generating sleep state information based on the preprocessed environment sensing information, and an operation of controlling the environment creation device based on the generated sleep state information.

[0055] The present invention relates to an electronic device for controlling an environment creating device, wherein the control unit controls the environment creating device in real time based on the generated sleep state information.

[0056] The present invention relates to an electronic device for controlling an environment creation device, wherein the control unit converts the environmental sensing information into information including a change in frequency component of the environmental sensing information along a time axis.

[0057] The present invention relates to an electronic device for controlling an environment creation device, wherein the control unit performs an operation of controlling the environment creation device based on the generated first environment creation information when the control unit generates first environment creation information based on the generated sleep state information, and performs an operation of controlling the environment creation device based on the generated second environment creation information when the control unit generates second environment creation information based on the generated user sleep state information after the environment creation device starts creating an environment based on the first environment creation information.

[0058] The present invention relates to an electronic device for controlling an environment creation device, wherein the generated first environment creation information is generated based on the sleep state information generated during a time corresponding to one or more epochs, and the generated second environment creation information is generated based on the sleep state information generated during a time corresponding to one or more epochs after the environment creation device starts creating an environment based on the first environment creation information.

[0059] The present invention relates to an environment creation system including an electronic device including a sensor for acquiring environmental sensing information, a control unit, and a communication unit for transmitting and receiving information via a network; a server for generating sleep state information based on the environmental sensing information; and an environment creation device, wherein the control unit performs an operation of preprocessing the acquired environmental sensing information and an operation of transmitting the preprocessed environmental sensing information to the server via the communication unit, the server performs an operation of generating sleep state information based on the preprocessed environmental sensing information received from the electronic device, and the control unit performs an operation of receiving the generated sleep state information from the server via the communication unit and an operation of controlling the environment creation device based on the received sleep state information.

[0060] The present invention relates to an environment creation system, wherein the control unit controls the environment creation device in real time based on the received sleep state information.

[0061] The present invention relates to an environment creation system, wherein the control unit controls the environment creation device in real time based on the received sleep state information.

[0062] The present invention relates to an environment creation system, in which the control unit receives information from the server via the communication unit to convert the environmental sensing information into information including changes in frequency components of the environmental sensing information over time.

[0063] The present invention relates to an environment creation system in which the control unit performs an operation of controlling the environment creation device based on first environment creation information generated based on the sleep state information received via the communication unit, and after the environment creation device starts creating an environment based on the first environment creation information, generates second environment creation information based on the user's sleep state information received via the communication unit, and performs an operation of controlling the environment creation device based on the generated second environment creation information.

[0064] The present invention relates to an environment creation system, wherein the generated first environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs, and the generated second environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating an environment based on the first environment creation information.

[0065] The present invention relates to an environment creation system including an electronic device including a sensor for acquiring environment sensing information, a control unit, and a communication unit for transmitting and receiving information via a network; a server that performs an operation of generating sleep state information based on the environment sensing information and an operation of generating environment creation information based on the sleep state information; and an environment creation device controlled based on the environment creation information, wherein the control unit performs an operation of performing preprocessing on the acquired environment sensing information and an operation of transmitting the preprocessed environment sensing information to the server via the communication unit, and the server performs an operation of generating sleep state information based on the preprocessed environment sensing information received from the electronic device, an operation of generating environment creation information for controlling the environment creation device based on the generated sleep state information, and an operation of transmitting the generated environment creation information from the environment creation device.

[0066] The present invention relates to an environment creation system, wherein the server includes a first server and a second server, the first server performs an operation of generating sleep state information based on the pre-processed environment sensing information received from the electronic device, and the second server performs an operation of generating environment creation information for controlling the environment creation device based on the generated sleep state information.

[0067] The present invention relates to an environment creation system, in which the server controls the environment creation device in real time based on the generated sleep state information.

[0068] The present invention relates to an environment creation system, in which the server converts the environmental sensing information received from the electronic device into information including changes in frequency components of the environmental sensing information along a time axis.

[0069] The present invention relates to an environment creation system, wherein the server includes a first server and a second server, the first server performs an operation of generating sleep state information based on the pre-processed environment sensing information received from the electronic device, and the second server performs an operation of generating environment creation information for controlling the environment creation device based on the generated sleep state information.

[0070] The present invention relates to an environment creation system, wherein the generated first environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs, and the generated second environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating an environment based on the first environment creation information.

[0071] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, the method including: an acquisition step of acquiring environmental sensing information; a preprocessing step of performing preprocessing on the acquired environmental sensing information; a generation step of generating sleep state information based on the preprocessed environmental sensing information; and a control step of controlling the electronic device that provides the predetermined scent based on the generated sleep state information.

[0072] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, by controlling the electronic device in real time based on the generated sleep state information.

[0073] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the generating step further includes a step of converting the environmental sensing information into information including changes in the frequency components of the environmental sensing information over time.

[0074] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the information including the change in frequency components over time is a spectrogram.

[0075] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the environmental sensing information includes sleep sound information.

[0076] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the sleep state information includes at least one of sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information.

[0077] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the control step further includes a step of causing the electronic device to provide a first scent, a step of generating second scent provision information based on the generated sleep state information of the user after the electronic device that provides the predetermined scent has started to provide the first scent, and a step of causing the electronic device that provides the predetermined scent to provide the second scent based on the generated second scent provision information.

[0078] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the step of generating the second scent provision information further includes a step of generating the second scent provision information based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information generated after the electronic device starts providing the first scent.

[0079] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the control step further includes the steps of generating first scent provision information based on the generated sleep state information, causing the electronic device that provides the predetermined scent to provide the first scent for a first period of time based on the generated first scent provision information, and generating second scent provision information based on the generated sleep state information of the user after the electronic device that provides the predetermined scent starts to provide the first scent, and causing the electronic device that provides the predetermined scent to provide the second scent for a second period of time based on the generated second scent provision information.

[0080] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first time and the second time are multiples of a predetermined minimum time unit.

[0081] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the step of generating the second scent provision information further includes a step of generating the second scent provision information based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information generated after the electronic device that provides the predetermined scent starts providing the first scent.

[0082] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first scent provision information and the second scent provision information include at least one of scent attribute information and scent provision adjustment information.

[0083] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein in the control step, the step of generating the first scent provision information further includes a step of generating the first scent provision information based on the sleep state information generated during a time corresponding to one or more epochs, and the step of generating the second scent provision information generates the second scent provision information based on the sleep state information generated during a time corresponding to one or more epochs after starting to provide the first scent.

[0084] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the epoch is set as data corresponding to a 30-second unit.

[0085] The present invention relates to a method for controlling an electronic device that provides a predetermined scent in response to predetermined scent-providing information, wherein at least one of the first scent or the second scent includes no scent, and at least one of the first scent-providing information or the second scent-providing information includes information that does not provide a scent.

[0086] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, the electronic device including a sensor that acquires environmental sensing information, a means for performing preprocessing on the acquired environmental sensing information, a means for generating sleep state information based on the preprocessed environmental sensing information, and a means for providing a predetermined scent in response to predetermined scent provision information.

[0087] The present invention relates to an electronic device that provides a predetermined scent in real time based on the generated sleep state information and provides a predetermined scent in response to predetermined scent provision information.

[0088] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and the means for generating sleep state information based on the preprocessed environmental sensing information converts the environmental sensing information into information including changes in the frequency components of the environmental sensing information over time.

[0089] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the information including the changes in frequency components over time is a spectrogram.

[0090] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, the environmental sensing information of which includes sleep sound information.

[0091] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the sleep state information includes at least one of sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information.

[0092] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the means for providing a predetermined scent based on the generated sleep state information provides a first scent to the user and, after starting to provide the first scent, provides a second scent based on the generated sleep state information of the user.

[0093] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and when providing the second scent, provides the second scent based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information generated after the provision of the first scent begins.

[0094] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the means for providing a predetermined scent based on the generated sleep state information provides a first scent for a first period of time based on the generated sleep state information, and after starting to provide the first scent, provides a second scent for a second period of time based on the generated sleep state information of the user.

[0095] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first time period and the second time period are multiples of a predetermined minimum time unit.

[0096] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and when providing the second scent, provides the second scent based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information that are generated after starting to provide the first scent to the user.

[0097] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first scent and the second scent are at least one of a scent based on scent attribute information and a scent based on scent provision adjustment information.

[0098] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein when providing the first scent, the first scent is provided based on the sleep state information generated over a period of time corresponding to one or more epochs, and the second scent is provided based on the sleep state information generated over a period of time corresponding to one or more epochs after starting to provide the first scent.

[0099] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the epoch is set as data corresponding to a unit of 30 seconds.

[0100] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein at least one of the first scent or the second scent includes unscented scent.

[0101] The present invention relates to an electronic device that provides a predetermined fragrance in response to predetermined fragrance provision information, the electronic device including: a sensor that acquires environmental sensing information; a means for performing preprocessing on the acquired environmental sensing information; a means for transmitting the preprocessed environmental sensing information to a server; a means for receiving sleep state information generated based on the transmitted environmental sensing information from the server; and a means for providing the predetermined fragrance based on the received sleep state information.

[0102] The present invention relates to an electronic device in which the means for providing the predetermined scent provides the predetermined scent in real time based on the received sleep state information, and provides the predetermined scent in response to predetermined scent provision information.

[0103] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and when the server converts the transmitted environmental sensing information into information including changes in the frequency components of the environmental sensing information over time, the means for receiving sleep state information generated based on the transmitted environmental sensing information from the server receives the converted information.

[0104] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the information including the changes in frequency components over time is a spectrogram.

[0105] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, the environmental sensing information of which includes sleep sound information.

[0106] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the sleep state information includes at least one of sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information.

[0107] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the means for providing a predetermined scent based on the received sleep state information provides a first scent and then provides a second scent based on the user's sleep state information received after the provision of the first scent begins.

[0108] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and when providing the second scent, provides the second scent based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information received after starting to provide the first scent.

[0109] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the means for providing a predetermined scent based on the received sleep state information provides a first scent for a first time period based on the received sleep state information, and provides a second scent for a second time period based on the received sleep state information of the user after starting to provide the first scent.

[0110] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first time period and the second time period are multiples of a predetermined minimum time unit.

[0111] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, and when providing the second scent, provides the second scent based on at least one of the user's sleep stage information, sleep stage probability information, sleep event information, and sleep event probability information received after starting to provide the first scent to the user.

[0112] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the first scent provision information and the second scent provision information include at least one of scent attribute information and scent provision adjustment information.

[0113] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein when providing the first scent, the first scent is provided based on the sleep state information received over a period of time corresponding to one or more epochs, and the second scent is provided based on the sleep state information received over a period of time corresponding to one or more epochs after starting to provide the first scent.

[0114] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein the epoch is set as data corresponding to a unit of 30 seconds.

[0115] The present invention relates to an electronic device that provides a predetermined scent in response to predetermined scent provision information, wherein at least one of the first scent or the second scent includes no scent, and the scent provision information includes information that no scent will be provided.

[0116] The present invention relates to an electronic device that controls a home appliance that provides a predetermined scent in response to predetermined scent provision information, the electronic device including a sensor that acquires environmental sensing information, a means for performing preprocessing on the acquired environmental sensing information, a means for generating sleep state information based on the preprocessed environmental sensing information, and a means for controlling the home appliance that provides the predetermined scent based on the generated sleep state information.

[0117] The present invention relates to an electronic device that controls a home appliance that provides a predetermined fragrance in response to predetermined fragrance provision information, the electronic device including: a sensor that acquires environmental sensing information; a means for performing preprocessing on the acquired environmental sensing information; a means for transmitting the preprocessed environmental sensing information to a server; a means for receiving sleep state information generated based on the transmitted environmental sensing information from the server; and a means for controlling the home appliance that provides the predetermined fragrance based on the received sleep state information.

[0118] The present invention relates to an electronic device that controls a home appliance that provides a predetermined fragrance in response to predetermined fragrance provision information, and when another electronic device acquires environmental sensing information, performs preprocessing on the acquired environmental sensing information, and generates sleep state information based on the preprocessed environmental sensing information, the electronic device includes: a receiving unit that receives the generated sleep state information; and means for controlling a home appliance that provides a predetermined fragrance in response to predetermined fragrance provision information.

[0119] The present invention relates to an electronic device that controls a home appliance that provides a predetermined fragrance in response to predetermined fragrance provision information, the electronic device including: a receiving unit that receives sleep state information generated based on the environmental sensing information transmitted from a server when another electronic device acquires environmental sensing information, performs preprocessing on the acquired environmental sensing information, and transmits the preprocessed environmental sensing information to the server; and means for controlling a home appliance that provides a predetermined fragrance in response to predetermined fragrance provision information.

[0120] In order to solve the above problem, there is provided a light control device for creating a sleep environment for a user, the light control device including: a sensing unit for acquiring acoustic information of a user; a transmitting unit for transmitting the acquired acoustic information of the user to a server; a receiving unit for receiving the generated sleep state information from the server when the server generates sleep state information based on the transmitted acoustic information of the user; a control unit for generating light control information based on the received sleep state information; and a light source unit for emitting light controlled based on the generated light control information.

[0121] To achieve this object, the present invention provides a light control device for creating a sleep environment for a user, wherein the receiving unit receives the generated average sleep onset delay time information of the user when the server generates average sleep onset delay time information of the user based on the transmitted acoustic information of the user, and the control unit generates light control information based on the average sleep onset delay time information of the user.

[0122] To achieve this object, a light control device for creating a user's sleeping environment is provided, in which the control unit generates light control information based on set time information.

[0123] To solve this problem, a light control device is provided that creates a sleeping environment for a user, in which, when the server determines that the user has fallen asleep and generates sleep state information, the receiving unit receives the sleep state information, and the control unit generates light control information based on the sleep state information.

[0124] To solve the problem, the control unit generates light control information to reduce the amount of light emitted from the light source unit or to control it to 0 lux when the average sleep onset delay time is reached based on light control information generated based on the average sleep onset time information of the user, thereby creating a sleep environment for the user.

[0125] To solve this problem, the control unit generates light control information to control the light emitted from the light source unit to be below a critical value when the user falls asleep earlier than the user's average sleep onset delay time, thereby providing a light control device that creates a sleep environment for the user.

[0126] To solve the above problem, the control unit generates light control information to maintain the light emitted from the light source unit at or below a first critical value when the user falls asleep later than the user's average sleep onset delay time, and to control the light emitted from the light source unit at or below a second critical value when the user falls asleep.

[0127] To solve the problem, the server generates sleep state information based on the transmitted acoustic information of the user, and generates biorhythm information of the user based on the generated sleep state information, and the receiving unit receives the generated biorhythm information from the server, and provides a light adjusting device that creates a sleeping environment for the user.

[0128] To solve this problem, the control unit provides a light control device that creates a user's sleeping environment by generating light control information so that biorhythm information received by the receiving unit from the server matches predetermined biotime information.

[0129] To achieve this object, the control unit generates light control information to increase the amount of light at a predetermined rate from a predetermined brightness to a brightness set by a user, starting from a critical time before the alarm time, when the control unit receives predetermined alarm time information, to provide a light control device for creating a user's sleeping environment.

[0130] To achieve this object, the present invention provides a light control device for creating a user's sleep environment, wherein the receiving unit receives predetermined alarm time information, and when the user's REM sleep is detected between the alarm time and a critical time, the control unit generates light control information such that the amount of light increases at a predetermined rate from a predetermined brightness to a brightness set by the user at a point where the predetermined time has elapsed after the user's REM sleep is detected.

[0131] To solve this problem, the present invention provides a light control device that creates a user's sleep environment, in which the receiver receives predetermined alarm time information, and if the user's REM sleep is not detected between the alarm time information and a critical time, the controller generates light control information to increase the amount of light at a predetermined gradient from a predetermined brightness set to a brightness set by the user before the critical time of the alarm time.

[0132] To solve this problem, the control unit provides a light control device that creates a sleep environment for a user, and generates light control information that causes the light source unit to emit light above a critical value when the user's wakefulness is not detected for a critical time or more after the alarm time.

[0133] To achieve this object, a light adjusting device for creating a user's sleep environment is provided, in which, when the receiving unit receives predetermined alarm time information, the control unit generates light adjustment information to increase the amount of light at a predetermined brightness set by the user at a predetermined gradient to a brightness set by the user from the alarm time until just before a critical time based on the generated biorhythm information of the user.

[0134] In order to solve the above problem, there is provided a light control device for creating a sleep environment for a user, the light control device including: a sensing unit for acquiring acoustic information of a user; a transmitting unit for transmitting the acquired acoustic information of the user to a server; a receiving unit for generating light control information based on the generated sleep state information when the server generates sleep state information based on the transmitted acoustic information of the user, and receiving the generated light control information; and a light source unit for emitting light controlled based on the generated light control information.

[0135] In order to solve this problem, there is provided a light control device for creating a sleep environment for a user, the light control device including: a sensing unit that acquires acoustic information of the user; a control unit that generates sleep state information based on the acquired acoustic information of the user and generates light control information based on the generated sleep state information; and a light source unit that emits light adjusted based on the generated light control information.

[0136] In order to solve this problem, there is provided a light control device for creating a sleep environment for a user, the light control device including: a sensing unit for acquiring acoustic information of a user; a transmitting unit for transmitting the acquired acoustic information of the user to a first server; a receiving unit for generating light control information based on the acquired acoustic information when a second server receives, from the first server, sleep state information generated in the first server based on the transmitted acoustic information, and receiving the light control information; and a light source unit for emitting light adjusted based on the received light control information.

[0137] In order to solve this problem, there is provided an apparatus for adjusting light of a light source device having a light source unit, the apparatus including: a sensing unit that acquires acoustic information; a memory unit in which an application can be recorded; and a processor unit in which the application can be executed, wherein the application generates sleep state information based on the acoustic information acquired from the sensing unit, generates light adjustment information based on the generated sleep state information, and transmits the light adjustment information to the light source unit.

[0138] In order to solve this problem, there is provided an apparatus for adjusting light of a light source device having a light source unit, the apparatus including: a sensing unit that acquires acoustic information; a memory unit in which a first application and a second application can be recorded; and a processor unit in which the first application and the second application can be executed, wherein the first application generates sleep state information based on the acoustic information acquired from the sensing unit, and the second application generates light adjustment information based on the sleep state information generated in the first application, and the apparatus for adjusting light of a light source device having a light source unit is configured to transmit the light adjustment information to the light source unit.

[0139] To achieve this object, there is provided a recording medium having a program recorded thereon, the recording medium performing the following steps: acquiring sleep sound information by preprocessing sound information acquired from a sensing unit; transmitting the acquired sleep sound information to a server via a transmitting unit; receiving the generated sleep sound information from the server via a receiving unit when the server generates sleep state information based on the transmitted sleep sound information of the user; generating light control information based on the received sleep state information; and emitting light controlled based on the generated light control information via a light source.

[0140] To solve the problem, in the receiving step, the server generates average sleep onset delay time information of the user based on the transmitted acoustic information of the user, and in the control step, a recording medium having a program recorded thereon is provided for receiving the generated average sleep onset delay time information of the user and generating light adjustment information based on the average sleep onset delay time information of the user.

[0141] To achieve this object, the control step provides a recording medium having recorded thereon a program for generating light adjustment information based on set time information.

[0142] To solve this problem, when the server determines that the user has fallen asleep and generates sleep state information, the receiving step receives the sleep state information, and the control step provides a recording medium on which a program is recorded that generates light adjustment information based on the sleep state information.

[0143] To achieve this object, the control step provides a recording medium having a program recorded thereon for generating light control information for controlling the amount of light emitted from the light source unit to be reduced or controlled to a predetermined brightness when the average sleep onset delay time is reached, based on light control information generated based on the average sleep onset delay time information of the user.

[0144] To solve this problem, the control step provides a recording medium having recorded thereon a program for generating light control information for controlling light emitted from the light source unit to be below a critical value when the user is detected to be asleep before reaching the user's average sleep onset delay time.

[0145] In order to solve this problem, a hot water mat that creates a sleeping environment for a user includes a sensing unit that acquires acoustic information of the user, a transmitting unit that transmits the acquired acoustic information of the user to a server, a receiving unit that receives the generated sleeping state information from the server when the server generates sleeping state information based on the transmitted acoustic information of the user, a control unit that generates temperature control information based on the received sleeping state information, and a heat control means that adjusts heat to a temperature adjusted based on the generated temperature control information.

[0146] To solve this problem, when the receiving unit receives sleep state information from the server indicating that the user is about to fall asleep, the control unit generates temperature adjustment information based on the user's set temperature, thereby providing a hot water mat that creates a sleeping environment for the user.

[0147] To solve this problem, a hot water mat is provided that creates a sleeping environment for a user by generating temperature adjustment information set to a predetermined temperature below or above the user-set temperature based on the received user sleep state information or the received user-set sleep latency when the receiving unit receives sleep state information from the server indicating that the user is in sleep latency, or when the receiving unit receives a user-set sleep latency.

[0148] In order to solve this problem, a hot water mat is provided that further includes a user body temperature measurement unit, and when the receiving unit receives sleep state information from the server that the user is in a sleep latency period and receives information that the user's body temperature has dropped from the user body temperature measurement unit, the control unit generates temperature adjustment information that is set to a temperature higher than a predetermined temperature from the user's set temperature, thereby creating a sleeping environment for the user.

[0149] To solve this problem, the server generates sleep state information based on the transmitted acoustic information of the user, and generates biorhythm information of the user based on the generated sleep state information, and the receiving unit receives the generated biorhythm information from the server, thereby providing a hot water mat that creates a sleeping environment for the user.

[0150] To solve this problem, a hot water mat is provided that creates a sleeping environment for a user, in which when the receiving unit receives sleep state information from the server indicating that the user is in the sleep latency period, the control unit generates set temperature adjustment information that changes the user's set temperature to a predetermined temperature based on the biorhythm information received by the receiving unit from the server.

[0151] To solve this problem, a hot water mat is provided in which the receiving unit receives sleep state information from the server indicating that the user is in the first deep sleep, and the control unit maintains the set temperature control information until a predetermined time has elapsed, creating a sleeping environment for the user.

[0152] To solve this problem, a hot water mat is provided that creates a sleeping environment for a user in which the control unit maintains the set temperature control information until the receiving unit receives sleep state information indicating the first REM sleep from the server.

[0153] To solve this problem, a hot water mat is provided that creates a sleeping environment for the user, in which when the receiving unit receives sleep state information from the server indicating that the user is in REM sleep, the control unit generates temperature adjustment information that applies a predetermined temperature change during the REM sleep based on the sleep state information received by the receiving unit from the server.

[0154] To solve this problem, a hot water mat that creates a sleeping environment for a user is provided, in which when the receiving unit receives from the server sleep state information indicating the user's wakefulness, REM state, or light sleep state, or sleep state information indicating a point between the REM state and the light sleep state, the control unit generates temperature control information that increases the temperature by a predetermined temperature between the detection point of the wakefulness, REM state, or light sleep state and the user's desired wakefulness point, or between a point between the detection point of the REM state and the light sleep state and the user's desired wakefulness point.

[0155] To solve this problem, the temperature control information for increasing the temperature by the predetermined temperature is provided such that when the receiving unit receives from the server sleep state information indicating that the user is in a light sleep state at the user's desired wake-up time, the control unit generates first temperature control information, and when the receiving unit receives sleep state information indicating that the user is in a deep sleep state, the control unit generates second temperature control information.

[0156] To solve this problem, a hot water mat is provided that creates a sleeping environment for a user, further including a user body temperature measurement unit, wherein the receiving unit receives sleep state information indicating an awake state from the server, and when the receiving unit receives user body temperature information indicating that the user's body temperature is higher than a predetermined temperature from the user body temperature measurement unit, the control unit generates temperature adjustment information to lower the temperature by the predetermined temperature.

[0157] To solve this problem, there is provided a hot water mat that creates a sleeping environment for a user, including a sensing unit that acquires acoustic information of a user, a transmitting unit that transmits the acquired acoustic information of the user to a server, a receiving unit that generates temperature control information based on the generated sleep state information when the server generates sleep state information based on the transmitted acoustic information of the user, and receives the generated temperature control information, and a heat control means that adjusts heat to a temperature adjusted based on the generated temperature control information.

[0158] In order to solve this problem, a hot water mat that creates a sleeping environment for a user is provided, which includes a sensing unit that acquires acoustic information of the user, a control unit that generates sleep state information based on the acquired acoustic information of the user and generates temperature control information based on the generated sleep state information, and a heat control means that adjusts heat to a temperature adjusted based on the generated temperature control information.

[0159] In order to solve this problem, a hot water mat that creates a sleeping environment for a user is provided, which includes a sensing unit that acquires acoustic information of the user, a transmitting unit that transmits the acquired acoustic information of the user to a first server, a receiving unit that generates temperature adjustment information based on the received sleep state information when a second server receives from the first server sleep state information generated based on the transmitted acoustic information, and a heat adjustment means that adjusts heat to the adjusted temperature based on the received temperature adjustment information.

[0160] In order to solve this problem, we provide an apparatus for adjusting the heat adjustment means of a hot water mat having a heat adjustment means, which includes a sensing unit that acquires acoustic information of a user, a memory unit in which an application can be recorded, and a processor unit in which the application can be executed, wherein the processor unit generates sleep state information based on the acoustic information acquired from the sensing unit via the application, generates temperature adjustment information based on the generated sleep state information, and transmits the generated temperature adjustment information to the hot water mat.

[0161] In order to solve this problem, we provide an apparatus for adjusting the heat adjustment means of a hot water mat having a heat adjustment means, which includes a sensing unit that acquires acoustic information, a memory unit in which a first application and a second application can be recorded, and a processor unit in which the first application and the second application can be executed, and in which the processor unit generates sleep state information based on the acoustic information acquired from the sensing unit via the first application, generates temperature adjustment information based on the sleep state information generated by the first application via the second application, and transmits the temperature adjustment information to the hot water mat.

[0162] To solve this problem, a recording medium having a program recorded thereon is provided, the recording medium having a program for performing the following steps: a step of preprocessing acoustic information acquired from a sensing unit to acquire sleep acoustic information; a step of transmitting the acquired sleep acoustic information to a server via a transmitting unit; a receiving step of receiving the generated sleep state information from the server via a receiving unit when the server generates sleep state information based on the transmitted sleep acoustic information of the user; a control step of generating temperature control information based on the received sleep state information; and a heat control step of adjusting heat to a controlled temperature based on the generated temperature control information.

[0163] In order to solve this problem, we provide an apparatus for adjusting the heat adjustment means of a hot water mat having a heat adjustment means, which includes a memory unit in which an application can be recorded and a processor unit in which the application can be executed, wherein the processor unit acquires acoustic information of a user via the application, the processor unit transmits the acquired acoustic information of the user via the application to a first server, the first server generates sleep state information based on the transmitted acoustic information of the user, the processor unit receives the sleep state information via the application and transmits the received sleep state information to a second server, the second server generates temperature adjustment information based on the received sleep state information, and the processor unit receives the generated temperature adjustment information via the application.

[0164] In order to solve this problem, we provide an apparatus for adjusting the heat adjustment means of a hot water mat having a heat adjustment means, which includes a sensing unit that acquires acoustic information of a user, a memory unit in which an application can be recorded, and a processor unit in which the application can be executed, wherein the processor unit transmits the acoustic information acquired from the sensing unit to a first server via the application, receives sleep state information from the first server acquired based on the acoustic information acquired from the sensing unit, transmits the acquired sleep state information to a second server, receives temperature adjustment information from the second server acquired based on the received sleep state information, and transmits the received temperature adjustment information to the hot water mat.

[0165] The cosmetic recommendation method according to the present invention includes the steps of calculating a sleep index of a user, generating cosmetic information corresponding to the calculated sleep index, and displaying the generated cosmetic information.

[0166] The step of generating the cosmetic information may generate the recommended cosmetic information based on a lookup table in which cosmetic information corresponding to sleep indices is recorded.

[0167] In addition, the step of generating the cosmetic information may include a step of generating a cosmetic recommendation model by learning to generate cosmetic information corresponding to a plurality of sleep index information, and a step of inputting the sleep index information into the cosmetic recommendation model and outputting the recommended cosmetic information as a result value.

[0168] Meanwhile, a cosmetics verification method according to the present invention includes the steps of receiving environmental sensing information from a user terminal of a user who has used a predetermined cosmetic product, acquiring at least one of sleep state information and sleep stage information of the user based on the environmental sensing information, generating a verification index for the predetermined cosmetic product using at least one of the sleep state information and sleep stage information, and verifying the effect of the predetermined cosmetic product on sleep quality based on the verification index.

[0169] The step of acquiring the sleep state information may include a step of generating a trained inference model using environmental sensing information as input, and a step of inputting the environmental sensing information received from the user terminal into the inference model and extracting the sleep state information as a result value.

[0170] A method for recommending sleep-related products according to one embodiment of the present invention includes the steps of acquiring sleep information of a user from one or more sensor devices, calculating a sleep index of the user based on the acquired sleep information of the user, and providing the generated sleep-related product information.

[0171] Here, the sleep information of the user acquired from the one or more sensor devices includes sleep acoustic information of the user.

[0172] In addition, the step of generating the sleep product recommendation information according to an embodiment of the present invention may further include the step of generating a lookup table in which sleep product information corresponding to the calculated sleep indicators is recorded.

[0173] Alternatively, the method for recommending sleep-related products according to an embodiment of the present invention may further include a step of receiving a verification indicator for sleep-related products, and in the step of generating sleep-related product recommendation information, the sleep-related product recommendation information may be generated based on the received verification indicator for sleep-related products and the calculated sleep indicator.

[0174] Alternatively, the method for recommending sleep-related products according to an embodiment of the present invention may further include a step of receiving an input action from a user, the input action including at least one of an action of swiping a sleep-related product, an action of inputting a keyword, or an action of selecting a keyword, and in the step of generating the sleep-related product recommendation information, the sleep-related product recommendation information may be generated based on the calculated sleep indexes and the input action.

[0175] Alternatively, the method for recommending sleep-related products according to an embodiment of the present invention may further include receiving statistical information based on user attributes, where the user attributes include at least one of the user's gender, age group, occupational group, residential area, race, whether or not the user has a pet, environmental factors, or non-environmental factors, and in generating the sleep-related product recommendation information, the sleep-related product recommendation information may be generated based on the calculated sleep indexes and the received statistical information based on the user's attributes.

[0176] Alternatively, the step of generating sleep-related product recommendation information according to an embodiment of the present invention may further include a step of generating a sleep-related product recommendation model by learning to generate sleep-related product recommendation information based on a number of sleep indices, and the calculated sleep indices may be used as inputs of the sleep-related product recommendation model, and the sleep-related product recommendation information may be output as a result value.

[0177] In addition, according to an embodiment of the present invention, the step of calculating the user's sleep index based on the acquired user's sleep information may further include a step of converting the frequency components of the acoustic information included in the user's sleep information into information including changes in the frequency components over time, and acquiring at least one of the user's sleep state information and sleep stage information based on the converted information.

[0178] Here, in the step of calculating the user's sleep index based on the acquired user's sleep information, the converted information may be a visualization of changes in the frequency components of the acoustic information over time.

[0179] Alternatively, in the step of calculating the sleep index of the user based on the acquired sleep information of the user, the converted information may be a spectrogram.

[0180] Meanwhile, a method for verifying sleep-related products according to one embodiment of the present invention includes the steps of acquiring a user's sleep information from one or more sensor devices, obtaining at least one of the user's sleep intention information, sleep state information, and sleep stage information based on the acquired user's sleep information, and verifying the effect of the sleep-related products on sleep based on at least one of the user's sleep intention information, sleep state information, and sleep stage information.

[0181] Here, the sleep information of the user acquired from the one or more sensor devices includes sleep acoustic information of the user.

[0182] Here, the step of verifying the effect of the sleep-related product on sleep according to one embodiment of the present invention may further include the step of generating a verification index for the sleep-related product, and the verification index may be generated in the form of a look-up table or a numerical index related to sleep.

[0183] In addition, the numerical index related to sleep according to one embodiment of the present invention is characterized in that it is calculated based on a lookup table or a numerical representation of sleep analysis results, and the numerical index related to sleep may be calculated based on at least one of the sleep onset delay time, sleep onset time, wake-up time, total sleep time, and sleep stage specific sleep time of a user using the sleep product.

[0184] Here, according to one embodiment of the present invention, the sleep analysis result is expressed numerically as a total sleep score, with 100 being the maximum score, which is calculated using a pre-set formula. The pre-set formula may be a formula that calculates the total score by substituting the score corresponding to each sleep stage of the user who used the sleep product based on the score corresponding to each sleep stage information.

[0185] Meanwhile, according to one embodiment of the present invention, the step of generating the verification indicator may further include a step of receiving a user's subjective judgment indicator (the user's subjective judgment indicator is calculated based on at least one of a string value, a numerical value, or an input action of the user), and the verification indicator of the sleep-related product generated in the step of generating the verification indicator may be generated by additionally taking into consideration the received user's subjective judgment indicator.

[0186] In addition, in a method for verifying sleep-related products according to one embodiment of the present invention, the step of acquiring at least one of the user's sleep intention information, sleep state information, and sleep stage information based on the acquired user's sleep information may be characterized by converting the acoustic information included in the user's sleep information into information including changes in frequency components over time, and acquiring at least one of the user's sleep state information and sleep stage information based on the converted information.

[0187] Here, in the step of acquiring at least one of the user's sleep intention information, sleep state information, and sleep stage information based on the acquired user's sleep information, the converted information may be a visualized representation of changes in frequency components of the acoustic information over a time axis.

[0188] Alternatively, in the step of acquiring at least one of the user's sleep intention information, sleep state information, and sleep stage information based on the acquired user's sleep information, the converted information may be a spectrogram.

[0189] To achieve the above object, the method for providing sleep environment creation information according to the present invention includes a step in which a smart home appliance acquires sleep sound information related to a user's sleep in real time through a microphone module; a step in which a user terminal receives the acquired sleep sound information, converts it into a spectrogram, and analyzes it to determine the user's sleep stage in real time; and a step in which the user terminal outputs a control signal for controlling the operation of the smart home appliance in real time in accordance with an event occurring in each of the determined sleep stages, wherein the step of outputting the control signal includes a step in which the smart home appliance provides a sleep environment to the user in response to the control signal.

[0190] To achieve the above object, the distinguished sleep stages of the method for providing information on creating a sleep environment according to the present invention include when entry into a bedroom is detected, when a user is lying down on the bed, when falling asleep is detected, when entering deep sleep is detected, when the occurrence of sleep apnea is detected, when awakening during sleep is detected, when the occurrence of REM sleep around the alarm time is detected, and when waking up is detected.

[0191] To achieve the above object, the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when it is detected that the user has lay down in bed, if the user lying down in bed presses a sleep button, the user terminal estimates the user's intention to fall asleep.

[0192] To achieve the above object, the smart home appliances in the method for providing information on creating a sleeping environment according to the present invention include at least one of an air conditioner, a humidifier, a dehumidifier, a smart speaker, an air purifier, a smart TV, a robot vacuum cleaner, lighting, a smart bed, a clothes management machine, a smart diffuser, a washing machine, a dryer, a water purifier, and a refrigerator.

[0193] To achieve the above object, the air conditioner of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when it detects that the user has laid down in bed, the air volume and brightness of the display are set, the type of airflow is changed to indirect airflow, and the temperature is set to a level that can shorten the time it takes for the user to fall asleep according to the user's personal record.

[0194] To achieve the above object, the air conditioner of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when the onset of sleep is detected, a temperature suitable for the user is set based on past matching data on the correlation between the user's sleep quality and temperature.

[0195] To achieve the above object, the air conditioner of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, if the occurrence of sleep apnea is detected, the air conditioner is set to a temperature to protect the throat and nose of the user, if the occurrence of awakening during sleep is detected, the air conditioner is set to a temperature to allow the user to fall asleep again, and if the awakening is detected, the air conditioner is set to a temperature and air volume to help the user wake up after waking up.

[0196] To achieve the above object, the humidifier and the dehumidifier of the method for providing information on creating a sleeping environment according to the present invention are characterized in that, when it is detected that the user has laid down in bed, the humidifier and the dehumidifier are set to a humidity level suitable for each user or a humidity level that can shorten the time it takes to fall asleep according to the user's personal record.

[0197] To achieve the above object, the humidifier and the dehumidifier of the method for providing information on creating a sleeping environment according to the present invention are characterized in that, when the onset of sleep is detected, the humidifier and the dehumidifier are activated to a low noise state, and when the onset of deep sleep is detected, the humidifier and the dehumidifier determine whether the user has sleep apnea or a state of awakening during sleep, and when sleep apnea is detected, the humidifier and the dehumidifier increase the humidity in the bedroom to alleviate the symptoms of sleep apnea.

[0198] To achieve the above object, the humidifier and the dehumidifier of the method for providing information on creating a sleeping environment according to the present invention are characterized in that when it is determined that the user has woken up during sleep, the humidity is set to allow the user to fall asleep again.

[0199] To achieve the above object, the smart speaker of the method for providing information on creating a sleep environment according to the present invention is characterized in that when it detects that the user has lay down in bed, it plays sleep-inducing sounds or predetermined sleep-inducing content according to the user's personal records.

[0200] To achieve the above object, the smart speaker of the method for providing information on creating a sleep environment according to the present invention is characterized in that, when it detects that the user has entered deep sleep, it determines whether the user has sleep apnea or is in an awake state during sleep, and if it determines that the user is in an awake state during sleep, it plays sleep-inducing content without voice.

[0201] To achieve the above object, the air purifier of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when it detects that the user has lay down in bed, it lowers the illumination of the LED to reduce noise and airflow generated during operation.

[0202] To achieve the above object, the air purifier of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when it is detected that the subject has entered deep sleep, the air purifier increases the air volume and switches to a rapid cleaning mode.

[0203] To achieve the above object, in the method for providing information on creating a sleeping environment according to the present invention, when the air purifier is a table type, if the waking-up is detected, the color of the mood light is changed according to the quality of sleep of the user when waking up.

[0204] To achieve the above object, the smart TV of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when it detects that the user has lay down in bed, it displays statistics of the user's recent sleep quality and the target sleep for that day based on the statistics.

[0205] To achieve the above object, the smart TV of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, if the wake-up is detected, a predetermined sleep report is displayed on the screen when the user wakes up.

[0206] To achieve the above object, the robot cleaner of the method for providing information on creating a sleeping environment according to the present invention is characterized in that, when the onset of sleep is detected, the robot cleaner senses the user's sleep state and operates in an automatic cleaning mode, and when the deep sleep is detected, the freedom of the cleaning area and cleaning time is increased.

[0207] To achieve the above object, the robot cleaner of the method for providing information on creating a sleeping environment according to the present invention is characterized in that if awakening is detected during the sleep, the robot cleaner temporarily suspends operation if it is in a cleaning state, and returns to the charging station when the occurrence of REM sleep is detected around the time of the alarm.

[0208] To achieve the above object, the method for providing sleep environment creation information according to the present invention includes a step in which a smart watch acquires sleep sound information related to a user's sleep in real time through a microphone module; a step in which a user terminal receives the acquired sleep sound information, converts it into a spectrogram, analyzes it, and determines the user's sleep stage in real time; and a step in which the user terminal outputs a control signal for controlling the operation of the smart watch in real time in accordance with an event occurring in each of the determined sleep stages, wherein the step of outputting the control signal includes a step in which the smart watch provides a sleep environment to the user in response to the control signal.

[0209] To achieve the above object, the smart watch of the present invention, which is a method for providing information on creating a sleep environment, provides a service for falling asleep through vibration when it detects that the user has lay down in bed, and when it detects that the user has fallen asleep, adjusts the strength of the vibration in inverse proportion to the degree to which the user's sleep state is determined, or adjusts the length of the vibration according to the average time it takes for each individual to fall asleep.

[0210] To achieve the above object, the pre-sleep service of the method for providing information on creating a sleep environment according to the present invention is characterized by including a breathing technique guide and a meditation guide.

[0211] To achieve the above object, the smart watch of the present invention, which is a method for providing information on creating a sleep environment, provides a light vibration to the user to interrupt the user's sleep apnea when sleep apnea is detected or predicted to occur, and provides a wake-up alarm via vibration when REM sleep is detected around the alarm time, or provides an alarm at a time when the user is likely to wake up according to the user's personal sleep record.

[0212] To achieve the above object, the smart watch of the method for providing information on creating a sleep environment according to the present invention is characterized in that, if wake-up is detected, a predetermined sleep report is provided via a screen at the time the user wakes up.

[0213] Meanwhile, information on the method for providing information on creating a sleeping environment according to the present invention to achieve the other object may be stored in a computer-readable recording medium.

[0214] Specific details of other embodiments are included in the "Detailed Contents for Carrying Out the Invention" and the accompanying "Drawings."

[0215] The advantages and / or features of the present invention and the methods for achieving them will become more apparent with reference to the various embodiments described in detail below in conjunction with the accompanying drawings.

[0216] However, the present invention is not limited to the configuration of each embodiment disclosed below, but may be embodied in various different forms. It should be understood that each embodiment disclosed in this specification is provided solely to ensure a complete disclosure of the present invention and to fully convey the scope of the present invention to those skilled in the art to which the present invention pertains, and that the present invention is defined by the scope of each claim. [Effects of the Invention]

[0217] According to one embodiment of the present invention, it is possible to predict the time when a user is awake and / or sleep state information, and to conveniently and accurately analyze the sleep of various users at home regardless of time and place.

[0218] Furthermore, the user does not need to wear a wearable device during sleep analysis, which allows the user to have more freedom of movement during sleep.

[0219] In addition, by collecting the results of multi-dimensional sleep tests worldwide, sleep sound data can be created, and acoustic AI can be used to establish a new standard for home environment sleep tracking, validating various races, ages, genders, and measurement environments.

[0220] In addition, it can learn various ambient noises, including routine noises and abnormal or intermittent noises, in the space surrounding the user's sleep environment to build an AI sleep stage analysis model.

[0221] In addition, sound AI and wireless communication sensing clinical datasets can be constructed by utilizing smartphone sound data and smart speaker sound data collected over a long period of time, along with multidimensional sleep tests of many clinicians.

[0222] In addition, it is possible to perform in-depth analysis of the user's sleep using smart home appliances and smartphones, and it is possible to perform sleep analysis of not only one person but also many people.

[0223] In addition, if a user experiences a sleep disorder, the sleep disorder can be appropriately alleviated, and if multiple people are sleeping in the same space, an alarm for alleviating the sleep disorder can be transmitted only to the user who has experienced the sleep disorder, thereby preventing the sleep of others from being disturbed.

[0224] Additionally, smart home appliances and / or smartphones can be used to monitor a user's physical activity status in real time 24 hours a day.

[0225] In addition, according to one embodiment of the present invention, an optimized sleeping environment for improving the quality of a user's sleep can be provided through sleep state information sensed in relation to the user's sleeping environment.

[0226] In particular, the quality of sleep can be significantly improved by creating an optimal sleeping environment that takes into account various factors such as air quality, temperature and / or humidity of the sleeping environment.

[0227] The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawings]

[0228] [Figure 1] Fig. 1(a) is a conceptual diagram showing a system in which various aspects of a computing device for creating a sleep environment based on sleep state information, according to one embodiment of the present invention, can be implemented. Fig. 1(b) is a conceptual diagram showing a system in which various aspects of a sleep environment adjusting device, according to yet another embodiment of the present invention, can be implemented. Fig. 1(c) is a conceptual diagram showing a system in which various aspects of various electronic devices, according to yet another embodiment of the present invention, can be implemented. [Figure 2a] 1 illustrates a block diagram of a computing device for creating a sleep environment based on sleep state information according to an embodiment of the present invention. [Figure 2b] 1 is a block diagram illustrating an environment creating device in which a sleep state information generating unit is implemented according to an embodiment of the present invention; [Figure 2c] 2 is a block diagram illustrating an environment creation device that receives sleep state information from a server and controls an environment creation unit according to an embodiment of the present invention; [Figure 2d] 1 is a block diagram illustrating a home appliance control device in which a sleep state information generating unit according to an embodiment of the present invention is implemented; [Figure 2e] 1 is a block diagram illustrating an environment creating device that receives sleep state information from a server and controls home appliances according to an embodiment of the present invention; [Figure 2f] 1 is a block diagram illustrating an environment creating device that receives sleep state information from another electronic device and controls a home appliance according to an embodiment of the present invention; [Figure 2g] 1 is a block diagram illustrating an environment creating device in which another electronic device senses environment sensing information, receives sleep state information from a server, and controls home appliances according to an exemplary embodiment of the present invention. [Figure 2h] 1 is a block diagram illustrating an environment creation device that receives sleep state information from a first server and receives environment creation information from a second server and controls home appliances according to an embodiment of the present invention. [Figure 2i] 1 is a block diagram illustrating an environment creation device in which a second server receives sleep state information from a first server, generates environment creation information, and controls home appliances in response to the received environment creation information, according to an embodiment of the present invention. [Figure 2j] 2 is a block diagram illustrating an environment creation device controlled via a network according to an embodiment of the present invention; FIG. [Figure 3] 1 is a diagram comparing the results of polysomnography (PSG) and the analysis results (AI results) using the AI ​​algorithm according to the present invention. [Figure 4] 1 is a diagram comparing the results of polysomnography (PSG) in relation to sleep apnea and hypopnea with the analysis results (AI results) using the AI ​​algorithm according to the present invention. [Figure 5] 1 is an exemplary diagram illustrating a process of acquiring sleep sound information from environmental sensing information according to an embodiment of the present invention; [Figure 6] 6(a) is an example diagram illustrating a method for acquiring a spectrogram corresponding to sleep acoustic information according to an embodiment of the present invention, and FIG. 6(b) is a conceptual diagram illustrating a privacy protection method using mel-spectrogram conversion for sleep acoustic information extracted from a user in a sleep analysis method according to the present invention. [Figure 7] 10 is an exemplary diagram illustrating environment creation information for each time point according to a user's sleep state, according to an embodiment of the present invention; FIG. [Figure 8] 1 illustrates an exemplary flow chart for providing a method for creating a sleep environment based on sleep state information, according to an embodiment of the present invention. [Figure 9] FIG. 2 is a schematic diagram illustrating one or more network functions associated with an embodiment of the present invention. [Figure 10] 1 shows an exemplary block diagram of a sleep environment adjusting device in accordance with an embodiment of the present invention; [Figure 11]11(a) shows an exemplary block diagram of a receiving module and a transmitting module related to an embodiment of the present invention, and FIG. 11(b) is a block diagram showing the configuration of a smart home appliance in an AI-based non-contact sleep analysis system according to the present invention. [Figure 12] 10 is an exemplary view illustrating a second sensor unit that detects whether a user is located in a previously set area according to an embodiment of the present invention; FIG. [Figure 13] 10 is a flowchart illustrating an example of a process of generating sleep state information through an automatic sleep measurement mode according to an embodiment of the present invention. [Figure 14] 1 is a flowchart illustrating an example of a process for creating an environment that induces a user to fall asleep, according to an embodiment of the present invention; [Figure 15] 1 is a flowchart illustrating an example of a process of changing a user's sleep environment during sleep and immediately before waking up, according to an embodiment of the present invention; [Figure 16] Figures 16(a) and (b) are conceptual diagrams for explaining the operation of an air conditioner according to one embodiment of the present invention, and Figures 16(c) and (d) are conceptual diagrams for explaining the operation of an air purifier according to one embodiment of the present invention. [Figure 17] Figure 17(a) is a block diagram showing the configuration of an air conditioner according to one embodiment of the present invention, and Figure 17(b) is a block diagram showing the configuration of an air purifier according to one embodiment of the present invention. [Figure 18] 18 is a diagram for explaining an example of the air conditioner shown in FIGS. 16 and 17. FIG. [Figure 19] 18 is a diagram for explaining another example of the air conditioner shown in FIGS. 16 and 17. FIG. [Figure 20] 18 is a diagram for explaining still another example of the air conditioner shown in FIGS. 16 and 17. FIG. [Figure 21] 18 is a diagram for explaining still another example of the air conditioner shown in FIGS. 16 and 17. FIG. [Figure 22] 18 is a diagram for explaining still another example of the air conditioner shown in FIGS. 16 and 17. FIG. [Figure 23] 23(a) and (b) are views illustrating a method of operating the indoor unit 500'' in sleep mode via the display unit 570'' of the indoor unit 500'' shown in FIGS. 20 and 21. FIGS. 23(c) and 23(d) are views of the display unit 4000 illustrating the sleep mode of the air purifier 700' according to an embodiment of the present invention. [Figure 24] FIG. 24(a) is a diagram illustrating a method for operating the indoor units 500′, 500″, 500″″, and 500″″ shown in FIGS. 18 to 22 in a sleep mode using the remote control 600 according to an embodiment of the present invention. FIG. 24(b) is a diagram illustrating an example of a display unit 4000 of an air purifier 700′ according to an embodiment of the present invention. [Figure 25] 25(a) and (b) are diagrams illustrating a method for operating the indoor units 500', 500'', 500''', and 500'''' shown in FIGS. 18 to 22 in sleep mode via a user terminal 10 according to an embodiment of the present invention. FIG. 25(c) is a diagram showing a screen of a first application for remotely controlling the air purifier 700'' in a user terminal 10 according to an embodiment of the present invention. FIG. 25(d) is a diagram showing a screen of an application for controlling the sleep mode of the air purifier 700'' according to an embodiment of the present invention. [Figure 26] 1 is a diagram illustrating a method for automatically operating an indoor unit or an air purifier in a sleep mode. [Figure 27] 27 is a diagram illustrating a time point of the sleep mode operation shown in FIG. 26; [Figure 28] 27 is a diagram illustrating a time point of the sleep mode operation shown in FIG. 26; [Figure 29] 29(a) and (b) are diagrams for explaining an example of the air purifier shown in FIG. 16 and FIG. [Figure 30] 30 is a view showing a state in which some parts of covers 1100 and 2100 of the air purifier 700' shown in FIG. 29 have been removed. [Figure 31]Fig. 31(a) is a diagram illustrating another example of the air purifier shown in Fig. 16 and Fig. 17. Fig. 31(b) is a diagram illustrating the time point of the sleep mode operation of the air purifier 700''' shown in Fig. 26. [Figure 32] 32(a) and 32(b) are diagrams illustrating sleep stage analysis using a spectrogram in the sleep analysis method according to the present invention, and sleep disorder determination using a spectrogram in the sleep analysis method according to the present invention. [Figure 33] Figure 33(a) is a diagram showing an experimental process for verifying the performance of the sleep analysis method according to the present invention, and Figure 33(b) is a graph verifying the performance of the sleep analysis method according to the present invention, comparing the results of a multi-dimensional sleep stimuli test (PSG result) with the analysis results using the AI ​​algorithm according to the present invention (AI result). [Figure 34] 1 is a table verifying the accuracy of the sleep analysis method according to the present invention, showing experimental result data analyzed according to age, sex, BMI, and presence or absence of disease. [Figure 35] 1 is a conceptual diagram illustrating an embodiment of a sleep analysis method according to the present invention, in which a smart speaker and a smartphone are used for easy understanding. [Figure 36] 36(a) is a flowchart illustrating a method for preventing and alleviating a sleep disorder using an AI-based non-contact sleep analysis system according to one embodiment of the present invention, and FIG. 36(b) is a flowchart illustrating a method for preventing and alleviating a sleep disorder using an AI-based non-contact sleep analysis system according to another embodiment of the present invention. [Figure 37] 1 is a diagram illustrating a traffic handling method when a sleep analysis method according to the present invention is performed in a cloud; [Figure 38] FIG. 1 is a conceptual diagram for explaining sleep analysis of one person and sleep analysis of many people in the sleep analysis method according to the present invention. [Figure 39] 2 is a flowchart illustrating the operation of an AI-based non-contact sleep analysis method according to the present invention. [Figure 40]1 is a flowchart illustrating various embodiments of smart home appliances used in a sleep analysis method according to the present invention. [Figure 41] 10 is a table illustrating an example of an operation of a sleep preparation stage among specific scenarios of a plurality of smart home appliances that operate in time series according to a user's sleep stages using a sleep analysis method according to the present invention. [Figure 42] FIG. 41 is a table showing examples of the operations in the stages from after falling asleep to before deep sleep, which are linked in chronological order among the scenarios. [Figure 43] FIG. 42 is a table showing an example of the operation of the stages from after deep sleep to before wake-up detection, which are linked in chronological order among the scenarios. [Figure 44] 44 is a table showing an example of the operation of the wake-up stage, which is linked to the scenario shown in FIG. 43 in chronological order. [Figure 45] FIG. 1 is a conceptual diagram showing a training method in a hospital environment using only microphone data S from a multi-dimensional sleep test according to a conventional sleep analysis method, in order to compare the sleep analysis method of the present invention with the prior art. [Figure 46] This is a conceptual diagram of a method for generating an AI sleep analysis model by reflecting various sounds in a home environment in accordance with the sleep analysis method of the present invention, in addition to the training method shown in Figure 45. [Figure 47] 10 is a table verifying the performance of the sleep analysis method according to the present invention after training in nine groups according to the type of residential noise. [Figure 48] 1 is a schematic diagram illustrating a 24-hour monitoring process of a user according to an AI-based non-contact sleep analysis system and a sleep analysis method according to the present invention. [Figure 49] 1 is a table showing mean per class results in comparison between smart home appliances and sleep analysis methods according to the present invention and products and devices from existing world-leading sleep tech companies. [Figure 50] FIG. 2 is a block diagram illustrating the operation of an AI-based non-contact sleep analysis system according to an embodiment of the present invention. [Figure 51]FIG. 2 is a block diagram illustrating the operation between components of an AI-based non-contact sleep analysis system according to an embodiment of the present invention. [Figure 52] 10 is a table showing exemplary operations in a sleep mode and a wake-up mode, whether or not an environment creating device is activated based on sleep state information, and the detailed product of the environment creating device, according to the location where the device is placed. [Figure 53] 1 is a block diagram illustrating the operation of a system for implementing a method for providing information on creating a sleeping environment according to the present invention; [Figure 54] FIG. 54 is a conceptual diagram for explaining the operation of a wearable device in the system shown in FIG. 53. [Figure 55] 1 is a conceptual diagram for explaining the operation of a wearable device according to the present invention in conjunction with a smartphone. [Figure 56] 1 is a table showing the operation of an air conditioner and a humidifier / dehumidifier, which are major smart home appliances in one embodiment that undergo frequent operation changes among one or more smart home appliances that operate according to a user's sleep stage flow using the method for providing sleep environment creation information of the present invention. [Figure 57] 10 is a table illustrating operations of a smart speaker and an air purifier among the main smart home appliances according to an embodiment of the present invention. [Figure 58] 10 is a table illustrating operations of a smart TV and a robot vacuum cleaner among the main smart home appliances according to an embodiment of the present invention. [Figure 59a] 54 is a flowchart illustrating the operation of a smartwatch in the system shown in FIG. 53 according to another embodiment of the present invention. [Figure 59b] 54 is a flowchart illustrating the operation of a smartwatch in the system shown in FIG. 53 according to another embodiment of the present invention. [Figure 60] 10 is a table illustrating the operation of a smartwatch according to another embodiment of the present invention. [Figure 61a] 1 shows a conceptual diagram of a sleep-related product recommendation system according to the present invention. [Figure 61b] 1 shows a conceptual diagram of a sleep-related product verification system according to the present invention. [Figure 61c] 10 shows a conceptual diagram of a sleep product recommendation / verification system according to yet another embodiment of the present invention. [Figure 62] 1 is a diagram showing an experimental process for verifying the performance of a sleep analysis method according to the present invention; [Figure 63] 1 is a graph verifying the performance of the sleep analysis method according to the present invention, which compares the results of polysomnography (PSG) with the analysis results using the AI ​​algorithm according to the present invention. [Figure 64] 1 is a graph verifying the performance of the sleep analysis method according to the present invention, which compares the results of polysomnography (PSG) in relation to sleep apnea and hypopnea with the analysis results using the AI ​​algorithm according to the present invention. [Figure 65a] 3 is a flowchart illustrating a sleep product recommendation method according to the present invention. [Figure 65b] 1 is a flowchart illustrating a sleep product verification method according to the present invention. [Figure 66] 1 is a diagram illustrating an overall structure of a sleep analysis model according to an embodiment of the present invention; [Figure 67] 1 is a diagram illustrating a feature extraction model and a feature classification model according to an embodiment of the present invention; [Figure 68a] 10 is a table illustrating a lookup table in which information on sleep-related products recommended in accordance with sleep indices is recorded. [Figure 68b] 10 is a table illustrating a lookup table in which information on sleep-related products recommended in accordance with sleep indices is recorded. [Figure 69] 10 is a table illustrating a lookup table in which information on sleep-related products recommended in accordance with sleep indices is recorded. [Figure 70]Figure 70a is a table illustrating a lookup table in which composition information is recorded when the sleep product is a composition, and Figure 70b is a table illustrating a lookup table in which fiber component information is recorded when the sleep product is made of fiber components. [Figure 71a] 10 is a table illustrating a case where a verification index for sleep-related products is displayed as a score. [Figure 71b] 10 is an exemplary diagram illustrating a case where a verification index for sleep-related products is displayed as a numerical evaluation of a user's sleep, according to an embodiment of the present invention; [Figure 71c] 10 is an exemplary diagram illustrating a case where a verification index for sleep-related products is displayed as a numerical evaluation of a user's sleep, according to an embodiment of the present invention; [Figure 72] 10 is a table illustrating obtaining sleep state information from multiple users and generating verification indicators for the multiple users. [Figure 73] 10 is a table illustrating a look-up table in which recommended sleep product information obtained through sleep evaluation statistics of multiple users for multiple sleep products is recorded. [Figure 74] 1 is a diagram illustrating a graphic user interface for receiving a user's input to calculate a subjective judgment index according to an embodiment of the present invention. [Figure 75] 1 is a block diagram illustrating a cosmetic recommendation method according to an embodiment of the present invention; [Figure 76] 1 is a block diagram illustrating a cosmetics verification method according to an embodiment of the present invention. [Figure 77] FIG. 1 is a conceptual diagram showing a hot water mat system for creating a sleeping environment. [Figure 78] FIG. 10 is a conceptual diagram showing a system of yet another embodiment of a hot water mat 30-1 for creating a sleeping environment. [Figure 79] FIG. 1 is a block diagram illustrating the operation between components of an AI-based non-contact sleep analysis system using a warm water mat to create a sleep environment. [Figure 80] 10 is a diagram illustrating a hot water mat for creating a sleeping environment, in which learning and inference by the hot water mat are performed by a server. [Figure 81] 1 is a diagram illustrating a hot water mat for creating a sleeping environment, in which learning, inference, and temperature control are performed by a server. [Figure 82] 1 is a diagram illustrating a hot water mat for creating a sleeping environment, in which learning, inference, and temperature control are performed by the hot water mat. [Figure 83] This is a diagram for explaining a hot water mat in which learning and inference using the hot water mat to create a sleeping environment are performed by a first server, and temperature control is performed by a second server. [Figure 84] 1 is a diagram illustrating a device for adjusting a heat adjusting means of a hot water mat, including a processor unit on which an application can be executed. [Figure 85] 1 is a block diagram illustrating a light adjusting device receiving sleep state information from a server according to an embodiment of the present invention; [Figure 86] 1 is a conceptual diagram illustrating a light control method for creating a user's sleeping environment according to an embodiment of the present invention; [Figure 87] 1 is a flowchart illustrating a method for analyzing sleep state information including a process of combining sleep acoustic information and sleep environment information as multimodal data according to an embodiment of the present invention. [Figure 88] 1 is a flowchart illustrating a method for analyzing sleep state information, including combining inferred sleep acoustic information and inferred sleep environment information as multimodal data, according to an embodiment of the present invention. [Figure 89] 1 is a flowchart illustrating a method for analyzing sleep state information including combining inferred sleep acoustic information with sleep environment information as multimodal data according to an embodiment of the present invention; [Figure 90]1 is a diagram illustrating consistency training according to an embodiment of the present invention; [Figure 91] 1 is a diagram illustrating a user modeling method in which a user swipes to check a preferred scent, among the emotional modeling methods according to the present invention; [Figure 92] 1 is a diagram illustrating a user modeling method for receiving input of text related to a user's favorite scent, among the emotion modeling methods according to the present invention; [Figure 93] 1 is a diagram illustrating a user modeling method for selecting keywords for a scent that a user likes, among the emotion modeling methods according to the present invention; [Figure 94] 1 is a diagram illustrating a user modeling method for receiving feedback on a scent provided to a user after waking up, among the emotional modeling methods according to the present invention; [Figure 95] 2 is a diagram illustrating how the scent providing device according to the present invention provides a scent to a sleeping user; DETAILED DESCRIPTION OF THE INVENTION

[0229] The advantages and features of the present invention, and methods for achieving them, will become apparent from the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various different forms. The present embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully convey the scope of the present invention to those skilled in the art. The present invention is defined only by the scope of the claims.

[0230] When describing the embodiments disclosed herein, if it is determined that a specific description of the related publicly disclosed technology may obscure the gist of the embodiments disclosed herein, the detailed description will be omitted. Furthermore, the attached drawings are merely provided to facilitate understanding of the embodiments disclosed herein, and the technical ideas disclosed herein should not be limited by the attached drawings, and should be understood to include all modifications, equivalents, or alternatives within the spirit and technical scope of the present invention.

[0231] The terms used in this specification are intended to describe the embodiments and are not intended to limit the present invention.

[0232] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the sense commonly understood by those of ordinary skill in the art to which the present invention pertains. Furthermore, commonly used and predefined terms are not to be interpreted ideally or excessively unless expressly defined otherwise.

[0233] In this specification, the singular includes the plural unless otherwise specified. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other elements other than the elements being mentioned. The same reference numerals refer to the same elements throughout this specification, and "and / or" includes each and every combination of one or more of the elements being mentioned. Although terms such as "first," "second," etc. are used to describe various elements, these elements are not limited by these terms. These terms are used merely to distinguish one element from another. Therefore, a first element referred to below may of course be a second element within the technical spirit of the present invention.

[0234] The terms "module" and "module" used herein refer to software or hardware components, such as FPGAs or ASICs, that perform a certain function. However, "module" and "module" are not limited to software or hardware. A "module" or "module" may be configured to reside on an addressable storage medium or to execute on one or more processors. Thus, by way of example, a "module" or "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functionality provided within a "module" or "module" may be combined into fewer components and "modules" or "modules" or further separated into additional components and "modules" or "modules."

[0235] In this specification, the term "computer" refers to any type of hardware device including at least one processor, and may also encompass software configurations operating on such hardware devices according to embodiments. For example, the term "computer" may refer to, but is not limited to, smartphones, tablet PCs, desktops, laptops, and user clients and applications running on each device.

[0236] Furthermore, the "smart home appliances" described below are devices with built-in microphones that can detect the user's breathing sounds and collect acoustic data, and may include smart speakers, smart TVs, smart lighting, smart mattresses, etc.

[0237] In addition, a "sleep track app" may refer to an application that transmits a user's sleep report to a smartphone using a PUI, VUI, or GUI and operates smart home appliances according to the results of the report.

[0238] In addition, "research interaction" may mean that new products to improve users' sleep quality are researched and developed in the relevant categories, such as fragrances, cosmetics, food, health functional foods, and hormones.

[0239] In addition, "Sleep Track App Research Interaction" may mean that sleep environment creation services and new products for improving sleep quality will be developed based on sleep analysis performed by the Sleep Track App.

[0240] In addition, "sleep management app interaction" may refer to interaction between the traditional sleep industry, related industries such as sports, hotels, cram schools, and the military, which enable sleep storytelling, and a sleep management app that enables sleep analysis without a hardware solution.

[0241] Additionally, "interaction from research interaction to sleep management app" may refer to the interaction between a new product that does not have a digital product and a sleep management app that is capable of sleep analysis without a hardware solution.

[0242] Those skilled in the art should further recognize that the various illustrative logical blocks, components, modules, circuits, means, logic, and algorithm steps described in connection with the embodiments disclosed herein can be embodied in electronic hardware, computer software, or any combination of both. To clearly illustrate the interchangeability of hardware and software, the various illustrative components, blocks, components, means, logic, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is embodied as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in various ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present subject matter.

[0243] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0244] Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited to this, and depending on the embodiment, at least some of each step may be performed by different devices.

[0245] [027 items]

[0246] [Overall structure]

[0247] 1(a) is a conceptual diagram illustrating a system in which various aspects of a computing device for creating a sleeping environment based on sleep state information, related to an embodiment of the present invention, can be implemented. The system according to an embodiment of the present invention may include a computing device 100, a user terminal 10, an external server 20, an environment creation device 30, and a network. Here, the system for implementing a method for creating a sleeping environment based on sleep state information shown in FIG. 1(a) is according to one embodiment, and its components are not limited to those of the embodiment shown in FIG. 1, and may be added, changed, or deleted as necessary.

[0248] Meanwhile, FIG. 1(b) is a conceptual diagram showing a system in which various aspects of a sleep environment adjusting device related to yet another embodiment of the present invention can be implemented.

[0249] A system according to an embodiment of the present invention may include a sleep environment adjusting device 400, a user terminal 10, an external server 20, and a network. Here, the system for implementing a method for creating a sleep environment based on sleep state information shown in Fig. 1(b) is according to one embodiment, and its components are not limited to those of the embodiment shown in Fig. 1(b), and may be added, changed, or deleted as necessary.

[0250] First, a system according to the embodiment shown in FIG. 1(a) will be described.

[0251] As shown in FIG. 1(a), in the present invention, a computing device 100, a user terminal 10, an external server 20, and an environment creation device 30 can mutually transmit and receive data for a system according to an embodiment of the present invention via a network.

[0252] Networks according to embodiments of the present invention can use various wired communication systems such as Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed ​​DSL (VDSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and Local Area Network (LAN). In addition, the networks presented herein can use various wireless communication systems such as Code Division Multi Access (CDMA), Time Division Multi Access (TDMA), Frequency Division Multi Access (FDMA), Orthogonal Frequency Division Multi Access (OFDMA), Single Carrier-FDMA (SC-FDMA), and other systems.

[0253] A network according to an embodiment of the present invention may be configured regardless of its communication mode, such as wired or wireless, and may be configured as various communication networks such as a short-range communication network (PAN: Personal Area Network) or a short-range communication network (WAN: Wide Area Network). The network may be the well-known World Wide Web (WWW) or may use wireless transmission technologies used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth (registered trademark). The technology described herein may be used in other networks as well as the networks mentioned above.

[0254] According to an embodiment of the present invention, the user terminal 10 may refer to a terminal carried by a user that can receive information related to the user's sleep through information exchange with the computing device 100. For example, the user terminal 10 may be a terminal associated with a user who wishes to improve their health through information related to their sleep habits. The user can obtain information related to their sleep through the user terminal 10. The sleep-related monitoring information may include, for example, sleep state information related to the time when the user fell asleep, the time when they fell asleep, the time when they woke up, etc., or sleep stage information related to changes in sleep stages during sleep. For example, the sleep stage information may refer to information on whether the user's sleep changed to light sleep, normal sleep, deep sleep, REM sleep, etc., for each time point during the user's eight hours of sleep last night. The specific description of the sleep stage information described above is merely exemplary, and the present invention is not limited thereto.

[0255] Meanwhile, FIG. 1(c) is a conceptual diagram showing a system in which various aspects of various electronic devices related to yet another embodiment of the present invention can be implemented.

[0256] The electronic device shown in FIG. 1(c) may perform at least one of the operations performed by various devices according to embodiments of the present invention.

[0257] For example, the operations performed by various devices according to embodiments of the present invention may include an operation of acquiring environmental sensing information, an operation of learning a sleep analysis model, an operation of inferring a sleep analysis model, an operation of acquiring sleep state information, an operation of controlling an electronic device, an operation of displaying sleep state information, and an operation of displaying environmental creation information.

[0258] Or, for example, it may include operations such as receiving information related to the user's sleep, sending or receiving environmental sensing information, determining the environmental sensing information, processing or manipulating data, processing services, providing services, analyzing the sleep state, constructing a learning data set based on information related to the user's sleep, storing information on acquired data or multiple learning data for neural network learning, generating environment creation information, determining environment creation information, operating an environment creation module based on the environment creation information, sending or receiving various information, and mutually transmitting and receiving data for the system according to an embodiment of the present invention via a network.

[0259] The electronic device shown in FIG. 1(c) may individually perform the operations performed by the various devices according to the embodiments of the present invention, or may perform one or more operations simultaneously or sequentially.

[0260] Referring to (c) of FIG. 1, the electronic devices 1a to 1d may be electronic devices within a predefined area 11a, which is an area where object state information such as information about the user's movements or breathing can be acquired.

[0261] On the other hand, referring to FIG. 1(c), the electronic devices 1a and 1d may be a device made up of a combination of two or more electronic devices.

[0262] Meanwhile, referring to FIG. 1(c), the electronic devices 1a and 1b may be electronic devices connected to a network within a pre-established area 11a.

[0263] Meanwhile, referring to FIG. 1(c), electronic devices 1c and 1d may be electronic devices that are not connected to the network within the already established area 11a.

[0264] On the other hand, referring to FIG. 1(c), the electronic devices 2a to 2b may be electronic devices that are outside the range of the already set area 11a.

[0265] On the other hand, referring to (c) of FIG. 1, there may be a network that interacts with electronic devices within the range of the already established area 11a, and there may be a network that interacts with electronic devices outside the range of the already established area 11a.

[0266] Here, the network interacting with electronic devices within the range of the pre-established area 11a can perform the role of transmitting and receiving information for controlling smart home appliances.

[0267] Furthermore, the network that interacts with the electronic device within the range of the pre-established area 11a may be, for example, a short-range network or a local network, whereas the network that interacts with the electronic device within the range of the pre-established area 11a may be, for example, a long-range network or a global network.

[0268] The specific explanation of the operation of the network shown in Figure 1(c) is the same as that explained through the drawings of Figure 1(a) or Figure 1(b), so duplicated explanation will be omitted.

[0269] On the other hand, referring to (c) of FIG. 1, there may be one or more electronic devices connected via a network outside the range of the previously set area 11a, and in this case, the electronic devices may process data in a distributed manner or perform one or more operations separately.

[0270] Alternatively, if there are one or more electronic devices connected via a network outside the range of the previously set area 11a, the electronic devices may perform operations independently of each other.

[0271] Hereinafter, various aspects according to embodiments of the present invention will be described with reference to FIG. 1(c), but the present invention is not limited thereto.

[0272] For example, according to one embodiment of the present invention, in an electronic device having environmental sensing and control functions implemented therein, steps may be performed: acquiring environmental sensing information; performing preprocessing on the acquired environmental sensing information; converting acoustic information contained in the preprocessed environmental sensing information into a spectrogram; generating sleep state information based on the converted spectrogram; and controlling the electronic device to create an environment based on the generated sleep state information.

[0273] Alternatively, as shown in FIG. 51, according to one embodiment of the present invention, in an electronic device having environmental sensing and control functions, the steps of acquiring environmental sensing information, performing preprocessing on the acquired environmental sensing information, converting acoustic information included in the preprocessed environmental sensing information into a spectrogram, and transmitting the converted spectrogram to an AI server 310 may be performed. If the AI ​​server 310 generates sleep state information through learning or inference based on the transmitted spectrogram, the electronic device may receive the sleep state information generated by the AI ​​server 310, and the electronic device may be controlled to create an environment based on the received sleep state information.

[0274] Alternatively, according to one embodiment of the present invention, there may be an electronic device for controlling a home appliance for creating an environment, and the electronic device may perform the steps of acquiring environmental sensing information, performing preprocessing on the acquired environmental sensing information, converting acoustic information included in the preprocessed environmental sensing information into a spectrogram, and generating sleep state information based on the converted spectrogram, and the electronic device may perform the steps of controlling the home appliance so that the home appliance can create an environment based on the generated sleep state information.

[0275] Alternatively, according to one embodiment of the present invention, there may be an electronic device for controlling a home appliance for creating an environment, and the electronic device may perform the steps of acquiring environmental sensing information, performing preprocessing on the acquired environmental sensing information, converting acoustic information included in the preprocessed environmental sensing information into a spectrogram, and transmitting the converted spectrogram to an AI server 310. If the AI ​​server 310 generates sleep state information through learning or inference based on the transmitted spectrogram, the electronic device may receive the sleep state information generated by the AI ​​server 310, and control the home appliance to create an environment based on the received sleep state information.

[0276] Alternatively, according to an embodiment of the present invention, there may be an electronic device for controlling a home appliance for creating an environment, and another electronic device may acquire environmental sensing information, convert acoustic information included in the acquired environmental sensing information into a spectrogram, and generate sleep state information based on the converted spectrogram. The electronic device may then perform a step of receiving sleep state information from the other electronic device and a step of controlling the home appliance to create an environment based on the received sleep state information. Here, the other electronic device may be a device different from the electronic device controlling the home appliance and may correspond to one or more other electronic devices. If there are multiple other electronic devices, the steps of acquiring environmental sensing information, converting acoustic information included in the environmental sensing information into a spectrogram, and generating sleep state information may be performed independently.

[0277] For example, according to one embodiment of the present invention, there is an electronic device for controlling a home appliance for creating an environment, and if another electronic device acquires environmental sensing information, converts acoustic information included in the acquired environmental sensing information into a spectrogram, and transmits the converted spectrogram to an AI server 310, the AI ​​server 310 generates sleep state information based on the transmitted spectrogram, and the electronic device receives the sleep state information generated by the AI ​​server 310 and controls the home appliance to create an environment based on the received sleep state information. Here, the description of the other electronic device is the same as that described above, and therefore, a repeated description will be omitted.

[0278] The various embodiments of the present invention described above are examples to illustrate that various operations such as acquiring environmental sensing information, preprocessing the environmental sensing information, converting spectrograms, generating sleep state information, and controlling electronic devices or home appliances (e.g., smart home appliances) do not necessarily occur within the same electronic device, but can occur in different devices, and this can occur in chronological order, simultaneously, or independently and individually. Therefore, the present invention is not limited to the various embodiments described above.

[0279] Hereinafter, various operations according to the present invention will be described using specific examples. However, as described above, the examples of electronic devices used in the following description are merely examples for the purpose of clarity, and do not limit the electronic devices that perform specific operations.

[0280] [Explanation for Figures 2b to 2i]

[0281] FIG. 2b is a block diagram illustrating an environment creating device in which a sleep state information generating means according to an embodiment of the present invention is implemented.

[0282] According to an embodiment of the present invention, the environment creation device 30 may include an environment sensing information acquisition sensor 40, a control unit 41, and an environment creation unit 42. Specifically, the control unit 41 may include a pre-processing performing means 41-1, a sleep state information generating means 41-2, and an environment creation unit control means 41-3.

[0283] According to the present invention, the environment sensing information acquisition sensor 40 of the environment creation device 30 can acquire environment sensing information from a user. The pre-processing performing means 41-1 can perform pre-processing on the environment sensing information acquired from the environment sensing information acquisition sensor 40. The sleep state information generating means 41-2 can generate sleep state information based on the environment sensing information pre-processed by the pre-processing performing means 41-1. The environment creation unit control means 41-3 can control the environment creation unit 42 to provide a predetermined fragrance based on the generated sleep state information.

[0284] The environment creation unit control means 41-3 can control the environment creation unit 42 to provide a predetermined scent in real time based on the generated sleep state information.

[0285] The sleep state information generating means 41-2 can convert the environmental sensing information into information including changes in the frequency components of the environmental sensing information over time. Specifically, the information including changes in the frequency components over time may be a spectrogram 300.

[0286] FIG. 2c is a block diagram illustrating an environment creating device that receives sleep state information from a server and controls an environment creating unit according to an embodiment of the present invention.

[0287] According to an embodiment of the present invention, the environment creation device 30 may include an environment sensing information acquisition sensor 40, a control unit 41, an environment creation unit 42, and a communication unit 46. Specifically, the control unit 41 may include a pre-processing unit 41-1 and an environment creation unit control unit 41-3.

[0288] According to the present invention, the environment sensing information acquisition sensor 40 of the environment creation device 30 can acquire environment sensing information from a user. The pre-processing performing means 41-1 can perform pre-processing on the environment sensing information acquired from the environment sensing information acquisition sensor 40. The communication unit 46 can transmit the pre-processed environment sensing information from the pre-processing performing means 41-1 to the server 20. As a result, the communication unit 46 can receive sleep state information generated by the server 20. The environment creation unit control means 41-3 can control the environment creation unit 42 to provide a predetermined fragrance based on the received sleep state information.

[0289] The environment creation unit control means 41-3 can control the environment creation unit 42 to provide a predetermined scent in real time based on the received sleep state information.

[0290] If the server 20 converts the environmental sensing information into information including changes in frequency components of the environmental sensing information over time, the communication unit 46 can receive the converted information. Specifically, the information including changes in frequency components over time may be a spectrogram 300.

[0291] FIG. 2d is a block diagram illustrating a home appliance control device in which a sleep state information generating means according to an embodiment of the present invention is implemented.

[0292] According to an embodiment of the present invention, an electronic device 50 for controlling a home appliance may include an environment sensing information acquisition sensor 51 and a control unit 52. Specifically, the control unit 52 may include a pre-processing unit 52-1, a sleep state information generating unit 52-2, and a home appliance control unit 52-3.

[0293] According to the present invention, an environment sensing information acquisition sensor 51 of an electronic device 50 that controls home appliances can acquire environment sensing information from a user. A preprocessing performing means 52-1 can perform preprocessing on the environment sensing information acquired from the environment sensing information acquisition sensor 51. A sleep state information generating means 52-2 can generate sleep state information based on the environment sensing information preprocessed by the preprocessing performing means 52-1. Thus, a home appliance control means 52-3 can control an environment creating device 30 to provide a predetermined fragrance based on the generated sleep state information.

[0294] The home appliance control means 52-3 can control the environment creating device 30 to provide a predetermined scent in real time based on the generated sleep state information.

[0295] The sleep state information generating means 52-2 can convert the environmental sensing information into information including changes in the frequency components of the environmental sensing information over time. Specifically, the information including changes in the frequency components over time may be a spectrogram 300.

[0296] FIG. 2e is a block diagram illustrating an environment creating device for receiving sleep state information from a server and controlling home appliances according to an embodiment of the present invention.

[0297] According to an embodiment of the present invention, an electronic device 50 for controlling a home appliance may include an environment sensing information acquisition sensor 51, a control unit 52, and a communication unit 56. Specifically, the control unit 52 may include a pre-processing unit 52-1 and a home appliance control unit 52-3.

[0298] According to the present invention, an environment sensing information acquisition sensor 51 of an electronic device 50 that controls home appliances can acquire environment sensing information from a user. A preprocessing performing means 52-1 can perform preprocessing on the environment sensing information acquired from the environment sensing information acquisition sensor 51. A communication unit 56 can transmit the preprocessed environment sensing information from the preprocessing performing means 52-1 to a server 20. As a result, the communication unit 56 can receive sleep state information generated by the server 20. A home appliance control means 52-3 can control an environment creation device 30 to provide a predetermined fragrance based on the received sleep state information.

[0299] The home appliance control means 52-3 can control the environment creating device 30 to provide a predetermined fragrance in real time based on the received sleep state information.

[0300] If the server 20 converts the environmental sensing information into information including changes in frequency components of the environmental sensing information over time, the communication unit 56 can receive the converted information. Specifically, the information including changes in frequency components over time may be a spectrogram 300.

[0301] FIG. 2F is a block diagram illustrating an environment creating device for receiving sleep state information from another electronic device and controlling a home appliance according to an embodiment of the present invention.

[0302] According to one embodiment of the present invention, an electronic device 61 for controlling a home appliance may include a sleep state information receiving means 61-1 and a home appliance control means 61-2.

[0303] According to the present invention, the other electronic device 60 can acquire environmental sensing information from a user. The other electronic device 60 can perform preprocessing on the acquired environmental sensing information. The other electronic device 60 can generate sleep state information based on the preprocessed environmental sensing information. As a result, the sleep state information receiving means 61-1 of the electronic device 61 that controls the home appliances can receive the sleep state information from the other electronic device 60. As a result, the home appliance control means 61-2 can control the environment creation device 30 to provide a predetermined fragrance based on the received sleep state information.

[0304] The home appliance control means 61-2 can control the environment creation device 30 in real time based on the received sleep state information.

[0305] The other electronic device 60 can convert the environmental sensing information into information including changes in frequency components of the environmental sensing information over time. As a result, the sleep state information receiving means 61-1 of the electronic device 61 that controls the home appliance can receive the converted information. Specifically, the information including changes in frequency components over time may be a spectrogram 300.

[0306] FIG. 2g is a block diagram illustrating an environment creating apparatus in which another electronic device senses environment sensing information, receives sleep state information from a server, and controls home appliances according to an embodiment of the present invention.

[0307] According to one embodiment of the present invention, an electronic device 61 for controlling a home appliance may include a sleep state information receiving means 61-1 and a home appliance control means 61-2.

[0308] According to the present invention, the other electronic device 60 can acquire environmental sensing information from a user. The other electronic device 60 can perform preprocessing on the acquired environmental sensing information. The other electronic device 60 can transmit the preprocessed environmental sensing information to the server 20. As a result, the sleep state information receiving means 61-1 of the electronic device 61 that controls the home appliances can receive the sleep state information from the server 20. As a result, the home appliance control means 61-2 can control the environment creation device 30 to provide a predetermined fragrance based on the received sleep state information.

[0309] The home appliance control means 61-2 can control the environment creation device 30 to provide a predetermined fragrance based on the received sleep state information.

[0310] The server 20 can convert the environmental sensing information into information including changes in frequency components of the environmental sensing information over time. As a result, the sleep state information receiving means 61-1 of the electronic device 61 that controls the home appliances can receive the converted information. Specifically, the information including changes in frequency components over time may be a spectrogram 300.

[0311] FIG. 2h is a block diagram illustrating an environment creation device that receives sleep state information from a first server and receives environment creation information from a second server and controls home appliances according to an embodiment of the present invention.

[0312] According to an embodiment of the present invention, the home appliance control device 70 includes an environment sensing information acquisition sensor 71, a control unit 72, and a communication unit 76. Specifically, the control unit 72 may include a pre-processing unit 72-1 and a home appliance control unit 72-2.

[0313] According to the present invention, the environmental sensing information acquisition sensor 71 of the home appliance control device 70 can acquire environmental sensing information from a user. The pre-processing performing means 72-1 can perform pre-processing on the environmental sensing information acquired from the environmental sensing information acquisition sensor 71. The communication unit 76 can transmit the pre-processed environmental sensing information from the pre-processing performing means 72-1 to the first server 20a. Then, the first server 20a generates sleeping state information, and the communication unit 76 receives the sleeping state information. The communication unit 76 again transmits the sleeping state information to the second server 20b, and the second server 20b generates environment creation information based on the transmitted sleeping state information. The communication unit 76 receives the environment creation information from the second server 20b, and the home appliance control means 72-2 can generate environment creation device control information for controlling the environment creation device 30 based on the received environment creation information, and control the environment creation device 30. If the first server 20a converts the environmental sensing information into information including changes in frequency components of the environmental sensing information over time, the communication unit 76 can receive the converted information. Specifically, the information including changes in frequency components over time may be a spectrogram 300.

[0314] Figure 2i is a block diagram illustrating an environment creation device in which a second server according to one embodiment of the present invention receives sleep state information from a first server, generates environment creation information, and controls home appliances in response to the environment creation information.

[0315] According to an embodiment of the present invention, the home appliance control device 80 includes an environment sensing information acquisition sensor 81, a control unit 82, and a communication unit 86. Specifically, the control unit 82 may include a pre-processing unit 82-1 and a home appliance control unit 82-2.

[0316] According to the present invention, the environmental sensing information acquisition sensor 81 of the home appliance control device 80 can acquire environmental sensing information from a user. The pre-processing performing means 82-1 can perform pre-processing on the environmental sensing information acquired from the environmental sensing information acquisition sensor 81. The communication unit 86 can transmit the pre-processed environmental sensing information from the pre-processing performing means 82-1 to the first server 20a. As a result, the first server 20a generates sleeping state information, and the first server 20a transmits the sleeping state information to the second server 20b, and the second server 20b generates environment creation information based on the transmitted sleeping state information. The communication unit 86 receives the environment creation information from the second server 20b, and the home appliance control means 82-2 can generate environment creation device control information for controlling the environment creation device 30 based on the received environment creation information, thereby controlling the environment creation device 30. If the first server 20a converts the environmental sensing information into information including changes in frequency components of the environmental sensing information along a time axis, the communication unit 86 can receive the converted information. Specifically, the information including changes in frequency components along a time axis may be a spectrogram 300.

[0317] FIG. 2j is a block diagram illustrating how an environment creation device is controlled via a network according to an embodiment of the present invention.

[0318] According to an embodiment of the present invention, the home appliance control device 70 includes an environment sensing information acquisition sensor 71, a control unit 72, and a communication unit 79. Specifically, the control unit 72 may include a pre-processing unit 72-1.

[0319] The preprocessing unit 72-1 of the control unit 72 performs preprocessing on the environmental sensing information and can transmit the preprocessed environmental sensing information through the communication unit 79 that transmits and receives information via a network.

[0320] The first server 20a may receive preprocessed environment sensing information via a network and generate sleep state information, and the network may transmit the received sleep state information to the second server 20b, so that the second server 20b may generate environment creation information based on the received sleep state information and control the environment creation device 30 in real time via the network.

[0321] [Acquisition of environmental sensing information]

[0322] In an embodiment, the environmental sensing information of the present invention may be acquired via an electronic device (e.g., the user terminal 10, etc.). The environmental sensing information may refer to sensing information acquired in a space where a user is located. The environmental sensing information may be sensing information acquired in relation to the user's activity or sleep using a non-contact method.

[0323] For example, the environmental sensing information may be sleep acoustic information acquired in a bedroom where the user sleeps. According to an embodiment, the environmental sensing information acquired through the user terminal 10 may be information that serves as a basis for acquiring the user's sleep state information in the present invention. As a specific example, sleep state information related to whether the user is before sleep, during sleep, or after sleep may be acquired through the environmental sensing information acquired in relation to the user's activity.

[0324] For example, the environmental sensing information may include information about the user's breathing and movement. To this end, the user terminal 10 may include a radar sensor as a motion sensor. The user terminal 10 may perform signal processing on the user's movement and distance measured through the radar sensor to generate a discrete waveform (breathing information) corresponding to the user's breathing.

[0325] For example, the environmental sensing information may include measurements obtained via sensors measuring the temperature, humidity, and lighting levels in a bedroom. To this end, the user terminal 10 may be equipped with sensors measuring the temperature, humidity, and lighting levels in a bedroom.

[0326] The user terminal 10 may refer to any type of entity in a system having a mechanism for communication with the computing device 100. For example, the user terminal 10 may include a personal computer (PC), a notebook computer, a mobile terminal, a smartphone, a tablet PC, an AI speaker, an AI TV, a wearable device, etc., and may include all types of terminals that can be connected to a wired / wireless network. The user terminal 10 may also include any server implemented by at least one of an agent, an application programming interface (API), and a plug-in. The user terminal 10 may also include an application source and / or a client application.

[0327] According to an embodiment of the present invention, the external server 20 may be a server that stores information on a plurality of training data for training a neural network. The plurality of training data may include, for example, health checkup information or sleep screening information. For example, the external server 20 may be at least one of a hospital server and an information server, and may be a server that stores information on a plurality of sleep screening test records, electronic health records, electronic medical records, etc. For example, the sleep screening test records may include information on the breathing and movements of a sleep screening subject during sleep, and information on corresponding sleep diagnosis results (e.g., sleep stages, etc.). The information stored in the external server 20 may be used as training data, verification data, and test data for training the neural network of the present invention.

[0328] The computing device 100 of the present invention can receive health checkup information or sleep screening information from the external server 20 and construct a training dataset based on the received information. The computing device 100 can generate a sleep analysis model for acquiring sleep state information corresponding to environmental sensing information by performing training on one or more network functions through the training dataset. A configuration for constructing a training dataset for neural network training of the present invention and a training method using the training dataset will be described in detail below.

[0329] According to the present invention, the external server 20 may be a digital device equipped with a processor, memory, and computing power, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone. The external server 20 may also be a web server that processes services. The types of servers described above are merely examples, and the present invention is not limited thereto.

[0330] According to an embodiment of the present invention, the environment creation device 30 may adjust a user's sleep environment. Specifically, the environment creation device 30 may include one or more environment creation modules, and may adjust the user's sleep environment by operating an environment creation module related to at least one of air quality, illuminance, temperature, wind direction, humidity, and sound in a space where the user is located based on environment creation information received from the computing device 100.

[0331] Also, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform the above-described operations.

[0332] According to an embodiment of the present invention, the environment creating device 30 may be embodied as a TV that provides images and videos and generates sound, an air purifier that can control air quality, a lighting device that can control light intensity (illuminance), a cooler / heater that can control temperature, an air conditioner that can adjust temperature and humidity, a humidifier / dehumidifier that can control humidity, an audio / speaker that can control sound, a styler that can manage clothes, blinds or curtains, a robot or vacuum cleaner, a washing machine or dryer, a water purifier, an oven or range, etc.

[0333] The environment creation information may be a signal generated from the computing device 100 based on the determination of the user's sleep state information. For example, the environment creation information may include information on lowering or increasing illuminance. If the environment creation device 30 is a lighting device, the environment creation information may include control information for gradually increasing the illuminance of 3000K white light from 0 lux to 250 lux 30 minutes before the predicted time of waking up.

[0334] For example, if the environment creating device 30 is an air purifier or an air conditioner, the environment creating information may include various information related to temperature and / or humidity adjustment, fine dust (fine dust, ultrafine dust, and ultra-ultrafine dust) removal, harmful gas removal, allergy care activation, deodorization / sterilization activation, dehumidification / humidification adjustment, airflow intensity adjustment, air purifier or air conditioner operation noise adjustment, LED lighting, smog-causing substance (SO2, NO2) management, household odor removal, etc. If the environment creating device 30 is an air conditioner, the environment creating information may include sleep space temperature and humidity adjustment, airflow intensity adjustment, operation noise adjustment, LED lighting, etc., based on the user's real-time sleep state.

[0335] For example, the environment creation information may include control information for adjusting at least one of temperature, humidity, wind direction, and sound. The above-described specific descriptions of the environment creation information are merely examples, and the present invention is not limited thereto.

[0336] According to an embodiment of the present invention, the one or more environment creation modules included in the environment creation device 30 may include, for example, at least one of a light control module, a temperature control module, a wind direction control module, a humidity control module, and a sound control module. However, without being limited thereto, the one or more environment creation modules may further include various environment creation modules that can bring about changes in the user's sleep environment. That is, the environment creation device 30 can adjust the user's sleep environment by driving one or more environment creation modules based on an environment control signal from the computing device 100.

[0337] According to an embodiment of the present invention, the computing device 100 may acquire sleep state information of a user and adjust the user's sleep environment based on the sleep state information. Specifically, the computing device 100 may acquire sleep state information related to whether the user is about to fall asleep, asleep, or asleep based on environmental sensing information, and adjust the sleep environment of the space in which the user is located based on the sleep state information. For example, if the computing device 100 acquires sleep state information indicating that the user is about to fall asleep, the computing device 100 may generate environment creation information related to light intensity and illuminance (e.g., 3000K white light, 30 lux illuminance) and air quality (fine dust concentration, toxic gas concentration, air humidity, air temperature, etc.) for inducing sleep based on the sleep state information. The computing device 100 may transmit the environment creation information related to light intensity and illuminance and air quality for inducing sleep to the environment creation device 30. In this case, the environment creation device 30 may adjust the light intensity and illuminance of the space where the user is located to an appropriate intensity and illuminance for inducing sleep (e.g., 3000K white light with an illuminance of 30 lux) based on the environment creation information received from the computing device 100. That is, the environment creation information generated in the computing device 100 may be transmitted to a lighting device, which is one embodiment of the environment creation device 30, to adjust the illuminance in the sleeping space.

[0338] In addition, the computing device 100 can generate environment creation information such as fine dust removal, harmful gas removal, allergy care activation, deodorization / sterilization activation, dehumidification / humidification adjustment, airflow intensity adjustment, operating noise adjustment of the environment creation device 30, and various information related to LED lighting based on the user's sleep state information. In addition, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform the above-described operations. For example, the environment creation information generated in the computing device 100 may be transmitted to an air purifier or air conditioner, which is an embodiment of the environment creation device 30, to adjust the temperature, humidity, or air quality in a room, vehicle, or sleeping space.

[0339] Hereinafter, when describing the operation of smart home appliances, the terms "sleep mode" and "wake-up mode" will be used for convenience. The "sleep mode" is a concept that includes the operation modes of smart home appliances when the user is preparing to go to bed, when the user is falling asleep, and when the user is asleep, respectively. The "wake-up mode" is a concept that includes the operation modes of smart home appliances when the user is about to wake up, when the user is waking up, and when the user is waking up, respectively.

[0340] [Explanation for Figure 52]

[0341] 52 is a table showing whether or not an environment creation device is activated based on sleep state information, and exemplary operations in sleep mode and wake-up mode, for each location where the environment creation device is placed and each detailed product. Specifically, the table shows whether or not an environment creation device 30 is activated based on sleep state information (going to sleep, falling asleep, sleeping, before waking up, waking up, after waking up), and exemplary operations in sleep mode and wake-up mode, for each location where the environment creation device 30 is placed and each detailed product of the environment creation device 30. The environment creation information may include control information for enabling the activation and operations in sleep mode and wake-up mode to be performed for each product.

[0342] The above-described specific descriptions regarding the sleep state information and the environment creation information are merely examples, and the present invention is not limited thereto.

[0343] [Explanation of environmental sensing information]

[0344] According to an embodiment of the present invention, the environmental sensing information utilized by the computing device 100 for analyzing the sleep state may include information acquired non-invasively during a user's activity in a space or sleep. For specific examples, the environmental sensing information may include sounds generated when the user turns over in their sleep, sounds associated with muscle movements, or sounds associated with the user's breathing during sleep. Alternatively, the environmental sensing information may include movement and distance information associated with the user's movement during sleep and breathing information generated based thereon.

[0345] According to an embodiment, the environmental sensing information may include sleep acoustic information, which may refer to acoustic information related to movement and breathing patterns occurring during the user's sleep, or may include sleep movement information, which may refer to information related to movement and breathing patterns occurring during the user's sleep.

[0346] In an embodiment, the environmental sensing information may be acquired through a user terminal 10 carried by a user. For example, the environmental sensing information related to the user's activities in a space may be acquired through a microphone module provided in the user terminal 10. Alternatively, the environmental sensing information related to the user's activities in a space may be acquired through a radar sensor provided in the user terminal 10.

[0347] Generally, a microphone module installed in a user terminal 10 carried by a user may be configured with a Micro-Electro Mechanical System (MEMS) since it must be installed in a relatively small user terminal 10. Such a microphone module can be manufactured in a very small size, but may have a lower signal-to-noise ratio (SNR) than a condenser microphone or a dynamic microphone. A low SNR means that the ratio of noise, which is sound that makes it difficult to identify the sound to be identified, is high, and the sound is difficult to identify (i.e., unknown).

[0348] In the present invention, the environmental sensing information to be analyzed may include acoustic information related to the user's breathing and movements acquired during sleep, i.e., sleep acoustic information. Such sleep acoustic information is information about very small sounds (i.e., sounds that are difficult to distinguish), such as the user's breathing and movements, and is acquired together with other sounds in the sleep environment. Therefore, if it is acquired through the above-mentioned microphone module with a low signal-to-noise ratio, it may be very difficult to detect and analyze.

[0349] According to an embodiment of the present invention, the computing device 100 may acquire sleep state information based on environmental sensing information acquired from the user terminal 10. Specifically, the computing device 100 may convert and / or adjust data to enable analysis of the unclearly acquired environmental sensing information containing a lot of noise, and may perform training on the artificial neural network using the converted and / or adjusted data. Once pre-training of the artificial neural network is complete, the trained neural network (e.g., an acoustic analysis model) may acquire sleep state information of the user based on data (e.g., converted and / or adjusted) acquired corresponding to the sleep acoustic information (e.g., spectrogram).

[0350] In an embodiment, the sleep state information may include not only information related to whether the user is sleeping, but also sleep stage information related to changes in the user's sleep stage during sleep. For example, the sleep state information may include sleep stage information indicating that the user was in REM sleep at a first time point and in lighter sleep at a second time point different from the first time point. In this case, the sleep state information may provide information that the user was in relatively deep sleep at the first time point and in lighter sleep at the second time point.

[0351] That is, when the computing device 100 acquires sleep sound information having a low signal-to-noise ratio through many commonly used user terminals for collecting sound (e.g., AI speakers, bedroom IoT devices, mobile phones, etc.), it processes the acquired data into data suitable for analysis and can provide sleep state information related to changes in sleep stages by processing the processed data. This eliminates the need for a microphone that contacts the user's body to acquire clear sound, and also allows the monitoring of sleep states in a typical home environment with just a software update without the need to purchase a separate additional device with a high signal-to-noise ratio, thereby providing an effect of increased convenience.

[0352] Although the computing device 100 and the environment creation device 30 are shown as separate entities in FIG. 1(a), in accordance with an embodiment of the present invention, the environment creation device 30 may be included within the computing device 100, and the sleep state measurement and environment adjustment operation functions may be performed as a single integrated device.

[0353] In addition, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform the above-described operations.

[0354] In an embodiment, the computing device 100 may be a terminal or a server, and may include any type of device. The computing device 100 may be a digital device equipped with a processor, memory, and computing capabilities, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone. The computing device 100 may also be a web server that processes services. The types of servers described above are merely examples, and the present invention is not limited thereto.

[0355] According to an embodiment of the present invention, the computing device 100 may be a server that provides a cloud computing service. More specifically, the computing device 100 may be a server that provides a cloud computing service, which is a type of Internet-based computing, in which information is processed by another computer connected to the Internet, rather than the user's computer. The cloud computing service may be a service that stores data on the Internet and allows users to access the data or programs they need anytime and anywhere via an Internet connection without having to install them on their own computers. Data stored on the Internet can be easily shared and transmitted with simple operations and clicks.

[0356] Furthermore, a cloud computing service may not only simply store data on an Internet server, but also allow desired tasks to be performed using the functions of web-based applications without the need to install a separate program, and may allow various people to simultaneously share documents and work together. The cloud computing service may be implemented in at least one form of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, or a container-based cloud server. That is, the computing device 100 of the present invention may be implemented in at least one form of the above-described cloud computing services. The specific descriptions of the above-described cloud computing services are merely examples and may include any platform for constructing the cloud computing environment of the present invention.

[0357] [Overall configuration of computing equipment]

[0358] The specific configuration, technical features, and effects of the technical features of the computing device 100 of the present invention will be described with reference to the accompanying drawings.

[0359] FIG. 2 illustrates a block diagram of a computing device for creating a sleep environment based on sleep state information in accordance with an embodiment of the present invention.

[0360] 2, the computing device 100 may include a network unit 110, a memory 120, and a processor 130. The computing device 100 is not limited to the components described above. That is, additional components may be included, or some of the components described above may be omitted, depending on the implementation of the present invention.

[0361] According to an embodiment of the present invention, the computing device 100 may include a network unit 110 that transmits and receives data to and from the user terminal 10, the external server 20, and the environment creating device 30. The network unit 110 may transmit and receive data, etc., for performing the method for creating a sleep environment using sleep state information according to an embodiment of the present invention, to and from other computing devices, servers, etc.

[0362] That is, the network unit 110 may provide a communication function between the computing device 100, the user terminal 10, the external server 20, and the environment creation device 30. For example, the network unit 110 may receive sleep examination records and electronic health records for a plurality of users from a hospital server. As another example, the network unit 110 may receive environmental sensing information related to a space in which the user is active from the user terminal 10. As another example, the network unit 110 may transmit environment creation information for adjusting the environment of the space in which the user is located to the environment creation device 30. Additionally, the network unit 110 may allow information transmission between the computing device 100, the user terminal 10, and the external server 20 by calling a procedure in the computing device 100.

[0363] The network unit 110 according to an embodiment of the present invention can use various wired communication systems such as a Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed ​​DSL (VDSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and a Local Area Network (LAN).

[0364] In addition, the network unit 110 presented in this specification can use various wireless communication systems that can be realized now and in the future, such as mobile communication systems such as 4G and 5G (LTE), and satellite communication systems such as Starlink.

[0365] In the present invention, the network unit 110 may be configured regardless of the communication mode, such as wired or wireless, and may be configured as various communication networks, such as a short-range communication network (PAN: Personal Area Network) or a short-range communication network (WAN: Wide Area Network). In addition, the network may be the well-known World Wide Web (WWW), or may use wireless transmission technology used for short-range communication, such as Infrared Data Association (IrDA) or Bluetooth (registered trademark). The technology described herein can be used not only with the networks mentioned above, but also with other networks.

[0366] According to an embodiment of the present invention, the memory 120 may store a computer program for performing a method for creating a sleep environment based on sleep state information according to an embodiment of the present invention, and the stored computer program may be read and driven by the processor 130. The memory 120 may also store any type of information generated or determined by the processor 130 and any type of information received by the network unit 110. The memory 120 may also store data related to the user's sleep. For example, the memory 120 may temporarily or permanently store input / output data (e.g., environmental sensing information related to the user's sleep environment, sleep state information corresponding to the environmental sensing information, or environment creation information based on the sleep state information).

[0367] According to an embodiment of the present invention, the memory 120 may include at least one type of storage medium selected from the group consisting of flash memory, hard disk, micro multimedia card, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. The computing device 100 may also operate in association with web storage that performs the storage function of the memory 120 over the Internet. The above description of memory is for illustrative purposes only, and the present invention is not limited thereto.

[0368] The computer program may include one or more instructions that, when loaded into memory 120, cause processor 130 to perform methods / operations according to various embodiments of the present invention. That is, processor 130 may perform methods / operations according to various embodiments of the present invention by executing one or more instructions.

[0369] In one embodiment, the computer program may include one or more instructions for performing a method for creating a sleep environment based on sleep state information, the method including acquiring sleep state information of a user, generating environment creation information based on the sleep state information, and transmitting the environment creation information to an environment creation device.

[0370] According to one embodiment of the present invention, processor 130 may be configured with one or more cores and may include a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), or other processors for data analysis, deep learning, etc.

[0371] The processor 130 may read a computer program stored in the memory 120 and perform data processing for machine learning according to an embodiment of the present invention. According to an embodiment of the present invention, the processor 130 may perform calculations for neural network training. The processor 130 may perform calculations for neural network training, such as processing input data for training in deep learning (DL), extracting features from the input data, calculating errors, and updating weights of the neural network using backpropagation.

[0372] In addition, at least one of the CPU, GPGPU, and TPU of the processor 130 may process network function training. For example, the CPU and GPGPU may both process network function training and data classification using the network function. In addition, in an embodiment of the present invention, processors of multiple computing devices may be used together to process network function training and data classification using the network function. In addition, a computer program executed in a computing device according to an embodiment of the present invention may be a CPU-, GPGPU-, or TPU-executable program.

[0373] As used herein, a network function may be used interchangeably with an artificial neural network or a neural network. As used herein, a network function may include one or more neural networks, in which case the output of the network function may be an ensemble of the outputs of the one or more neural networks.

[0374] As used herein, a model may include a network function. A model may include one or more network functions, in which case the output of the model may be an ensemble of the outputs of the one or more network functions.

[0375] The processor 130 may read a computer program stored in the memory 120 to provide a sleep analysis model according to an embodiment of the present invention. According to an embodiment of the present invention, the processor 130 may perform calculations to calculate environment creation information based on sleep state information. According to an embodiment of the present invention, the processor 130 may perform calculations to train the sleep analysis model. The sleep analysis model will be described in more detail below.

[0376] According to the present invention, sleep information related to the quality of a user's sleep can be inferred based on a sleep analysis model. Environmental sensing information acquired from a user in real time or periodically is input as an input value to the sleep analysis model, and data related to the user's sleep is output.

[0377] The learning of the sleep analysis model and the inference based thereon may be performed by the computing device 100 of FIG. 1(a). That is, both the learning and the inference may be designed to be performed by the computing device 100. However, in another embodiment, the learning may be performed by the computing device 100, but the inference may be performed by the user terminal 10. Also, the learning may be performed by the computing device 100, but the inference may be performed by the environment creation device 30 embodied in smart home appliances (various home appliances such as air conditioners, TVs, lighting, refrigerators, air purifiers, etc.). Also, in another embodiment, the learning may be performed by the sleep environment control device 400 of FIG. 1(b). That is, both the learning and the inference may be performed by the sleep environment control device 400.

[0378] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0379] According to an embodiment of the present invention, the processor 130 may generally process the overall operation of the computing device 100. The processor 130 may process signals, data, information, etc. input or output via the components described in detail above, or may run applications stored in the memory 120, thereby providing or processing appropriate information or functions to the user terminal.

[0380] According to an embodiment of the present invention, processor 130 may acquire sleep state information of the user. Acquisition of sleep state information according to an embodiment of the present invention may involve acquiring or loading sleep state information stored in memory 120. Acquisition of sleep state information may also involve receiving or loading data from another storage medium, another computing device, or another processing module within the same computing device via wired or wireless communication.

[0381] Also, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0382] [Sleep status information]

[0383] In one embodiment, the sleep state information may include information related to whether the user is asleep. Specifically, the sleep state information may include at least one of first sleep state information indicating that the user is before sleeping, second sleep state information indicating that the user is asleep, and third sleep state information indicating that the user has fallen asleep. In other words, if first sleep state information is inferred related to a user, processor 130 may determine that the user is in a pre-sleep (i.e., before falling asleep) state; if second sleep state information is inferred, processor 130 may determine that the user is in a sleeping state; and if third sleep state information is obtained, processor 130 may determine that the user is in a post-sleep (i.e., wake-up) state.

[0384] The sleep state information may be acquired based on environmental sensing information, which may include sensing information acquired in a space where the user is located in a non-contact manner.

[0385] According to an embodiment, the processor 130 may acquire environmental sensing information. Specifically, the environmental sensing information may be acquired via the user terminal 10 carried by the user. For example, environmental sensing information related to a space in which the user is active may be acquired via the user terminal 10 carried by the user, and the processor 130 may receive the environmental sensing information from the user terminal 10. The environmental sensing information may be acoustic information acquired in a non-contact manner during the user's daily life. For example, the environmental sensing information may include various acoustic information acquired during the user's daily life, such as acoustic information related to cleaning, acoustic information related to cooking food, acoustic information related to TV viewing, and sleep acoustic information acquired during sleep. In an embodiment, the sleep acoustic information acquired during the user's sleep may include acoustic information generated by the user turning over in bed during sleep, acoustic information related to muscle movement, or acoustic information related to the user's breathing during sleep. That is, the sleep acoustic information in the present invention may refer to acoustic information related to the user's movement patterns and breathing patterns during sleep.

[0386] [Sleep analysis information and sleep stage information]

[0387] Sleep analysis analyzes various information such as time to fall asleep, time to wake up, and total sleep time. According to an embodiment, the processor 130 can extract sleep stage information. The sleep stage information can be extracted based on environmental sensing information of the user. Sleep stages can be classified into non-REM (NREM) sleep and rapid eye movement (REM) sleep, and NREM sleep can be further classified into multiple stages (e.g., two stages: light and deep, and four stages: N1 to N4). Sleep stages can be defined as general sleep stages, or various sleep stages can be arbitrarily set by the designer. Sleep stage analysis can predict not only sleep quality but also sleep disorders (e.g., sleep apnea) and their underlying causes (e.g., snoring).

[0388] In sleep analysis, changes in sleep stages can be analyzed and a hypnogram can be generated to enable changes in the analyzed sleep stages to be identified, thereby allowing the user's sleep cycles to be identified.

[0389] FIG. 3 is a graph comparing the results of polysomnography (PSG) and the results of analysis using the AI ​​algorithm according to the present invention.

[0390] As shown in Figure 3, the sleep stage information obtained by the present invention not only closely matches that of a multidimensional sleep test, but also includes more precise and meaningful information related to sleep stages (Wake, Light, Deep, REM). The hypnogram shown at the bottom of Figure 3 shows the probability of which of four classes (Wake, Light, Deep, REM) the user will fall into in 30-second increments when predicting a sleep stage based on input user audio information. Here, the four classes represent the awake state, light sleep state, deep sleep state, and REM sleep state, respectively.

[0391] Figure 4 compares the results of a polysomnography (PSG) test (PSG result) and the analysis results (AI result) using the AI ​​algorithm according to the present invention in relation to sleep apnea and hypopnea. The hypnogram shown at the bottom of Figure 4 shows the probability of the user being diagnosed with one of two disorders (sleep apnea or hypopnea) in 30-second intervals when predicting a sleep disorder based on input of user audio information.

[0392] Using the sleep stage information according to the present invention, as shown in FIG. 4, the sleep stage information obtained according to the present invention is not only highly consistent with the sleep polymorphism test, but also includes more precise analytical information related to apnea and hypopnea.

[0393] According to the present invention, the processor 130 may generate environment creation information according to the sleep stage information. For example, if the sleep stage is the Light stage or the N1 stage, the processor 130 may generate environment creation information for controlling environment creation devices (such as an air conditioner, lighting, or air purifier) ​​to induce deep sleep.

[0394] For example, some smart home appliances 800 according to embodiments of the present invention may be configured to sound an alarm if REM sleep is detected within 30 minutes of the wake-up time set by the user.

[0395] This is because if an alarm sounds during REM sleep, the user will wake up more refreshed. The sleep management app of the present invention can detect REM in real time while the user is sleeping and deliver auditory or tactile stimuli to the user to wake them up within the specified time.

[0396] In addition, some smart home appliances 800 according to embodiments of the present invention may detect periods of unstable breathing based on sleep acoustic information while the user is sleeping, and may provide vibrotactile stimulation to the user to encourage the user to return to stable breathing.

[0397] Generally, if sleep apnea persists, the sympathetic nervous system becomes activated, which may later lead to cardiovascular diseases. Therefore, when an unstable breathing period is detected in real time during the user's sleep through the sleep management app of the present invention, auditory and tactile stimuli can be transmitted to the user through a part of the smart home appliance 800 according to an embodiment of the present invention to interrupt the user's unstable breathing.

[0398] Obstructive sleep apnea can be screened in stages based on body movement information or user posture information.

[0399] The sleep analysis analyzes the quality of sleep, sleep stages, and the presence or absence of sleep apnea based on sleep acoustic information. The sleep acoustic information may refer to acoustic information related to breathing generated during a user's sleep.

[0400] Sleep analysis analyzes the user's sleep stages through pre-processing of the user's sleep acoustic information and AI algorithms. The specific analysis method will be explained in more detail below.

[0401] [Identifying singular points using pre-configured pattern detection]

[0402] According to an embodiment of the present invention, the processor 130 may acquire sleep state information based on the environmental sensing information. Specifically, the processor 130 may identify a singular point where pre-established pattern information is detected in the environmental sensing information. Here, the pre-established pattern information may be related to breathing and movement patterns associated with sleep. For example, in a wakeful state, the entire nervous system is activated, resulting in irregular breathing patterns and frequent body movements. In addition, the throat muscles may not be relaxed, resulting in very little breathing sound.

[0403] On the other hand, when the user is sleeping, the autonomic nervous system is stabilized, breathing becomes regular, body movements become less, and breathing sounds become louder. That is, the processor 130 may identify, as a singular point, a time point at which sound information of a predetermined pattern associated with regular breathing, little body movements, or little breathing sounds is detected in the environmental sensing information. The processor 130 may also acquire sleep sound information based on the environmental sensing information acquired based on the identified singular point. The processor 130 may identify a singular point associated with the user's sleep time point in the environmental sensing information acquired in a time series, and acquire sleep sound information based on the singular point.

[0404] FIG. 5 is an exemplary diagram illustrating a process of acquiring sleep sound information 210 from environmental sensing information 200 according to an embodiment of the present invention.

[0405] 5, the processor 130 may identify a singular point 201 associated with a point in time at which a previously established pattern is identified from the environmental sensing information 200. The processor 130 may acquire sleep sound information 210 based on sound information acquired after the identified singular point. The sound-related waveforms and singular points in FIG. 5 are merely examples for understanding the present invention, and the present invention is not limited thereto.

[0406] That is, the processor 130 can identify specific points related to the user's sleep from the environmental sensing information, and extract and acquire only the sleep sound information from a huge amount of sound information (i.e., the environmental sensing information) based on the specific points. This can automate the process of the user recording their sleep time, providing convenience and contributing to improving the accuracy of the acquired sleep sound information.

[0407] In addition, in an embodiment, the processor 130 may acquire sleep state information related to whether the user is asleep or not based on the singular point 201 identified from the environmental sensing information 200. Specifically, the processor 130 may determine that the user is asleep if the singular point 201 is not identified, and may determine that the user is asleep after the singular point 201 is identified. Furthermore, the processor 130 may identify a time point (e.g., a wake-up time) at which a previously set pattern is not observed after the singular point 201 is identified, and may determine that the user has fallen asleep, i.e., woken up, if the time point is identified.

[0408] That is, the processor 130 can acquire sleep state information related to whether the user is before sleep, during sleep, or after sleep based on whether a singular point 201 is identified in the environmental sensing information 200 and whether a previously set pattern is continuously sensed after the singular point is identified.

[0409] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0410] In addition, the processor 830 included in the smart home appliance 800 according to an embodiment of the present invention may identify a singular point 201 associated with a point in time at which a previously established pattern is identified from the environmental sensing information 200.

[0411] According to the present invention, the processor 830 can acquire the sleep acoustic information 210 based on the acoustic information acquired after the identified singular point 201, with the identified singular point 201 as a reference.

[0412] The acoustic waveforms and singularities in FIG. 5 are merely examples for understanding the present invention, and the present invention is not limited thereto.

[0413] That is, the processor 830 included in the smart home appliance 800 according to an embodiment of the present invention can identify singular points 201 related to the user's sleep from the acoustic information, and extract and acquire only the sleep acoustic information 210 from a vast amount of environmental sensing information (i.e., acoustic information) based on the singular points 201.

[0414] This provides convenience by automating the process of recording the user's sleep time, and can also contribute to improving the accuracy of the acquired sleep acoustic information.

[0415] In addition, in an embodiment, the processor 830 may acquire sleep state information related to whether the user is about to fall asleep or is asleep based on the singular point 201 identified from the environmental sensing information 200. Specifically, the processor 830 may determine that the user is about to fall asleep if the singular point 201 is not identified, and may determine that the user is asleep after the singular point 201 is identified.

[0416] In addition, after identifying singular point 201, processor 830 identifies a time point (e.g., the time of waking up) at which the already set pattern is not observed, and if that time point is identified, it can determine that the user has fallen asleep, i.e., woken up.

[0417] That is, the processor 830 can acquire sleep state information related to whether the user is before sleep, during sleep, or after sleep based on whether a singular point 201 is identified in the environmental sensing information 200 and whether a previously set pattern is continuously sensed after the singular point is identified.

[0418] On the other hand, the processor 830 can obtain the sleep state information based on the sleep acoustic information rather than the environmental sensing information 200 .

[0419] In the present invention, the sleep state information of the user is grasped in advance using sleep acoustic information during the primary sleep analysis, so that the reliability of the analysis of the sleep state can be further improved.

[0420] The sleep analysis method according to the present invention generates an inference model through deep learning of environmental sensing information, and the inference model extracts the user's sleep state and sleep stage.

[0421] To put it simply again, environmental sensing information 200, including sleep acoustic information, is converted into a spectrogram, and an inference model is generated based on the spectrogram.

[0422] In this case, in sleep analysis using acoustic information, the protection of user privacy cannot be overlooked, and the present invention uses a process of preprocessing the environmental sensing information 200 to protect the user's privacy.

[0423] As described above, an inference model for extracting a user's sleep state and sleep stage is generated through deep learning of the environmental sensing information 200. Briefly again, the environmental sensing information 200 including acoustic information, etc., is converted into a spectrogram, and an inference model can be generated based on the spectrogram.

[0424] As described above, the inference model can be built in the computing device 100 shown in FIG. 1(a) or the sleep environment adjustment device 400 shown in FIG. 1(b).

[0425] Thereafter, environmental sensing information including user acoustic information acquired through the user terminal 10 is input to the inference model, and sleep state information and / or sleep stage information is output as a result value. At this time, learning and inference may be performed by the same entity, or may be performed by separate entities. That is, both learning and inference may be performed by the computing device 100 of FIG. 1(a) or the environmental control device 400 of FIG. 1(b). Learning may be performed by the computing device 100, but inference may be performed by the user terminal 10. Learning may be performed by the computing device 100, but inference may be performed by the environment creation device 30 implemented in smart home appliances (such as air conditioners, TVs, lighting, refrigerators, air purifiers, etc.).

[0426] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the operations described above.

[0427] [Sleep analysis model and sleep analysis method]

[0428] According to an embodiment of the present invention, the sleep stage information may be obtained through a sleep analysis model that analyzes the sleep stages of a user based on environmental sensing information. That is, the sleep stage information of the present invention may be obtained through a sleep analysis model.

[0429] According to an embodiment of the present invention, processor 130 or processor 830 may acquire environmental sensing information and acquire sleep sound information based on the environmental sensing information. In this case, the sleep sound information is information related to sounds acquired while the user is sleeping, and may include, for example, sounds generated by the user turning over in their sleep, sounds related to muscle movements, or sounds related to the user's breathing while sleeping.

[0430] Hereinafter, a sleep analysis method according to an embodiment of the present invention will be described with reference to the drawings.

[0431] FIG. 32(a) is a diagram for explaining sleep stage analysis using a spectrogram in the sleep analysis method according to the present invention.

[0432] FIG. 32(b) is a diagram for explaining the determination of a sleep disorder using a spectrogram in the sleep analysis method according to the present invention.

[0433] FIG. 33(a) is a diagram showing an experimental process for verifying the performance of the sleep analysis method according to the present invention.

[0434] Figure 33(b) is a graph verifying the performance of the sleep analysis method according to the present invention, comparing the results of a sleep polymorphism test (PSG result) with the analysis results (AI result) using the AI ​​algorithm according to the present invention.

[0435] As shown in FIG. 32(a), when the sleep sound information of the user is input, the corresponding sleep stage (Wake, REM, Light, Deep) may be immediately inferred.

[0436] Furthermore, secondary analysis based on sleep acoustic information can extract the time points at which sleep disorders (sleep apnea, hyperventilation) and snoring occur through the singular points of the Mel spectrum corresponding to sleep stages.

[0437] As shown in Figure 32(b), if a breathing pattern is analyzed in one melspectrogram and characteristics corresponding to a sleep apnea or hyperpnea event are detected, the time point can be determined as the time point at which a sleep disorder occurred. In this case, a step of classifying the event as snoring rather than sleep apnea or hyperpnea through frequency analysis may be further included.

[0438] As shown in Figure 33(a), a user's sleep video and sleep sounds are acquired in real time, and the acquired sleep sound information is immediately converted into a spectrogram. At this time, a pre-processing process of the sleep sound information may be performed. The spectrogram is input into a sleep analysis model, and the sleep stages are immediately analyzed.

[0439] When compared with the results of polysomnography (PSG), it was confirmed that the results of the sleep analysis model using sleep acoustic information as input were highly accurate.

[0440] The hypnogram at the bottom of Figure 33(a) shows the probability of which of four classes (Wake, Light, Deep, REM) the user will fall into in 30-second increments when predicting their sleep stage based on the input of the user's sleep acoustic information. Here, the four classes represent the awake state, light sleep state, deep sleep state, and REM sleep state, respectively.

[0441] As shown in Figure 33(b), the sleep analysis results obtained by the present invention are not only highly consistent with the sleep polymorphism test, but also contain more precise and meaningful information related to sleep stages (Wake, Light, Deep, REM).

[0442] [Spectrogram generation and acquisition]

[0443] FIG. 6(a) is an exemplary diagram illustrating a method for acquiring a spectrogram corresponding to sleep acoustic information according to an embodiment of the present invention.

[0444] According to the present invention, a sleep analysis model can be generated using a spectrogram generated based on sleep audio information. If sleep audio information expressed as audio data were used directly, the amount of information would be very large, resulting in a significant increase in the amount of calculation and time required. Furthermore, the inclusion of unwanted signals would reduce calculation accuracy. Furthermore, if all of a user's audio signals were transmitted to a server, privacy could be infringed. The present invention removes noise from the sleep audio information, converts it into a spectrogram (Mel spectrogram), and then trains the spectrogram to generate a sleep analysis model. This reduces the amount of calculation and time required, and even protects personal privacy.

[0445] The processor 130 or the processor 830 according to the embodiment of the present invention can generate a spectrogram 300 corresponding to the sleep acoustic information 210 as shown in FIG. 6(a).

[0446] Raw data (sleep acoustic information) that is the basis for generating the spectrogram 300 can be input. The raw data can be acquired from the start time to the end time entered by the user via a user terminal, or from the time when the user operates the terminal (e.g., setting an alarm) to the time corresponding to the terminal operation (e.g., the alarm setting time). The raw data can also be acquired by automatically selecting a time based on the user's sleep pattern. Alternatively, the user's intended sleep time can be automatically determined and acquired based on sounds (such as the user's voice, breathing sounds, sounds from peripheral devices (TV, washing machine), etc.) or changes in illumination.

[0447] Although not shown in FIG. 6(a), a preprocessing process of input raw data may be further included. The preprocessing process includes a noise reduction process of the raw data. In the noise reduction process, noise (e.g., white noise) contained in the raw data is removed. The noise reduction process may be performed using algorithms such as spectral gating and spectral subtraction to remove background noise. Furthermore, in the present invention, the noise reduction process may be performed using a deep learning-based noise reduction algorithm. That is, a noise reduction algorithm specialized for the user's breathing and respiratory sounds may be used through deep learning. In particular, the present invention may generate a spectrogram based only on the amplitude of the raw data, excluding the phase, but is not limited thereto. This not only protects privacy but also reduces data volume and improves processing speed.

[0448] According to an embodiment of the present invention, the processor 130 or the processor 830 may perform a fast Fourier transform on the sleep acoustic information 210 to generate a spectrogram 300 corresponding to the sleep acoustic information 210 .

[0449] Spectrogram 300 is used to visualize and understand sound or waves, and may be a combination of waveform and spectrum characteristics. Spectrogram 300 may show differences in amplitude according to changes in the time axis and frequency axis as differences in print density or display color.

[0450] The preprocessed acoustic-related raw data may be cut into 30-second units and converted into a mel spectrogram. As a result, a 30-second mel spectrogram may have dimensions of 20 frequency bins x 1201 time steps. In the present invention, a split-cut method is used to convert a rectangular mel spectrogram into a square form, thereby enabling the storage of information.

[0451] The present invention can simulate breathing measured in various home environments by adding various noises generated in home environments to clean breathing. Because sounds have an additive nature, they can be added to each other. However, adding an original audio signal such as MP3 or PCM and converting it into a mel spectrogram can consume a lot of computing resources.

[0452] Therefore, the present invention proposes a method for converting breathing and noise into mel spectrograms and adding them to the mel spectrograms, thereby simulating breathing measured in various home environments and utilizing the simulated results for deep learning model training, thereby ensuring robustness in various home environments.

[0453] In the present invention, the sleep sound information 210 may be very quiet because it is related to sounds associated with breathing and body movements acquired during the user's sleep. Accordingly, the processor 130 or 830 can convert the sleep sound information into a spectrogram 300 to perform analysis on the sounds. In this case, as described above, the spectrogram 300 includes information indicating how the frequency spectrum of the sound changes over time, so that breathing or movement patterns associated with relatively quiet sounds can be easily identified, thereby improving the efficiency of the analysis.

[0454] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform the above-described operations.

[0455] According to an embodiment, each spectrogram may be configured to have a frequency spectrum with different density according to various sleep stages. Specifically, it may be difficult to predict whether a sleeper is in at least one of an awake state, a REM sleep state, a light sleep state, and a deep sleep state based solely on a change in the energy level of the sleep sound information. However, by converting the sleep sound information into a spectrogram, it may be possible to easily detect changes in the spectrum of each frequency, thereby enabling analysis corresponding to small sounds (e.g., breathing and body movements).

[0456] Furthermore, processor 130 or processor 830 may process spectrogram 300 as an input to a sleep analysis model to acquire sleep stage information. Here, the sleep analysis model is a model for acquiring sleep stage information related to changes in a user's sleep stage, and may input sleep acoustic information acquired during the user's sleep and output sleep stage information. In an embodiment, the sleep analysis model may include a neural network model configured via one or more network functions.

[0457] [Network Functions and Neural Networks]

[0458] In an embodiment of the present invention, the sleep analysis model may include a neural network model configured via one or more network functions. The sleep analysis model is configured by one or more network functions, and the one or more network functions may be configured by a set of interconnected computational units that may generally be referred to as "nodes." Such "nodes" may also be referred to as "neurons." The one or more network functions are configured to include at least one or more nodes. The nodes (or neurons) that configure the one or more network functions may be interconnected by one or more "links."

[0459] FIG. 9 is a schematic diagram illustrating one or more network functions associated with an embodiment of the present invention.

[0460] A deep neural network (DNN) can refer to a neural network that contains multiple hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to understand the latent structures of data.

[0461] That is, it is possible to grasp the latent structure of a photo, text, video, audio, or music (e.g., what object is in the photo, what is the content and emotion of the text, what is the content and emotion of the audio, etc.). The deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q-network, a U-network, a Siamese network, etc. The above description of the deep neural network is merely an example, and the present invention is not limited thereto.

[0462] In one embodiment of the present invention, the network function may include an autoencoder, which may be a type of artificial neural network for generating output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between the input and output layers.

[0463] The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and may be reduced and expanded symmetrically from the bottleneck layer to the output layer (symmetric to the input layer). The nodes in the dimensionality reduction layer and the dimensionality restoration layer may be symmetric or asymmetric.

[0464] An autoencoder according to an embodiment of the present invention may perform nonlinear dimensionality reduction. The number of input layers and output layers may correspond to the number of sensors remaining after preprocessing of input data. The autoencoder structure may have a structure in which the number of nodes in hidden layers included in the encoder decreases with increasing distance from the input layer.

[0465] If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, it may not be able to convey a sufficient amount of information, so it may be kept above a certain number (e.g., more than half of the number in the input layer).

[0466] Neural networks can be trained using at least one of supervised learning, unsupervised learning, and semi-supervised learning. The purpose of training a neural network is to minimize the error in the output.

[0467] In neural network training, training data is repeatedly input into the neural network, the error between the neural network output and the target for the training data is calculated, and the error of the neural network is backpropagated from the output layer to the input layer of the neural network in a direction to reduce the error, thereby updating the weight values ​​of each node of the neural network.

[0468] In supervised learning, training data in which each training data is labeled with a correct answer (i.e., labeled training data) is used, while in unsupervised learning, each training data may not be labeled with a correct answer. That is, for example, in the case of supervised learning for data classification, the training data may be data in which each training data is labeled with a category.

[0469] Labeled training data is input to a neural network, and an error can be calculated by comparing the neural network's output (category) with the label of the training data. As another example, in unsupervised learning for data classification, the input training data can be compared with the neural network output to calculate the error.

[0470] The calculated error is backpropagated from the neural network in the backward direction (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated by backpropagation. The amount of change in the connection weights of each node to be updated can be determined by the learning rate.

[0471] The neural network calculations on the input data and backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations of the neural network learning cycle.

[0472] For example, in the early stages of neural network learning, a high learning rate can be used to quickly ensure a certain level of performance, thereby increasing efficiency, and in the later stages of learning, a lower learning rate can be used to increase accuracy.

[0473] In training a neural network, the training data may generally be a subset of the actual data (i.e., the data to be processed using the trained neural network), and therefore there may be training cycles in which the error decreases for the training data but increases for the actual data.

[0474] Overfitting is a phenomenon in which excessive learning from training data leads to an increase in errors in real data. For example, a neural network that has learned to recognize cats by showing them yellow cats may not be able to recognize cats when it sees a cat that is not yellow.

[0475] Overfitting can increase the error of machine learning algorithms. Various optimization methods can be used to prevent overfitting. To prevent overfitting, methods such as increasing the amount of training data, regularization, and dropout, which omits some nodes of the network during the training process, can be applied.

[0476] Throughout this specification, the terms computational model, neural network, network function, and neural network may be used interchangeably (hereinafter, the term neural network will be used interchangeably). The data structure may include a neural network.

[0477] The data structure including the neural network may then be stored on a computer-readable medium. The data structure including the neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and a loss function for training the neural network.

[0478] A data structure including a neural network may include any of the components disclosed above, such as data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, a loss function for training the neural network, and any combination thereof. In addition to the above-described components, a data structure including a neural network may include any other information that determines the characteristics of the neural network.

[0479] Furthermore, the data structure may include all types of data used or generated in the computational process of a neural network, and is not limited to the above. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network may be composed of a collection of interconnected computational units that may generally be referred to as nodes. Such nodes may also be referred to as neurons. A neural network is composed of at least one or more nodes.

[0480] Within a neural network, one or more nodes connected via links can form a relative relationship of input node and output node. The concepts of input node and output node are relative, and any node that is in an output node relationship with one node can also be in an input node relationship with another node, and vice versa.

[0481] As mentioned above, the relationship between input nodes and output nodes may be created around links. As shown in Figure 8, one input node may be connected to one or more output nodes via links, and vice versa.

[0482] In a relationship between an input node and an output node connected via a link, the value of the output node may be determined based on data input to the input node, and the node interconnecting the input node and the output node may have a weight.

[0483] The weights may be variable and may be varied by a user or an algorithm so that the neural network can perform a desired function. For example, if one or more input nodes are interconnected to one output node by respective links, the output node may determine its output node value based on the values ​​input to the input nodes connected to the output node and the weights set for the links corresponding to each input node.

[0484] As described above, a neural network has one or more nodes interconnected via one or more links to form input and output node relationships within the neural network. The characteristics of a neural network may be determined by the number of nodes and links within the neural network, the relationship between the nodes and links, and the weights assigned to each link.

[0485] For example, if two neural networks have the same number of nodes and links but different weights between the links, the two neural networks can be recognized as different from each other.

[0486] Some of the nodes that make up the neural network can be organized into a layer based on their distance from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers.

[0487] The distance from the initial input node can be defined as the minimum number of links that must be traversed to reach the node from the initial input node.

[0488] However, this definition of layers is arbitrary for illustrative purposes, and the number of layers within a neural network may be defined in a manner different from that described above. For example, a layer of nodes may be defined by its distance from the final output node.

[0489] The initial input node may refer to one or more nodes in a neural network to which data is directly input without passing through a link in relation to other nodes, or may refer to a node in a neural network that does not have other input nodes connected to it via a link in relation to nodes based on a link.

[0490] Similarly, a final output node may refer to one or more nodes in a neural network that do not have an output node relative to other nodes. A hidden node may refer to a node that constitutes a neural network and is neither the first input node nor the last output node. A neural network according to an embodiment of the present invention may have more nodes in the input layer than in the hidden layer closer to the output layer, and the number of nodes may decrease as one moves from the input layer to the hidden layer.

[0491] A neural network may include one or more hidden layers. Hidden nodes in a hidden layer can receive inputs from the outputs of previous layers and surrounding hidden nodes. The number of hidden nodes in each hidden layer may be the same or different.

[0492] The number of nodes in the input layer may be determined based on the number of data fields in the input data, and may be the same as or different from the number of hidden nodes. The input data input to the input layer can be operated on by the hidden nodes in the hidden layer and output by a fully connected layer (FCL), which is the output layer.

[0493] [Feature extraction model and feature classification model]

[0494] According to one embodiment of the present invention, the sleep analysis model may include a feature extraction model that extracts one or more features for each predetermined epoch, and a feature classification model that classifies each of the features extracted through the feature extraction model into one or more sleep stages to generate sleep stage information.

[0495] According to an embodiment, a feature extraction model can analyze the time series frequency patterns of the spectrogram 300 to extract features related to breathing sounds and patterns.

[0496] In one embodiment, the feature extraction model may be constructed as part of a neural network model (e.g., an autoencoder) that has been pre-trained via a training dataset, where the training dataset may comprise a plurality of spectrograms and a plurality of sleep stage information corresponding to each spectrogram.

[0497] In one embodiment, the feature extraction model may be constructed via a proprietary deep learning model (e.g., an autoencoder) trained via a training dataset. The feature extraction model may be trained via supervised or unsupervised learning methods. The feature extraction model may be trained via the training dataset to output output data similar to the input data.

[0498] In more detail, only the core feature data (or features) of the spectrogram input through the encoder can be learned through the hidden layer, and the remaining information can be lost. In this case, the output data of the hidden layer during the decoding process through the decoder may be an approximation of the input data (i.e., the spectrogram) rather than a perfect copy. In other words, the autoencoder can learn to adjust the weights so that the output data and the input data are as similar as possible.

[0499] Each of the spectrograms included in the training data set may be tagged with sleep stage information, and each of the spectrograms may be input to an encoder, and the output corresponding to each spectrogram may be matched with the tagged sleep stage information and stored.

[0500] Specifically, when a first training data set (i.e., a plurality of spectrograms) tagged with first sleep stage information (e.g., light sleep) is used as input, features associated with the input can be matched and stored with the first sleep stage information. In an embodiment, one or more features associated with the output can be represented in a vector space.

[0501] In this case, the feature data output corresponding to each of the first training data sets may be located relatively close to each other in the vector space because they are output via spectrograms associated with the first sleep stage. That is, the encoder may be trained so that multiple spectrograms output similar features corresponding to each sleep stage.

[0502] In the case of an encoder, it can be trained to extract features that allow a decoder to well restore input data. Therefore, the trained autoencoder of the feature extraction model 410 can be implemented through an encoder to extract features (i.e., multiple features) that allow the input data (i.e., spectrogram) to be well restored.

[0503] The encoder that constructs the feature extraction model through the above-described learning process can extract features corresponding to the spectrogram 300 when it receives the spectrogram 300 (e.g., a spectrogram transformed to correspond to sleep acoustic information) as input.

[0504] In an embodiment, the processor 130 or the processor 830 may extract features by processing the spectrogram 300 generated corresponding to the sleep audio information 210 as an input of a feature extraction model. Here, since the sleep audio information 210 is time-series data acquired in a time-series manner during a user's sleep, the processor 130 or the processor 830 may divide the spectrogram 300 corresponding to the sleep audio information 210 into predetermined epochs. For example, the processor 130 or the processor 830 may divide the spectrogram 300 corresponding to the sleep audio information 210 into 30-second intervals to acquire a plurality of spectrograms. For example, if sleep audio information is acquired during a user's 7-hour (i.e., 420-minute) sleep, the processor 130 or the processor 830 may divide the spectrogram into 30-second intervals to acquire 840 spectrograms.

[0505] Alternatively, in the case of an embodiment such as that shown in Fig. 1(c), at least one of the electronic devices shown in Fig. 1(c) may perform at least one of the above-described operations. The specific numerical values ​​for the sleep time, the division time unit of the spectrogram, and the number of divisions are merely examples, and the present invention is not limited thereto.

[0506] According to an embodiment of the present invention, processor 130 or processor 830 processes each of the divided spectrograms as an input of a feature extraction model to extract a plurality of features corresponding to each of the plurality of spectrograms. For example, if the number of spectrograms is 840, the number of features extracted by the feature extraction model may also be 840. The specific numerical values ​​related to the number of spectrograms and features described above are merely examples, and the present invention is not limited thereto.

[0507] Additionally, processor 130 or processor 830 may process the plurality of features output through the feature extraction model with the input of a feature classification model to obtain sleep stage information. In an embodiment, the feature classification model may be a neural network model modeled to predict sleep stages corresponding to the features.

[0508] For example, the feature classification model may include a fully connected layer and classify features into at least one of the sleep stages. For example, when a first feature corresponding to a first spectrogram is input, the feature classification model may classify the first feature as light sleep.

[0509] The feature classification model may perform multi-epoch classification, which predicts sleep stages of various epochs by inputting spectrograms related to various epochs. The multi-epoch classification does not provide one piece of sleep stage analysis information corresponding to a spectrogram of a single epoch (i.e., one spectrogram corresponding to 30 seconds), but may input spectrograms corresponding to multiple epochs (i.e., a combination of spectrograms each corresponding to 30 seconds) to simultaneously estimate various sleep stages (e.g., changes in sleep stages over time).

[0510] For example, because breathing patterns change more slowly than electroencephalograms or other biological signals, accurate sleep stage estimation may be possible only by observing how the patterns change between past and future points in time. For example, a feature classification model may input 40 spectrograms (e.g., 40 spectrograms, each corresponding to 30 seconds) and perform prediction on the central 20 spectrograms. That is, by closely examining all spectrograms 1 to 40, sleep stages may be predicted through classification corresponding to spectrograms 10 to 20. The specific numerical values ​​for the number of spectrograms described above are merely examples, and the present invention is not limited thereto.

[0511] That is, in the process of estimating sleep stages, rather than performing sleep stage prediction for each single spectrogram, spectrograms corresponding to multiple epochs are used as input so that all information related to the past and future can be taken into consideration, thereby improving the accuracy of the output.

[0512] As described above, the processor 130 or the processor 830 may acquire a spectrogram based on the sleep acoustic information. In this case, the spectrogram may be converted to facilitate analysis of breathing or movement patterns associated with relatively small sounds. The processor 130 or the processor 830 may also generate sleep stage information based on the acquired spectrogram using a sleep analysis model including a feature extraction model and a feature classification model. In this case, the sleep analysis model may input spectrograms corresponding to multiple epochs to perform sleep stage prediction so that information related to both the past and future can be considered, thereby outputting more accurate sleep stage information.

[0513] That is, the processor 130 or the processor 830 may utilize the sleep analysis model as described above to output sleep stage information corresponding to the sleep acoustic information.

[0514] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform the above-described operations.

[0515] According to an embodiment, the sleep stage information may be information related to sleep stages that change during the user's sleep. For example, the sleep stage information may indicate whether the user's sleep changed to light sleep, normal sleep, deep sleep, REM sleep, etc. at each point during the user's eight hours of sleep last night. The specific description of the sleep stage information described above is merely an example, and the present invention is not limited thereto.

[0516] [User privacy protection method]

[0517] FIG. 6(b) is a conceptual diagram illustrating a privacy protection method using mel-spectrogram conversion for sleep acoustic information extracted from a user in the sleep analysis method according to the present invention.

[0518] As shown in Figure 6(b), the audio information extracted from the user or the raw data of sleep audio information extracted from it undergoes a noise reduction preprocessing process, in which noise (e.g., white noise) contained in the raw data is removed.

[0519] The noise reduction process may be performed using algorithms such as spectral gating and spectral subtraction to remove background noise.

[0520] Furthermore, in the present invention, the noise removal process can be performed using a deep learning-based noise reduction algorithm. The deep learning-based noise reduction algorithm can be a noise reduction algorithm specialized for a user's breathing and respiratory sounds, in other words, a noise reduction algorithm learned through a user's breathing and respiratory sounds.

[0521] The raw data from which noise has been removed is then converted into a Mel-Spectrogram, which is a sequence of simplified vectors in the frequency domain for a given input text.

[0522] In this case, a method of generating a mel spectrogram based only on the amplitude obtained by removing the phase from raw data can be used, which not only protects privacy but also reduces data volume and improves processing speed. However, in other embodiments, a mel spectrogram can be generated using both the phase and amplitude.

[0523] In the present invention, a sleep analysis model is generated using the mel spectrogram 300 generated based on the sleep acoustic information 210. If the sleep acoustic information expressed as audio data were used as is, the amount of information would be very large, which would significantly increase the amount of calculation and the calculation time. Furthermore, since unwanted signals would be included, the calculation accuracy would be reduced. Furthermore, if all of the user's audio signals were transmitted to the external server 20 or the AI ​​server 310, there would be a risk of privacy violation.

[0524] In the present invention, after noise is removed from sleep acoustic information using the above-mentioned method, the information is converted into a Mel spectrogram, and the Mel spectrogram is trained to generate a sleep analysis model, thereby reducing the amount of calculation and calculation time, and even protecting personal privacy.

[0525] In this case, the de-identification of sound data may be performed on natural language and respiratory sounds, which may be converted into natural language converted Mel spectrograms and respiratory sound converted Mel spectrograms, respectively. In the sleep analysis according to the present invention, only information required for the analysis model is used, thereby improving the calculation speed and reducing the calculation load.

[0526] [Sleep analysis method accuracy verification and implementation example]

[0527] FIG. 34 is a table verifying the accuracy of the sleep analysis method according to the present invention, showing experimental result data analyzed according to age, sex, BMI, and presence or absence of disease.

[0528] FIG. 34 is a conceptual diagram for easy understanding showing a case where a smart speaker and a smartphone are used as one embodiment of the sleep analysis method according to the present invention.

[0529] Unlike the polysomnography test method used in hospitals, the sleep analysis method according to the present invention allows the lighting to be turned on / off during the test, and the room temperature and humidity to be freely adjusted.

[0530] In other words, sleep analysis can be performed conveniently and flexibly in various non-hospital environments using only the smart home appliance 800 and smartphone 900, which allows for verification in various real-world situations due to the ultra-low latency, going beyond verification in a fixed hospital environment.

[0531] As a result, as shown in Figure 34, experimental results showed consistently high accuracy even for subjects with a wide range of ages, genders, BMIs, sleep apnea, and limb movement disorders.

[0532] 34, for ease of understanding, the smart home appliance 800 is assumed to be a smart speaker 804, but is not limited thereto. That is, the smart home appliance 800 may be embodied as a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop personal computer (PC), a laptop personal computer, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, a wearable device (e.g., smart glasses, a head-mounted device (HMD), electronic clothing, an electronic bracelet, an electronic necklace, an electronic appcessory, an electronic toy, a smart watch), a smart mirror, a kiosk, etc.

[0533] Furthermore, the smart home appliance 800 may be embodied as a smart home appliance such as a TV, a digital video disk (DVD) player, an audio system, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box, a game console, an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame, various medical devices, a home robot, or an internet of things device (e.g., a light bulb, various sensors, an electric or gas meter, a sprinkler system, a fire alarm, a thermostat, a street light, a toaster, exercise equipment, a hot water tank, a heater, a boiler, etc.). Furthermore, the smart home appliance 800 may be embodied as a piece of furniture or part of a building / structure, an electronic board, an electronic signature receiving device, a projector, etc., or may be a combination of one or more of the various devices mentioned above.

[0534] Alternatively, for example, in an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may correspond to one or more combinations of the various devices described above.

[0535] Therefore, the sleep analysis method according to the present invention enables a user's deep sleep to be analyzed conveniently and easily through a smart home appliance 800 such as a smartphone 900 or a smart speaker 804, regardless of time and place, even outside of a hospital.

[0536] [Configuration of non-contact sleep analysis system]

[0537] FIG. 50 is a block diagram illustrating the operation of the AI-based non-contact sleep analysis system according to the present invention, which includes one or more smart home appliances 800, a sleep track app, an autonomous vehicle 801, and a living space 802.

[0538] FIG. 51 is a block diagram illustrating the operation between components of an AI-based non-contact sleep analysis system according to the present invention, and includes a smart home appliance 800, a smartphone 900, and an AI server 310.

[0539] As shown in FIG. 50, the smart home appliance 800 according to the present invention can perform more versatile and precise sleep analysis by acquiring the user's sleep acoustic information using a built-in microphone and performing sleep analysis (non-contact sleep analysis) using the acquired information.

[0540] That is, it can verify various real-world situations beyond the environmental verification such as hospital-based sleep testing, accurately detect not only insomnia but also sleep apnea and sleep hypopnea events in real time, and provide sleep diagnosis solutions for a wide range of ages, genders, races, BMIs, and the presence or absence of diseases.

[0541] 51, a smart home appliance 800 and a smartphone 900 work together to perform sleep analysis of a user. The smart home appliance 800 and the smartphone 900 may be paired via Bluetooth or other wireless communication methods.

[0542] According to the present invention, the smartphone 900 can perform sleep analysis based on the sleep acoustic information of the user acquired from the smart home appliance 800.

[0543] At this time, the user's sleep sound information may be acquired from the smart home appliance 800 and transmitted to the smartphone 900, or may be acquired independently through a microphone built into the smartphone 900.

[0544] 51, the sleep stage analysis is performed in a non-contact manner via the smart home appliance 800 and the smartphone 900. The user can check the sleep stage analysis results derived from the smartphone 900 on the screen of the smartphone 900.

[0545] Thus, even if the user is not wearing the smart home appliance 800, the smart home appliance 800 needs to be appropriately positioned around the user to receive at least a portion of the input signal for the sleep analysis (e.g., body movement information) or the input signal for the sleep analysis (sleep acoustic information).

[0546] In particular, in order to extract information about body movements, it is preferable to place the sensor in an area where it can sense at least the user's movements (e.g., under the pillow, on top of the mattress, etc.).

[0547] On the other hand, since sound is transmitted in a radial direction, when only sleep sound information is used, there is an advantage that information can be collected and analyzed regardless of the user's position or the distance or angle between the user and the smart home appliance 800.

[0548] Therefore, the smart home appliance 800 of the present invention does not necessarily have to be worn by the user, but can perform the sleep stage analysis described above as long as it is properly placed within a predetermined radius (e.g., 4 to 5 m) in the user's sleep space, regardless of the user's position, distance, or angle to the user. Specific numerical values ​​for the radius are merely examples, and the present invention is not limited thereto.

[0549] In one embodiment, when the smart home appliance 800 is not worn by a user, the smart home appliance 800 may transmit a predetermined signal to prompt the user to place the smart home appliance 800 closer to the user so that the smart home appliance 800 can receive an input signal (sleep acoustic information) for sleep analysis. The predetermined signal may be a vibration, an alarm, a text, an LED, or the like.

[0550] The radius between the user and the smart home appliance 800 may be extracted by the smart home appliance 800 or by the smartphone 900.

[0551] That is, since the user's sleeping space is fixed, the location of the smart home appliance 800 can be tracked to determine whether the smart home appliance 800 is placed in an appropriate location.

[0552] Meanwhile, the smart home appliance 800 may be a sleep product (device) that is not a device that can be worn by a user and is used while the user is sleeping.

[0553] For example, a smart speaker 804 may be used as one of the smart home appliances 800. The smart speaker 804 may include an acoustic sensor therein to measure various types of acoustic information.

[0554] The smart speaker 804 may perform primary sleep analysis using acoustic information acquired through the acoustic sensor. The smart speaker 804 may be paired with the smartphone 900, and information measured by the smart speaker 804 or primary sleep analysis results analyzed by the smart speaker 804 may be transmitted to the smartphone 900. In this case, the smart speaker 804 may include a communication module.

[0555] A smart mattress may be used as one of the smart home appliances 800. The smart mattress may include an acoustic sensor therein to measure various acoustic information.

[0556] The smart mattress can perform primary sleep analysis using acoustic information. The smart mattress can be paired with the smartphone 900 and transmit information measured by the smart mattress or primary sleep analysis results analyzed by the smart mattress to the smartphone 900. In this case, the smart mattress may include a communication module.

[0557] Meanwhile, the smart mattress may include various modules for adjusting temperature (temperature adjustment module, infrared irradiation module, cooling module), and the temperature may be adjusted based on the final sleep stage analysis result, which improves the quality of the user's sleep.

[0558] Meanwhile, the smart speaker 804 and smart mattress mentioned above may include a vibration module or an alarm module to alleviate and improve sleep disorders, which will be described later. That is, if sleep apnea, snoring, sleep hyperpnea, REM sleep, etc. are detected, the vibration module or alarm module of the smart speaker 804 or smart mattress may be activated to transmit tactile or auditory stimulation to the user.

[0559] In addition, in an autonomous vehicle 801 or a recently constructed residential space 802, one or more smart devices can be linked to the Sleep Track app to build and operate an AI-based non-contact sleep analysis system according to the present invention.

[0560] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0561] The above-described types of smart home appliances and spaces are merely examples, and the present invention is not limited thereto.

[0562] [Smart home appliances in a sleep analysis system]

[0563] FIG. 11(b) is a block diagram showing the configuration of smart home appliances in the AI-based non-contact sleep analysis system according to the present invention.

[0564] The smart home device 800 according to the present invention includes a communication unit 810, a sensor unit 820, a processor 830, a memory 840, and an alarm unit 850. Various components for performing other functions of the smart home device 800 may also be included.

[0565] That is, depending on the implementation of the embodiments of the present invention, additional components may be included, or some of the components may be omitted, or two or more components may be integrated into one component.

[0566] The communication unit 810 transmits and receives data to and from the smartphone 900 and the AI ​​server 310 via a wireless communication network. The wireless communication network may include short-range wireless communication networks such as Z-wave, Zigbee, Wi-Fi, Bluetooth (BLE), LTE-M, LoRa (long range), Narrowband Internet of Things (NB-IoT), and Infrared Data Association (IrDA). Furthermore, the wireless communication network may include, but is not limited to, a 2G mobile communication network such as a wireless LAN (WLAN), wireless broadband (Wibro), wireless fidelity (Wifi), WiMax (world interoperability for microwave access), a global system for mobile communication (GSM) or code division multiple access (CDMA), a 3G mobile communication network such as a wideband code division multiple access (WCDMA) or CDMA2000, a 3.5G mobile communication network such as a high speed downlink packet access (HSDPA) or high speed uplink packet access (HSUPA), a 4G, 5G, or 6G mobile communication network such as a long term evolution (LTE) network or an LTE-Advanced network.

[0567] According to an embodiment of the present invention, the sensor unit 820 may include a microphone module for extracting sleep acoustic information of the user. The microphone module may be configured with a micro-electromechanical system (MEMS) for application to small devices. Such a microphone module can be manufactured in a very small size and may have a much lower signal-to-noise ratio (SNR) than a condenser microphone or a dynamic microphone.

[0568] In this case, the sleep acoustic information is information on acoustic signals during sleep, and interacts closely with sleep itself, so that it can be acquired without the need to wear a separate wearable device such as a smart watch or smart ring.

[0569] According to one embodiment of the present invention, the sensor unit 820 may include an air pressure sensor, a grip sensor, a color sensor, an IR (infrared) sensor, a temperature sensor, a humidity sensor, and an illuminance sensor.

[0570] According to an embodiment of the present invention, the memory 840 may store a computer program for performing sleep analysis, and the stored computer program may be read and executed by the processor 830, which will be described later. The memory 840 may also store any type of information generated or determined by the processor 830 and any type of information received by the communication unit 810. The memory 840 may also store data related to the user's sleep.

[0571] For example, memory 840 may provide temporary or permanent storage of input / output data.

[0572] According to the present invention, the memory 840 may be embodied as at least one type of storage medium including, but not limited to, a flash memory type, a hard disk type, a multimedia card micro type, a card-type memory (e.g., SD or XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0573] In accordance with the present invention, a computer program may include one or more instructions that, when loaded into memory 840, cause processor 830 to perform methods / operations according to various embodiments of the present invention. That is, processor 830 may perform methods / operations according to various embodiments of the present invention by executing one or more instructions.

[0574] According to one embodiment of the present invention, the processor 830 may be configured with one or more cores and may include a central processing unit (CPU) of a smart home device, a general purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), or other processors for data analysis and deep learning.

[0575] According to an embodiment of the present invention, the processor 830 may read a computer program stored in the memory 840 to perform data processing for machine learning according to an embodiment of the present invention. According to an embodiment of the present invention, the processor 830 may perform calculations for training a neural network.

[0576] According to the present invention, the processor 830 can perform calculations for neural network learning, such as processing input data for learning in deep learning (DL), extracting features from the input data, calculating errors, and updating the weights of the neural network using backpropagation.

[0577] In addition, at least one of the CPU, GPGPU, and TPU of the processor 830 can process the learning of the network function.

[0578] For example, a CPU and a GPGPU can both process network function learning and data classification using the network function. Also, in one embodiment of the present invention, processors of multiple smart home appliances can be used together to process network function learning and data classification using the network function.

[0579] Furthermore, the computer program executed by the smart home device 800 according to an embodiment of the present invention may be a CPU, GPGPU, or TPU executable program.

[0580] In accordance with the present invention, a network function may be used interchangeably with an artificial neural network or neural network. A network function may include one or more neural networks, in which case the output of the network function may be an ensemble of the outputs of the one or more neural networks.

[0581] According to the present invention, a model (inference model) may include a network function. A model may include one or more network functions, in which case the output of the model may be an ensemble of the outputs of the one or more network functions.

[0582] According to the present invention, the processor 830 may provide a sleep analysis model according to an embodiment of the present invention by reading a computer program stored in the memory 840. According to an embodiment of the present invention, the processor 830 may perform sleep analysis of a user based on sleep acoustic information using the sleep analysis model.

[0583] In other words, a user's breathing during sleep contains a lot of information for analyzing sleep, not only about body movements and breathing sounds during sleep, but also about various sleep disorders (e.g., sleep apnea, sleep hypopnea, snoring), etc., so when artificial intelligence (AI) is used, high accuracy can be expected.

[0584] As shown in (b) of Figure 32, during the sleep stage, the user's breathing pattern and regularity, sounds of movement during sleep, and breathing may be measured, and recovery sounds after an apnea event and unstable breath sounds between hypopnea events may be measured.

[0585] Furthermore, if the frequency patterns of breathing sounds are analyzed, it will be possible to make fundamental predictions about the causes of snoring and sleep apnea.

[0586] In particular, sleep breathing sounds are the breathing of a user while sleeping, and as shown in Figure 35, this information can be conveniently measured outside of a hospital via various smart home appliances 800 such as a smartphone 900 or a smart speaker 804.

[0587] According to an embodiment of the present invention, the processor 830 may perform calculations for training a sleep analysis model. Based on the sleep analysis model, sleep information related to the user's sleep stages, sleep quality, occurrence of sleep disorders, etc. may be inferred. Sleep acoustic information acquired from the user in real time or periodically is input as an input value to the sleep analysis model, which outputs data related to the user's sleep (data related to sleep stages, sleep quality, occurrence of sleep disorders, etc.).

[0588] Meanwhile, the smart home appliance 800 according to the present invention may further include an alarm unit 850. The alarm unit 850 is a means for providing tactile or auditory feedback to the user when a sleep disorder such as sleep apnea occurs during primary and secondary sleep analysis.

[0589] For example, the alarm unit 850 may be implemented as an actuator, a vibration module, or a haptic module that generates vibrations, or may be implemented as a speaker module that generates sounds.

[0590] Meanwhile, in the present invention, the sleep state information may be information related to whether or not the user is asleep. Specifically, the sleep state information may include at least one of first sleep state information indicating that the user is before sleeping, second sleep state information indicating that the user is sleeping, and third sleep state information indicating that the user has fallen asleep.

[0591] In other words, if first sleep state information is inferred in relation to a user, processor 830 can determine that the user is in a pre-sleep (i.e., pre-bedtime) state, if second sleep state information is inferred, processor 830 can determine that the user is in a sleeping state, and if third sleep state information is obtained, processor 830 can determine that the user is in a post-sleep (i.e., wake-up) state.

[0592] The sleep state information may be acquired based on environmental sensing information, which may be sensing information acquired in a space where the user is located in a non-contact manner.

[0593] For example, the processor 830 can extract sleep state information based on environmental sensing information acquired by the sensor unit 820 (such as acoustic information related to cleaning, acoustic information related to cooking food, acoustic information related to watching TV, and sleep acoustic information acquired during sleep).

[0594] In this case, the sleep sound information acquired during the user's sleep may include sounds generated when the user turns over in sleep, sounds related to muscle movements, breathing sounds during sleep, etc. That is, the sleep sound information in the present invention may refer to sound information related to the breathing pattern associated with the user's sleep.

[0595] According to the present invention, sleep stages may be classified into NREM (non-REM) sleep and REM (rapid eye movement) sleep, and NREM sleep may be further classified into multiple stages (e.g., two stages of Light and Deep, and four stages of N1 to N4). The sleep stages may be defined based on commonly used sleep stages, or may be arbitrarily set in various ways according to the designer.

[0596] According to one embodiment of the present invention, through the analysis of sleep stages, it is possible to predict not only sleep quality but also sleep disorders (e.g., sleep apnea) and their underlying causes (e.g., snoring).

[0597] According to the present invention, the processor 830 may acquire sleep state information based on the acoustic information acquired from the smart home device 800. Specifically, the processor 830 may identify a singular point where information of a pre-defined pattern is detected in the acoustic information.

[0598] Here, the preset pattern information may be related to sleep-related breathing patterns, for example, in a wakeful state, the entire nervous system is activated, leading to irregular breathing patterns and frequent body movements.

[0599] In addition, the throat muscles do not relax, so breathing noises may be very quiet. On the other hand, when the user is asleep, the autonomic nervous system stabilizes, breathing becomes regular, and breathing noises may become louder.

[0600] That is, the processor 830 may identify a point in time when acoustic information of a predetermined pattern associated with regular breathing, small breathing sounds, etc. is detected as a singular point in the acoustic information. The processor 830 may also acquire sleep acoustic information based on the acoustic information acquired with reference to the identified singular point 201.

[0601] The processor 830 may identify specific points associated with the user's sleep time points from the time-series acquired acoustic information, and acquire sleep acoustic information based on the specific points.

[0602] Furthermore, according to one embodiment of the present invention, for example, in an embodiment such as (c) of FIG. 1, at least one of the electronic devices shown in (c) of FIG. 1 may perform at least one of the above-described operations.

[0603] [Comparison between conventional sleep analysis methods and the sleep analysis method of the present invention]

[0604] FIG. 45 is a conceptual diagram showing a training method using only microphone data S from a multi-dimensional sleep test in a hospital environment according to a conventional sleep analysis method, in order to compare the sleep analysis method of the present invention with the conventional technology.

[0605] FIG. 46 is a conceptual diagram of a method for generating an AI sleep analysis model by incorporating various sounds in a home environment in accordance with the sleep analysis method of the present invention into the training method shown in FIG.

[0606] Here, waveform (a) is the waveform of sleep multivariate test microphone data S in a hospital environment, waveform (b) is the waveform of various noise data N generated in a home environment, and waveform (c) is a waveform that combines waveforms (a) and (b).

[0607] FIG. 47 is a table verifying the performance of the sleep analysis method according to the present invention after training by dividing the subjects into nine groups according to the type of residential noise, and shows experimental result data tested on groups 1 to 9 (group 0 to group 8).

[0608] The types of residential noise are as follows: Group 1 is the sound of rain and wind; Group 2 is the sound of electric fans and air conditioners; Group 3 is the sound of TVs, telephones, and video recorders; Group 4 is the sound of cars, motorcycles, and other vehicles; Group 5 is the sound of clocks; Group 6 is the sound of people talking and voices; Group 7 is the sound of electronic products; Group 8 is noise between rooms / floor areas; and Group 9 is the sound of pets.

[0609] As shown in Figure 45, in a conventional training method using only sleep multivariate test microphone data S in a hospital environment, the sleep multivariate test microphone data S collected at the hospital is input and output through a first AI sleep analysis model, and sleep analysis and diagnosis labels reflecting classification loss are generated and fed back.

[0610] On the other hand, the training method when using home multi-dimensional sleep test microphone data H is as follows.

[0611] First, as shown in FIG. 46, various noise data N generated in a home environment are combined with the sleep polymorphism test microphone data S used in the training method (a) in which only the sleep polymorphism test microphone data S is used in a conventional hospital environment and input.

[0612] If this combined data S+N is input and output through a second AI sleep analysis model, a consistency loss occurs.

[0613] When this consistency loss is added to the classification loss generated by the training method shown in Figure 20, a third AI sleep analysis model is generated.

[0614] At this time, the first and second AI sleep analysis models add correlation between each other's output data.

[0615] [Comparison of user's 24-hour monitoring processor and average results per class]

[0616] FIG. 48 is a schematic diagram illustrating a 24-hour monitoring process of a user using an AI-based non-contact sleep analysis system and a sleep analysis method according to the present invention.

[0617] FIG. 49 is a table showing the mean per class results of comparing the smart home appliances and sleep analysis method according to the present invention with products and devices from existing world-leading sleep tech companies.

[0618] In the past, when analyzing a user's activity, rest, sleep, and other patterns using only a conventional smartwatch, there was a problem that the sleep analysis was interrupted if the smartwatch was removed during sleep. The present invention uses a smartphone 900 linked to a smart home appliance 800 to seamlessly monitor all of a user's activities in real time, even when the smartwatch is removed during sleep.

[0619] For example, by automatically activating the smartphone 900 when the smartwatch is removed, plugged into a charger, placed on a charging pad, etc., it is possible to continuously analyze the user's activity, rest, sleep, etc. In this case, the smartphone 900 can be activated when it is time to sleep even when not adjacent to the smart home appliance 800.

[0620] In this manner, continuity of measurement of the user's activities, including sleep, can be ensured. For example, as shown in Fig. 48, 24-hour data can be obtained via the smartphone 900. The data can then be processed into various reports and provided to the user.

[0621] The user touches the screen of the smartphone 900 to start recording sleep, and receives a sleep analysis result report (such as bedtime, time to fall asleep, sleep duration, and time taken to wake up after the alarm) analyzed in the manner described above. Alarms (such as alarms with gradually increasing volume according to individual sleep stages) may be automatically generated according to the sleep stage, and all-day care services such as user profiling (such as sleep information, preferred content, and content recommendations based on age group / gender / occupation group) and recommendations for customized sleep / exercise / diet / cosmetics / behavioral guidelines optimized for individual sleep patterns may be provided.

[0622] The present invention displays weight / blood pressure and sleep apnea, insomnia, or exercise and insomnia as a sleep measurement record, which can motivate users to change their behavior to improve their health. In other words, the present invention can improve the user's compliance with behavioral changes in a very natural way.

[0623] For example, if a user is overweight, sleep apnea is not uncommon, and weight loss can help reverse sleep apnea, so the present invention can be integrated with diet, exercise, and weight tracking in a healthcare app.

[0624] That is, the sleep apnea history allows for real-time sleep apnea detection and accurate behavioral intervention of the present invention.

[0625] Furthermore, since sleep apnea can cause high blood pressure, if periods of unstable breathing are regular, the present invention can be used to track and manage blood pressure.

[0626] In other words, weight loss helps lower blood pressure in the human body, so when weight loss is successful, the PSQI can be used to objectively compare sleep quality before and after.

[0627] Additionally, exercise (except within three hours before bedtime) helps alleviate insomnia, and outdoor activities increase the amount of time spent in natural light, improving the user's mood.

[0628] It will also be possible to receive recommendations for various exercise programs from the health care app.

[0629] In addition, the present invention can show the user correlations such as stress levels and sleep, or premenstrual syndrome and insomnia, which can help the user to re-evaluate their own health condition.

[0630] In other words, by showing the correlation between stress levels and sleep quality, an interesting element is added, and it becomes possible to create psychiatric questionnaires to be provided in healthcare apps according to the user's stress levels and level of depression.

[0631] In addition, if a user complains of insomnia as one of the symptoms of premenstrual syndrome, the calendar in the menstrual cycle tracking function can be used to compare sleep data and record sleep efficiency, allowing the user to receive a health check related to their physiological phenomena.

[0632] Meanwhile, one of the important aspects of sleep stage analysis is to determine whether a user wakes up during sleep or whether a true wake-up has occurred. In other words, the WAKE stage, which is the stage at which a user wakes up, must be analyzed properly, and sleep sound signals are a very useful factor in detecting whether the user is in the true WAKE stage.

[0633] While conventional EEG measurement in sleep multidimensional analysis merely confirms changes in EEG when the user is awake, the sleep audio signal used in the sleep stage analysis of the present invention shows precursor signals (sound patterns, movement patterns, etc.) before the user wakes up (before reaching the WAKE stage), and through this, the WAKE stage can be predicted and detected.

[0634] The AI ​​sleep stage analysis model, trained through a large amount of data, will be able to more accurately determine the wake stage, especially based on sleep sound signals. Furthermore, whether a user wakes up from sleep is influenced by their body's biorhythm or external factors (such as ambient noise), the model can also be used to determine the wake stage.

[0635] The present invention builds an AI sleep stage analysis model by learning the user's sleep environment, i.e., various ambient noises such as routine noises and abnormal or intermittent noises in the surrounding space, thereby enabling more clear and reliable prediction and detection of the wake stage.

[0636] In fact, as shown in Figure 49, when compared with solutions from existing world-leading sleep tech companies, the wake accuracy was calculated to be 43% higher than existing wearables and 52% higher than existing contactless systems, and the average wake / sleep accuracy was calculated to be 16% higher than existing wearables and 20% higher than existing contactless systems.

[0637] In addition, the average accuracy of Wake / NREM / REM (3C) was calculated to be 15% higher than existing wearable devices and 25% higher than existing contactless devices.

[0638] In addition, the sleep analysis of the present invention using sleep acoustic information is highly versatile and can be applied to a variety of devices, as anyone can perform sleep analysis as long as they have a device including a microphone.

[0639] The sleep analysis method, sleep disorder alleviation and prevention method, sleep disorder improvement method, and sleep monitoring method according to the present invention may be provided by a server that provides a cloud computing service. More specifically, the sleep analysis method, sleep disorder alleviation and prevention method, sleep disorder improvement method, and sleep monitoring method according to the present invention may be performed by a server that provides a cloud computing service, which is a type of Internet-based computing and processes information on a computer connected to the Internet other than the user's computer.

[0640] That is, in the embodiment shown in Figures 50 and 51, various sleep acoustic information acquired by the smart home appliance 800 and the smartphone 900 is transmitted to the AI ​​server 310, and the AI ​​server 310 performs sleep analysis using the information and then transmits the results back to the smart home appliance 800 and the smartphone 900.

[0641] According to another embodiment of the present invention, various sleep acoustic information acquired by the smartphone 900 may be converted into a spectrogram by the smartphone 900 and transmitted to the AI ​​server 300. In this case, the AI ​​server 310 may perform sleep analysis using the spectrogram.

[0642] According to yet another embodiment of the present invention, various sleep acoustic information acquired by the smartphone 900 is converted into a spectrogram by the AI ​​server 310, and the AI ​​server 310 can perform sleep analysis using the spectrogram.

[0643] A cloud computing service may be a service that stores materials on the Internet and allows users to access them anytime and anywhere via an Internet connection without having to install the necessary materials or programs on their own computers, and allows materials stored on the Internet to be easily shared and transmitted with simple operations and clicks. A cloud computing service may not only simply store materials on an Internet server, but also allow users to perform desired tasks using application functions provided on the web without installing additional programs, and may allow various people to share documents and work on them simultaneously.

[0644] In addition, the cloud computing service may be implemented in at least one form of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, or a container-based cloud server. That is, the smart home appliance 800 of the present invention may be implemented in at least one form of the above-mentioned cloud computing services. The specific descriptions of the above-mentioned cloud computing services are merely examples and may include any platform for building a cloud computing environment of the present invention.

[0645] The sleep analysis method, sleep disorder alleviation and prevention method, sleep disorder improvement method, and monitoring method according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, singly or in combination.

[0646] The program instructions recorded on the medium may be those specially designed and constructed for the present invention, or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc.

[0647] Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code that may be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0648] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0649] [Generation of environmental creation information]

[0650] According to an embodiment of the present invention, the processor 130 or the processor 830 may generate environment creation information based on the sleep state information and / or the sleep stage information.

[0651] The sleep state information is information related to whether the user is sleeping or not, and may include at least one of first sleep state information indicating that the user is about to fall asleep, second sleep state information indicating that the user is sleeping, and third sleep state information indicating that the user has fallen asleep. Hereinafter, the step of generating environment creation information will be described in detail using the processor 130 as an example.

[0652] According to the embodiment, the processor 130 may generate the first environment creation information based on the first sleeping state information. Specifically, when the processor 130 acquires the first sleeping state information indicating that the user is before sleeping, the processor 130 may generate the first environment creation information based on the first sleeping state information.

[0653] According to an embodiment, the first environment creation information may be information regarding light intensity and illuminance for inducing natural sleep. Specifically, the first environment creation information may be control information for supplying 3000K white light at an illuminance of 30 lux from a sleep induction point until the second sleep state information is acquired.

[0654] According to an embodiment, the sleep induction time point may be determined by the processor 130. Specifically, the processor 130 may determine the sleep induction time point through information exchange with the user's user terminal 10. For a specific example, the user may set the time point at which they intend to fall asleep via the user terminal 10 and transmit the time point to the processor 130. The processor 130 may determine the sleep induction time point based on the time point at which the user intends to fall asleep from the user terminal 10. For example, the processor 130 may determine the sleep induction time point to be 20 minutes before the time point at which the user intends to fall asleep. For a specific example, if the time point at which the user intends to fall asleep set by the user is 11:00, the processor 130 may determine 10:40 as the sleep induction time point. The specific numerical values ​​for the above-mentioned time points are merely examples, and the present invention is not limited thereto.

[0655] According to an embodiment, the processor 130 may acquire the user's sleep intention information based on the environmental sensing information and determine a sleep induction time point based on the sleep intention information. The sleep intention information may be information that quantitatively indicates the user's intention to sleep. For example, the higher the user's sleep intention, the closer to 10 the calculated sleep intention information may be, and the lower the user's sleep intention, the closer to 0 the calculated sleep intention information may be.

[0656] Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may perform at least one of the above-described operations.

[0657] The specific numerical values ​​for the sleep intention information described above are merely examples, and the present invention is not limited thereto.

[0658] [Acquisition of sleep intention information]

[0659] According to an embodiment of the present invention, processor 130 or processor 830 may acquire sleep intention information based on environmental sensing information. Alternatively, in the case of an embodiment such as that shown in FIG. 1(c), at least one of the electronic devices shown in FIG. 1(c) may acquire the sleep intention information. Hereinafter, the step of acquiring sleep intention information will be described in detail using processor 130 as an example.

[0660] According to an embodiment, processor 130 may identify types of sounds included in the environmental sensing information. Processor 130 may calculate sleep intention information based on the number of identified types of sounds. Processor 130 may calculate a lower sleep intention information as the number of types of sounds increases, and may calculate a higher sleep intention information as the number of types of sounds decreases. For example, if there are three types of sounds included in the environmental sensing information (e.g., the sound of a vacuum cleaner, the sound of a TV, and a user's voice), processor 130 may calculate the sleep intention information as 2 points. Also, if there is one type of sound included in the environmental sensing information (e.g., a washing machine), processor 130 may calculate the sleep intention information as 6 points. The specific numerical values ​​related to the types of sounds included in the environmental sensing information and the sleep intention information described above are merely examples, and the present invention is not limited thereto.

[0661] That is, the processor 130 may acquire sleep intention information related to the user's intention to sleep according to the number of types of sounds included in the environmental sensing information. For example, the more types of sounds are identified, the lower the sleep intention information indicating the user's low sleep intention (i.e., sleep intention information with a low score) may be output.

[0662] In addition, in an embodiment, the processor 130 may generate or record an intention score table by pre-matching different intention scores for each of a plurality of pieces of sound information. For example, first sound information related to a washing machine may be pre-matched with an intention score of 2, second sound information related to a humidifier may be pre-matched with an intention score of 5, and third sound information related to sound may be pre-matched with an intention score of 1. The processor 130 may pre-match relatively high intention scores for sound information related to the user's sleep (e.g., sounds generated by the user's activities, such as vacuuming, washing dishes, and voices) and relatively low intention scores for sound information unrelated to the user's sleep (e.g., sounds unrelated to the user's activities, such as vehicle noise and the sound of falling rain) to generate the intention score table. The specific numerical values ​​for the intention scores matched to each piece of sound information described above are merely examples, and the present invention is not limited thereto.

[0663] The processor 130 may acquire sleep intention information based on the environmental sensing information and the intention score table. Specifically, the processor 130 may record an intention score corresponding to a time point at which at least one of a plurality of sounds included in the intention score table is identified in the environmental sensing information, and the intention score may be matched to the identified sound. For example, if the sound of a vacuum cleaner is identified corresponding to a first time point in the process of acquiring the environmental sensing information in real time, the processor 130 may match two intention scores to the sound of the vacuum cleaner and record the matched score at the first time point. Each time various sounds are identified in the process of acquiring the environmental sensing information, the processor 130 may match the intention score to the identified sound and record the matched score at the corresponding time point.

[0664] In an embodiment, the processor 130 may acquire sleep intention information based on the sum of intention scores acquired during a predetermined time period (e.g., 10 minutes). For example, the higher the intention score acquired during the 10 minutes, the higher the sleep intention information may be acquired, and the lower the intention score acquired during the 10 minutes, the lower the sleep intention information may be acquired. The specific numerical values ​​for the predetermined time period described above are merely examples, and the present invention is not limited thereto.

[0665] That is, the processor 130 may acquire sleep intention information related to the user's intention to sleep according to the characteristics of the sound included in the environmental sensing information. For example, the more sounds related to the user's activity are identified, the lower the sleep intention information indicating the user's low sleep intention (i.e., sleep intention information with a low score) may be output.

[0666] [Determining environmental creation information and operating smart home appliances]

[0667] According to an embodiment of the present invention, the processor 130 or the processor 830 may determine the environment creation information based on the sleep state information and / or the sleep intention information.

[0668] In addition, various smart home appliances 800 according to embodiments of the present invention can be operated based on the environment creation information.

[0669] Alternatively, in the case of an embodiment such as that shown in Fig. 1(c), at least one of the electronic devices shown in Fig. 1(c) may perform at least one of the above-described operations. Hereinafter, the determination of environment creation information and the operation of smart home appliances will be described in detail with reference to the drawings.

[0670] [Overall behavior]

[0671] FIG. 8 illustrates an exemplary flow chart for providing a method for creating a sleep environment based on sleep state information according to an embodiment of the present invention.

[0672] According to one embodiment of the present invention, the method may include a step of acquiring sleep state information of a user (S100).

[0673] According to an embodiment of the present invention, the method may include generating environment creation information based on sleep state information (S200).

[0674] According to an embodiment of the present invention, the method may include a step of transmitting environment creation information to the environment creation device 30 (S300).

[0675] 8 may be reordered, and at least one step may be omitted or added, as necessary. That is, the above-described steps are merely one embodiment of the present invention, and the scope of the present invention is not limited thereto.

[0676] FIG. 39 is a flowchart illustrating the operation of the AI-based non-contact sleep analysis method according to the present invention.

[0677] FIG. 40 is a flow chart showing various embodiments of smart home appliances that can be used in the sleep analysis method according to the present invention.

[0678] The overall operation of the AI-based non-contact sleep analysis method according to the present invention will be briefly described below with reference to FIGS. 50, 51, and 39.

[0679] A sleep analysis app can be downloaded to the smartphone 900 (S1000).

[0680] At least one smart home appliance 800 can collect the user's sleep sound information in real time and transmit it to the server 310 (S2000).

[0681] The smartphone 900 can simultaneously collect the user's sleep acoustic information in real time and transmit it to the server 310 (S3000).

[0682] The server 310 may transmit a sleep analysis result report learned by AI to the smartphone 900 (S4000).

[0683] The smartphone 900 may output a control signal for controlling the operation of at least one smart home appliance 800 (S5000).

[0684] At least one smart home appliance 800 can provide a customized sleep environment to the user (S6000).

[0685] Next, the detailed operation of the AI-based non-contact sleep analysis method according to the present invention will be described with reference to FIGS. 50, 51, and 40.

[0686] First, it can be determined whether the smart home appliance 800 has a built-in microphone (S7000).

[0687] If the answer is yes, the sleep analysis app (hereinafter referred to as the sleeptrack app) according to the present invention can be downloaded to the smartphone 900 (S7100), and if no, the sleeptrack app can be linked to apps already installed on the smartphone 900 (S7200).

[0688] Here are the features of the Sleep Track app:

[0689] As a sleep analysis app that detects users' real-time sleep stages and breathing instability periods, it uses a database that can store weekly and monthly sleep quality indicators and sleep environment, and a dashboard that can derive service insights through usage sessions, sleep statistics, etc., to calculate a highly accurate overnight sleep stage graph (hypnogram), sleep evaluation indicators, and breathing instability indicators.

[0690] In addition, the SleepTrack app allows for seamless monitoring and data collection between daily life and sleep in a contactless manner without the need for a separate wearable device.

[0691] This not only increases the degree of freedom of the body during sleep, but also allows for accurate timing of the wake time, which is the basis of all sleep therapy, and allows for convenient and accurate analysis of various types of user sleep at home, regardless of time or place.

[0692] The Sleep Track app can be used for the following purposes:

[0693] It not only provides an individual sleep pattern analysis report for each user to create an optimal sleeping environment based on the sleep analysis results by intervening in the user's sleep based on real-time sleep tracking, but also provides alarms, sleep hygiene guides, and sound content for falling asleep / waking up tailored to individual sleep stages.

[0694] In addition, it can recommend content that can help create behavioral correction and sleep routines, such as customized exercise and eating patterns, optimized for individual sleep patterns, as well as user profiles such as user-specific sleep information, preferred content, sleep BTI, and responsiveness to recommended content, according to age group, gender, and occupation.

[0695] Meanwhile, in step S7100, it is determined whether the smart home appliance can create a sleep environment (S8000). Here, the sleep environment may include temperature, humidity, light, sound, head and body position, scent, etc.

[0696] In step (S8000), if the answer is yes, the SleepTrack app may be activated and a research interaction may be generated (S810), and if no, it may be determined whether the device is capable of providing customer value, i.e., data, based on sleep analysis via various user interfaces (e.g., PUI, VUI, and / or GUI) (S9000).

[0697] If the result of step S9000 is affirmative, the sleep track application is activated (S9100), and if the result is negative, the operation may be terminated since the introduction of the sleep track application is meaningless.

[0698] For example, smart home appliances that reach step (S8100) may include air conditioners and / or air purifiers that adjust temperature, humidifiers and / or dehumidifiers that adjust humidity, blinds and / or curtains that adjust light, smart speakers 804 that adjust lights and sounds, smart beds that adjust the position of the user's head and body, smart diffusers that adjust fragrances, smart devices with healthcare apps installed, etc.

[0699] In addition, smart home appliances that reach step (S9100) may include TVs, clothing management machines, robot vacuum cleaners, washing machines and / or dryers, refrigerators, smart appliances with healthcare apps installed, etc.

[0700] In addition, application fields that can reach "sleep management app interaction" other than step (S8100) and step (S9100) may be industries related to fragrance, cosmetics, health functions, traditional sleep industry, sports, hotels, cram schools, fire departments, and government agencies.

[0701] At...

Claims

1. 1. A method for controlling an environment creation device, comprising: an acquisition stage for acquiring environmental sensing information; a preprocessing step of performing preprocessing on the acquired environmental sensing information; generating sleep state information based on the preprocessed environmental sensing information; and controlling the environment creating device based on the generated sleep state information. A method for controlling an environment creation device.

2. In the control step, controlling the environment creating device in real time based on the generated sleep state information; A method for controlling the environment creation device of claim 1.

3. The generating step comprises: further comprising converting the environmental sensing information into information containing a change in frequency component of the environmental sensing information over time. A method for controlling the environment creation device of claim 1.

4. The control step comprises: generating first environment creation information based on the generated sleep state information; causing the environment creation device to create an environment based on the first environment creation information; generating second environment creation information based on the generated sleep state information of the user after the environment creation device starts creating the environment based on the first environment creation information; and causing the environment creation device to create an environment based on the generated second environment creation information. A method for controlling the environment creation device of claim 2.

5. The step of generating the first environment creation information includes: generating the first environment creation information based on the sleep state information generated during a time period corresponding to one or more epochs; The step of generating the second environment creation information includes: generating the second environment creation information based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating the environment based on the first environment creation information; A method for controlling the environment creation device of claim 4.

6. In an electronic device for controlling an environment creation device, a sensor for acquiring environmental sensing information; a control unit that performs an operation of performing preprocessing on the acquired environment sensing information, an operation of generating sleep state information based on the preprocessed environment sensing information, and an operation of controlling the environment creating device based on the generated sleep state information; Including, An electronic device that controls the environmental creation device.

7. The control unit performing an operation of controlling the environment creating device in real time based on the generated sleep state information; An electronic device for controlling the environment creation device according to claim 6.

8. The control unit converting the environmental sensing information into information including a change in frequency component of the environmental sensing information over time; An electronic device for controlling the environment creation device according to claim 6.

9. The control unit generates first environment creation information based on the generated sleep state information, performing an operation of controlling the environment creation device based on the generated first environment creation information; After the environment creation device starts creating an environment based on the first environment creation information, if the second environment creation information is generated based on the generated sleeping state information of the user, performing an operation of controlling the environment creation device based on the generated second environment creation information; An electronic device for controlling the environment creation device according to claim 7.

10. The generated first environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs; The generated second environment creation information is The sleep state information is generated based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating the environment based on the first environment creation information. An electronic device for controlling the environment creation device according to claim 9.

11. In the environmental creation system, an electronic device including a sensor for acquiring environmental sensing information, a control unit, and a communication unit for transmitting and receiving information via a network; a server that performs an operation of generating sleep state information based on environmental sensing information; Environment creation device and Including, the control unit performs an operation of performing preprocessing on the acquired environmental sensing information and an operation of transmitting the preprocessed environmental sensing information to the server via the communication unit; The server performs an operation of generating sleep state information based on the pre-processed environmental sensing information received from the electronic device; The control unit receives the generated sleep state information from the server via the communication unit, and controls the environment creating device based on the received sleep state information. Environmental creation system.

12. The control unit performing an operation of controlling the environment creating device in real time based on the received sleep state information; The environment creation system according to claim 11.

13. The control unit receiving information for converting the environmental sensing information into information including a change in frequency component of the environmental sensing information over time from the server via the communication unit; The environment creation system according to claim 11.

14. the control unit controls the environment creation device based on first environment creation information generated based on the sleep state information received through the communication unit; After the environment creation device starts creating an environment based on the first environment creation information, generating second environment creation information based on the user's sleep state information received through the communication unit; performing an operation of controlling the environment creation device based on the generated second environment creation information; The environment creation system according to claim 12.

15. The generated first environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs; The generated second environment creation information is The sleep state information is generated based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating the environment based on the first environment creation information. The environment creation system according to claim 14.

16. In the environmental creation system, an electronic device including a sensor for acquiring environmental sensing information, a control unit, and a communication unit for transmitting and receiving information via a network; a server that performs an operation of generating sleep state information based on the environmental sensing information and an operation of generating environment creation information based on the sleep state information; an environment creation device controlled based on the environment creation information; the control unit performs an operation of performing preprocessing on the acquired environmental sensing information and an operation of transmitting the preprocessed environmental sensing information to the server via the communication unit; the server performs an operation of generating sleep state information based on the pre-processed environment sensing information received from the electronic device, an operation of generating environment creation information for controlling the environment creation device based on the generated sleep state information, and an operation of transmitting the generated environment creation information to the environment creation device. Environmental creation system.

17. The server a first server and a second server; the first server performs an operation of generating sleep state information based on the pre-processed environmental sensing information received from the electronic device; The second server generates environment creation information for controlling the environment creation device based on the generated sleep state information. The environment creation system according to claim 16.

18. The server performing an operation of controlling the environment creating device in real time based on the generated sleep state information; The environment creation system according to claim 16.

19. The server converting the environmental sensing information received from the electronic device into information including a change in frequency component of the environmental sensing information over time; The environment creation system according to claim 16.

20. The server performing an operation of controlling the environment creation device based on first environment creation information generated based on the generated sleep state information; After the environment creation device starts creating an environment based on the first environment creation information, generating second environment creation information based on the generated sleep state information; performing an operation of controlling the environment creation device based on the generated second environment creation information; The environment creation system according to claim 17.

21. The generated first environment creation information is generated based on the sleep state information generated during a time period corresponding to one or more epochs; The generated second environment creation information is The sleep state information is generated based on the sleep state information generated during a time period corresponding to one or more epochs after the environment creation device starts creating the environment based on the first environment creation information.

20. The environment creation system according to claim 19.

Images