Method for providing a graphic user interface that displays information or evaluations related to a user's sleep.

Abstract

The present invention relates to a method for generating a graphic user interface that shows an evaluation of a user's sleep, comprising: a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices - the sleep information includes the user's sleep acoustic information; a sleep phrase generation step of generating a sleep phrase that includes at least two words that show an evaluation of the user's sleep based on the acquired sleep information; and a step of displaying a graphic user interface that includes the generated phrase, wherein the method for generating one or more graphic user interfaces that show an evaluation of a user's sleep is as described above.

Description

Technical Field

[0007]

[0001] The present invention relates to a method for providing a graphic user interface that shows information or evaluation related to a user's sleep.

Background Art

[0002] There are various methods for maintaining and improving health, such as exercise and dietary therapy. However, it is most important to manage sleep well, which occupies about 30% or more of a day.

[0003] However, despite the simple labor substitution of machines and the leisure in life, modern people cannot get sound sleep due to irregular eating habits, lifestyle habits, and stress, and are suffering from sleep disorders such as insomnia, hypersomnia, sleep apnea syndrome, nightmares, night terrors, sleepwalking, etc.

[0004] According to the National Health Insurance Service, it is shown that the number of domestic sleep disorder patients has been increasing by about 8% annually from 2014 to 2018, and the number of patients who received medical treatment for sleep disorders in 2018 reached about 570,000 in the country.

[0005] Although sound sleep is recognized as an important factor affecting physical or mental health and the interest in sound sleep is increasing, in order to improve sleep disorders, it is necessary to directly visit a specialized medical institution, separate examination fees are required, and continuous management is difficult, so the efforts of users for treatment are insufficient.

[0006] Korean Patent Publication No. 10-2023-0012133 discloses an electronic device that provides a user interface according to a sleep state and an operating method thereof. It is disclosed that it can detect abnormal REM sleep events and provide a user interface based on action information specified in response thereto.

[0007] However, conventional technology detects abnormal REM sleep stages by looking at the overall sleep stage, which may lead to lower accuracy in measuring those sleep stages and a failure to detect sleep-related events in real time. Furthermore, it has the limitation of not providing general consumers, who are not sleep experts, with an intuitive assessment of their own sleep quality.

[0008] As a result, research has recently been progressing on methods for general users to evaluate the quality of their own sleep through sleep stage analysis, and on detecting sleep disorders that occur during sleep.

[0009] Korean Patent Publication No. 10-2023-0012133 discloses an electronic device and method of operation that provides a user interface corresponding to sleep states. It discloses that the device can detect abnormal REM sleep events and provide a user interface based on action information specified in response to them.

[0010] However, conventional technology detects abnormal REM sleep stages by looking at the overall sleep stage, which may lead to low accuracy in measuring those sleep stages. Furthermore, since the basic unit of measurement consists of several minutes, there is a risk that sleep-related events may not be detected in real time.

[0011] On the other hand, Figures 31a to 31e are diagrams showing hypnograms of sleep stage information represented by a conventional sleep measurement interface.

[0012] However, conventional technology, when representing graphs showing sleep stages, does not represent the shapes assigned to the first and second sleep stages discretely but rather continuously. This can lead to the misconception that one must pass through other sleep stages in the process of moving from the first to the second sleep stage.

[0013] Furthermore, as shown in Figure 31c or Figure 31d, even if the figures representing the wakefulness stage were displayed separately from the figures representing other sleep stages, the continuous display of figures corresponding to other sleep stages in the areas allocated to other sleep stages made it difficult to clearly understand which sleep stage was occurring at that particular time.

[0014] As a result, research is currently underway to intuitively represent sleep state information, including sleep stages, through graphs. [Overview of the project] [Means for solving the problem]

[0015] In one embodiment of the present invention to solve the above-mentioned problems, a method is disclosed for providing a graphic user interface that indicates the date and / or time when information regarding the user's sleep was acquired.

[0016] A method for providing a graphic user interface according to one embodiment of the present invention may include a sleep information acquisition step of acquiring user sleep information from one or more sleep information sensor devices - said user sleep information includes user sleep acoustic information.

[0017] A method for providing a graphic user interface according to one embodiment of the present invention is:

[0018] The process may include a step to acquire sleep state information of the user based on the acquired sleep information of the user - the user's sleep state information includes at least one of the following: information on when the user falls asleep, information on the duration of sleep, or information on when the user wakes up.

[0019] A method for providing a graphic user interface according to one embodiment of the present invention may include a step of generating information indicating the time when the sleep information was acquired, based on at least one of the following: information on when the user fell asleep, information on the duration of sleep, or information on when the user woke up, which is included in the acquired sleep state information of the user.

[0020] A method for providing a graphic user interface according to one embodiment of the present invention may include the step of displaying a graphic user interface that includes information indicating the time of acquisition.

[0021] The acquisition time display information generation step according to one embodiment of the present invention is characterized by generating information that displays the time when the sleep information was acquired, based on the wake-up time information included in the sleep state information.

[0022] Furthermore, the acquisition time display information generation step according to one embodiment of the present invention is characterized in that information displaying the time when the sleep state information displayed on the graphic user interface was acquired is generated using a figure that displays the date of the wake-up time included in the wake-up time information, and the figure is one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of straight lines and curves.

[0023] Furthermore, in the acquisition time display information generation step according to one embodiment of the present invention, if the same date contains N wake-up times (where N is 0 or a natural number), the information displaying the date to be shown in the graphic user interface can be generated to include N shapes.

[0024] The acquisition time display information generation step according to one embodiment of the present invention is characterized by generating information that displays the time when sleep information was acquired based on the sleep onset time information included in the sleep state information.

[0025] Furthermore, the acquisition time display information generation step according to one embodiment of the present invention generates information to display the time when the sleep state information displayed on the graphic user interface was acquired, using a figure that displays the date of the time of falling asleep included in the sleep onset time information, and the figure is characterized in that it is one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure made up of a combination of straight lines and curves.

[0026] Also, in the acquisition time display information generation stage according to an embodiment of the present invention, when N (N is 0 or a natural number) wake-up times are included on the same date, information for displaying the date displayed on the graphic user interface can be generated to include N figures.

[0027] The acquisition time display information generation stage according to an embodiment of the present invention can be characterized by generating information for displaying the time when sleep information is acquired based on the bedtime information and wake-up time information included in the sleep state information.

[0028] Also, in the acquisition time display information generation stage according to an embodiment of the present invention, when the date of the bedtime included in the bedtime information and the date of the wake-up time included in the wake-up time information are different from each other, information for displaying the time when the sleep state information displayed on the graphic user interface is acquired is generated as a figure having a continuous form that displays from the date of the bedtime to the date of the wake-up time, and the figure can be characterized as any one of figures composed of a point, a polygon, a circle, an ellipse, a sector, or a combination of a straight line and a curve.

[0029] Alternatively, in the acquisition time display information generation stage according to an embodiment of the present invention, when the date of the bedtime included in the bedtime information and the date of the wake-up time included in the wake-up time information are the same as each other, information for displaying the date displayed on the graphic user interface is generated as a figure having a continuous form that connects two different points, and the figure can be characterized as any one of figures composed of a point, a polygon, a circle, an ellipse, a sector, or a combination of a straight line and a curve.

[0030] The acquisition time display information generation stage according to an embodiment of the present invention generates information for displaying the time when the sleep information is acquired based on the wake-up time information included in the sleep state information, the information for displaying the time when the sleep information is acquired includes time zone information, and the time zone information may be generated so that the time zone to which the wake-up time belongs is displayed based on the definition of a preset time zone.

[0031] Furthermore, in the stage of generating information displaying the time of acquisition according to one embodiment of the present invention, the definition of the already set time period which is the basis for determining the time period to which the wake-up time belongs may be defined as follows: if it is between 12:00 a.m. (midnight) and before 5:00 a.m., it is the early morning period; if it is between 5:00 a.m. and before 9:00 a.m., it is the morning period; if it is between 9:00 a.m. and before 5:00 p.m., it is the daytime period; if it is between 5:00 p.m. and before 9:00 p.m., it is the evening period; and if it is between 9:00 p.m. and before 12:00 a.m. (midnight), it is the nighttime period.

[0032] Furthermore, the acquisition time display information generation step according to one embodiment of the present invention generates information to display the time when the sleep state information displayed on the graphic user interface was acquired, using a figure that displays the date and time of the wake-up time information included in the wake-up time information, and the figure is characterized in that it is one of a point, polygon, circle, ellipse, sector, or a figure consisting of a combination of straight lines and curves.

[0033] Furthermore, the graphic generated by the embodiment of the present invention may be characterized in that at least one of its types or colors is displayed differently depending on the time period information included in the wake-up time information.

[0034] Furthermore, a graphic user interface according to one embodiment of the present invention further includes text for displaying the time of day information, wherein the text for displaying the time of day information includes the keyword "dawn" if the time of day to which the time of waking belongs is the early morning time, the keyword "morning" if it is the morning time, the keyword "daytime" if it is the daytime time, the keyword "evening" if it is the evening time, and the keyword "night" if it is the nighttime time.

[0035] The sleep state information acquisition step according to one embodiment of the present invention may include the step of converting the sleep acoustic information contained in the acquired sleep information into information that includes the time axis changes of the frequency components of the sleep acoustic information, and performing an analysis on the converted information.

[0036] Furthermore, in the sleep state information acquisition stage according to one embodiment of the present invention, the converted information may be a spectrogram.

[0037] Furthermore, the sleep state information acquisition step according to one embodiment of the present invention is characterized by acquiring the sleep state information based on dividing the spectrogram into 30-second epochs.

[0038] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, comprising: a sleep information receiving step of receiving user sleep information, including user sleep acoustic information, from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring user sleep state information based on the received user sleep information; a step of generating one or more graphic user interfaces for receiving desired wake-up time information; a step of receiving desired wake-up time information via one or more graphic user interfaces for receiving desired wake-up time information; an alarm time information generation step of generating alarm time information based on the acquired sleep state information and the received desired wake-up time information; a graphic user interface generation step of generating one or more graphic user interfaces for performing an alarm function based on the generated alarm time information; and a display step of displaying the generated graphic user interfaces via a display device.

[0039] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, where, in the sleep state information acquisition stage, if the sleep state information indicates that the user is in REM sleep within a time range including the desired wake-up time information, then, after a predetermined time after REM sleep is indicated, the alarm time information generation stage generates alarm time information based on the sleep state information.

[0040] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes a graphic user interface generation step that includes a trigger area for activating the alarm function via an alarm function activation trigger method.

[0041] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes a graphic user interface generation step for starting sleep measurement together with the activation of the alarm function when a trigger area for activating the notification function is activated, and a graphic user interface generation step for starting sleep measurement together with the deactivation of the alarm function when the trigger area for activating the notification function is not activated.

[0042] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, the graphic user interface generation step of which, if the trigger area for activating the notification function is not activated, provides information corresponding to sleep where the wake-up time is not determined.

[0043] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes a graphic user interface generation step that includes a slide area for selecting the desired wake-up time information via a slide method, and the step for receiving the desired wake-up time information further includes a step for receiving the desired wake-up time information via the slide area.

[0044] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, comprising a graphic user interface generation step including a slide area for selecting the desired wake-up time information, wherein when the slide area is slid in a first direction, the time of the desired wake-up time information is delayed, and when the slide area is slid in a second direction, the time of the desired wake-up time information is advanced.

[0045] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, the graphic user interface generation step of which includes a graphic user interface generation step that indicates a range of scheduled wake-up times based on the received desired wake-up time information, and the range of scheduled wake-up times indicates the period from a predetermined time before the desired wake-up time to the desired wake-up time.

[0046] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes, upon receiving the desired wake-up time information via the slide area, a graphic user interface generation step that displays the predicted sleep time based on one of the following: predicted wake-up time information inferred from the received user sleep state information, predicted sleep time, or a combination of predicted sleep time and sleep efficiency.

[0047] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes a step of generating a graphic user interface that, upon receiving the desired wake-up time information via the slide area, displays an estimated sleep time based on the difference between the received desired wake-up time and the current time.

[0048] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, wherein the graphic user interface generation step includes a step of generating a graphic user interface that includes a screen indicating that sleep measurement is in progress, upon receiving input to start sleep measurement via a graphic user interface for activating the alarm function and starting sleep measurement.

[0049] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, comprising the steps of generating a graphic user interface that includes a screen indicating that sleep measurement is in progress, a graphic user interface generation step indicating that the alarm function is activated, a graphic user interface generation step indicating the range of the scheduled wake-up time, and a graphic user interface generation step indicating the wave patterns of the user's sleep acoustic information.

[0050] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, further comprising the steps of generating a graphic user interface that includes a screen indicating that sleep measurement is in progress, and generating a graphic user interface that allows the user to terminate sleep measurement.

[0051] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, wherein the graphic user interface generation step includes a step of generating a graphic user interface that includes a screen indicating that sleep measurement is in progress, if the trigger area for activating the notification function is not activated and an input to start sleep measurement is received via the graphic user interface for starting sleep measurement.

[0052] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, comprising the steps of generating a graphic user interface including a screen indicating that sleep measurement is in progress, a graphic user interface generation step indicating that the trigger area for activating the alarm function is not activated, a graphic user interface generation step indicating the range of the scheduled wake-up time, and a graphic user interface generation step indicating the wave patterns of the user's sleep acoustic information.

[0053] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, further comprising the steps of generating a graphic user interface that includes a screen indicating that sleep measurement is in progress, and generating a graphic user interface that allows the user to terminate sleep measurement.

[0054] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, wherein the graphic user interface generation step includes a step of displaying a pop-up window so that the user can check the predicted sleep time if the predicted sleep time exceeds a predetermined time.

[0055] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, the graphic user interface generation step of which includes a graphic user interface generation step that, if the predicted sleep time is less than a predetermined time, provides information that the sleep measurement results during sleep can be viewed for a predetermined time or longer.

[0056] To solve this problem, the present invention provides a method for providing an alarm based on a user's sleep state information, comprising: a sleep information receiving step of receiving user sleep information including the user's sleep acoustic information from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring user sleep state information including the user's average sleep time information based on the received user sleep information; an alarm time information generation step of generating alarm time information based on the acquired sleep state information and the user's desired wake-up time information; and an alarm sound provision step of providing an alarm sound based on the generated alarm time information.

[0057] To solve this problem, the method provides an alarm based on the user's sleep state information, wherein the time range including the user's desired wake-up time information includes the time from a predetermined time before the user's desired wake-up time to the user's desired wake-up time, and if the sleep state information does not indicate that the user is in REM sleep within the time range, the alarm time information is generated at the user's desired wake-up time.

[0058] To solve this problem, the method provides an alarm based on the user's sleep state information, wherein the time range including the user's desired wake-up time information includes the time from a predetermined time before the user's desired wake-up time to the user's desired wake-up time, and within the time range, if the sleep state information indicates that the user is in REM sleep, the alarm time information is generated at the time it indicates that the user is in REM sleep.

[0059] To solve this problem, the present invention provides a method for providing an alarm based on a user's sleep state information, further comprising a sleep information conversion step, which converts the user's sleep acoustic information in the time domain into information in the frequency domain.

[0060] To solve this problem, the present invention provides a method for providing an alarm based on a user's sleep state information, wherein the sleep information reception step further includes a sleep information inference step in which the user's sleep acoustic information is used as input to a sleep information inference deep learning model to infer information about sleep.

[0061] To solve this problem, the present invention provides a method for providing alarm time information based on a user's sleep state information, comprising: a sleep state information acquisition step of acquiring the user's sleep state information; a wake-up time information reception step of receiving the user's desired wake-up time information; and an alarm time information generation step of generating alarm time information based on the acquired sleep state information and the received desired wake-up time information, wherein in the alarm time information generation step, if the sleep state information indicates that the user is in REM sleep during the time from a predetermined time before the desired wake-up time to the desired wake-up time, the alarm time information is set to the time when the user is in REM sleep.

[0062] To solve this problem, the present invention provides a method for providing alarm time information based on the user's sleep state information, in the alarm time information generation stage, where, if the sleep state information indicates that the user is in REM sleep during the period from a predetermined time before the desired wake-up time to the desired wake-up time, the alarm time information is set to a time after a predetermined time from the time the user is in REM sleep.

[0063] To solve this problem, the present invention provides a method for providing alarm time information based on the user's sleep state information, in the alarm time information generation stage, setting the alarm time information to the point in time when the sleep state information indicates that the user is in REM sleep, after a predetermined time before the desired wake-up time and up to the desired wake-up time.

[0064] To solve this problem, the present invention provides a method for providing alarm time information based on the user's sleep state information, wherein, in the alarm time information generation stage, if the confidence level of the sleep state information indicating that the user is in REM sleep decreases after a predetermined time has passed from before the desired wake-up time until the desired wake-up time, the alarm time information is generated at the point when the confidence level of the sleep state information decreases.

[0065] In an embodiment of the present invention for solving the above-mentioned problems, a method for providing a graphic user interface that displays information about the user's sleep is disclosed.

[0066] A method for providing a graphic user interface according to one embodiment of the present invention may include a sleep information acquisition step of acquiring user sleep information from one or more sleep information sensor devices - user sleep information includes user sleep acoustic information.

[0067] A method for providing a graphic user interface according to one embodiment of the present invention may include a sleep state information acquisition step, which involves acquiring user sleep state information in real time based on acquired user sleep information - the user's sleep state information includes multiple sleep stage information.

[0068] Alternatively, a method for providing a graphic user interface according to one embodiment of the present invention may include a sleep state information acquisition step, which involves acquiring user sleep state information based on acquired user sleep information, and which includes a plurality of sleep stage information.

[0069] A method for providing a graphic user interface according to one embodiment of the present invention may include a sleep state information graph generation step, which generates a graph showing the user's sleep state information in proportion to time, based on acquired sleep state information.

[0070] A method for providing a graphic user interface according to one embodiment of the present invention may include the step of displaying the graphic user interface, which includes a generated graph.

[0071] Furthermore, a method for providing a graphic user interface according to one embodiment of the present invention may include a sleep state information graph generation step, which generates a graph showing the user's sleep state information in proportion to time based on acquired sleep state information, the graph including a plurality of rectangles.

[0072] Herein, a method for providing a graphic user interface according to one embodiment of the present invention may include the step of generating a sleep state information graph that includes a plurality of rectangles, each corresponding to a plurality of sleep stages.

[0073] Furthermore, the graphic user interface provided by one embodiment of the present invention includes a plurality of regions, each assigned to a plurality of sleep stages, and each of the plurality of rectangles included in the sleep state information graph may be displayed only in the region assigned to the corresponding sleep stage among the plurality of regions.

[0074] Furthermore, a method for providing a graphic user interface according to one embodiment of the present invention may include a sleep state information graph generation step, which generates a graph showing the user's sleep state information in accordance with time based on acquired sleep state information - the graph showing the user's sleep state information discretely represents the sleep state information.

[0075] Furthermore, a method for providing a graphic user interface according to one embodiment of the present invention may include the step of generating a sleep state information graph, each containing a plurality of discrete figures corresponding to a plurality of sleep stages.

[0076] Furthermore, a graphic user interface provided by one embodiment of the present invention may include a plurality of regions, each assigned to a plurality of sleep stages, each of which has an assigned height and a pitch assigned between the plurality of regions, and each of the plurality of figures included in the sleep state information graph may be discrete based on the pitch.

[0077] Furthermore, at least one of the boundaries between multiple areas included in the graphic user interface provided by one embodiment of the present invention may be represented in such a way that they are distinguished by lines.

[0078] Furthermore, a method for providing a graphic user interface according to one embodiment of the present invention may include the step of generating a sleep state information graph in which each of the multiple figures corresponds to a plurality of sleep stages, but at least one of the multiple figures is separated from the other figures.

[0079] Furthermore, according to one embodiment of the present invention, in the step of generating a sleep state information graph that includes the remaining figures and figures separated from each other, the figures separated from each other may correspond to the wakefulness stage.

[0080] Furthermore, according to one embodiment of the present invention, the multiple figures, each corresponding to one of the multiple sleep stages, may be represented as figures of the same form, and specifically, they may be represented as rectangles.

[0081] Furthermore, a graphic user interface according to one embodiment of the present invention may include a plurality of regions assigned to each sleep stage, wherein the plurality of figures corresponding to each of the plurality of sleep stages are displayed only in the region assigned to the corresponding sleep stage among the plurality of regions, and when the figures corresponding to a sleep stage are displayed separately from the remaining figures, the figures are not displayed in the regions assigned to other sleep stages.

[0082] Furthermore, in the sleep state information acquisition stage according to one embodiment of the present invention, the user's sleep state information can be acquired in 30-second intervals.

[0083] Furthermore, in the sleep state information graph generation stage according to one embodiment of the present invention, a graph showing sleep stage information in 30-second increments, which is included in the acquired sleep state information, can be generated.

[0084] Furthermore, in the sleep state information acquisition step according to one embodiment of the present invention, the acoustic information in the time domain included in the acquired sleep information is converted into information in the frequency domain, and the user's sleep state information can be acquired based on the converted information in the frequency domain.

[0085] Here, the information in the frequency domain according to one embodiment of the present invention may be a spectrogram or a Mel spectrogram to which a Mel scale has been applied.

[0086] Furthermore, in the sleep state information acquisition stage according to one embodiment of the present invention, the sleep state information can be acquired based on dividing the spectrogram into 30-second epochs.

[0087] Furthermore, each graph generated in the sleep state information graph generation stage according to one embodiment of the present invention may be represented by a different color assigned to correspond to each of the multiple sleep stage information included in the user's sleep state information.

[0088] Furthermore, in the sleep state information graph generation stage according to one embodiment of the present invention, a graph may be generated that displays the user's sleep onset time information or wake-up time information, which is included in the sleep state information acquired in the sleep state information acquisition stage, together with the sleep state information graph.

[0089] Furthermore, the graph generated in the sleep state information graph generation stage according to one embodiment of the present invention is characterized in that the user's sleep onset time information or wake-up time information, which is displayed together with the sleep state information graph, is numerically represented and generated so as to be connected to the sleep state information graph by a line and displayed in parallel.

[0090] The present invention provides a method for initiating sleep measurement via a user device, comprising: a sensing step of sensing a sleep measurement initiation trigger; and an initiation step of initiating sleep measurement at the time the trigger is sensed based on the sleep measurement initiation trigger.

[0091] The present invention provides a method for initiating sleep measurement via a user device, characterized in that the sleep measurement initiation trigger is sensed after the set user sleep preference time or during the period set in the sleep mode, when at least one of the user sleep preference time and sleep mode—the sleep mode being a mode in which the output of information indicating the occurrence of events related to an application running on the user device is restricted.

[0092] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is an exercise or movement applied to the user device, and in the sensing stage, if the user device senses the exercise or movement, sleep measurement is initiated at the time the initiation stage senses the exercise or movement.

[0093] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is a user's sleep preference voice input to the user device, and in the sensing stage, if the user device senses the user's sleep preference voice, the initiation stage starts sleep measurement at the time the user's sleep preference voice is sensed.

[0094] The present invention provides a method for initiating sleep measurement via a user device, wherein, within a predetermined time range set in the sensing stage, if the user device senses the start of charging, the recognition stage recognizes the charging start sensing time as the sensing time for the sleep measurement start trigger.

[0095] The present invention provides a method for initiating sleep measurement via a user device, wherein the charging initiated by the user device is wired charging.

[0096] The present invention provides a method for initiating sleep measurement via a user device, wherein the charging initiated by the user device is wireless charging.

[0097] The present invention provides a method for initiating sleep measurement via a user device, wherein, in the sensing stage, if the lock mode of the display screen of the user device is released after a predetermined time, the recognition stage recognizes the time of the lock mode release sensing as the sensing time for the sleep measurement start trigger.

[0098] The present invention provides a method for initiating sleep measurement via a user device, wherein, in the sensing stage, if the display unit of the user device senses a swipe input, the recognition stage recognizes the time when the swipe is sensed as the sensing time for the sleep measurement start trigger.

[0099] The present invention provides a method for initiating sleep measurement via a user device, wherein the swipe input is performed by the user's finger.

[0100] The present invention provides a method for initiating sleep measurement via a user device, wherein the swipe input is performed on the user's palm.

[0101] The present invention provides a method for initiating sleep measurement via a user device, wherein the recognition step recognizes the point in time when both of the two or more trigger sensing operations are sensed as the sensing point of the sleep measurement start trigger.

[0102] The present invention provides a method for initiating sleep measurement via a user device, comprising: a receiving step of receiving transmitted sleep measurement start trigger information when another device connected to the user device on a network senses sleep measurement start trigger information and transmits the sensed sleep measurement start trigger information; and an initiating step of initiating sleep measurement based on the received sleep measurement start trigger information at the time the trigger information is sensed.

[0103] The present invention provides a method for initiating sleep measurement via a user device, characterized in that the sleep measurement initiation trigger is sensed after the set user's desired sleep time or during the period set in the sleep mode, when at least one of the user device and the other device is set to the user's desired sleep time and a sleep mode—the sleep mode being a mode in which the output of information indicating the occurrence of events related to an application running on the user device or the other device is restricted.

[0104] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is physical contact between the other device and the user device, and the initiation step is to start sleep measurement when physical contact between the other device and the user device is sensed.

[0105] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is physical contact between the other device and the user's body, and the initiation step is to begin sleep measurement when physical contact between the other device and the user's body is sensed.

[0106] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is a user sleep measurement initiation trigger sound input to the other device, and the initiation step is to start sleep measurement at the time the user sleep measurement initiation trigger sound input to the other device is sensed.

[0107] The present invention provides a method for initiating sleep measurement via a user device, wherein the sleep measurement initiation trigger is a signal that the other device has started charging after a predetermined time, and the initiation step is to start sleep measurement at the time the other device starts charging.

[0108] The present invention provides a method for initiating sleep measurement via a user device, comprising: a sensing step of sensing information related to the user; a start determination information generation step of determining a sleep measurement start time based on the sensed user-related information; and a start step of starting sleep measurement at the determined start time.

[0109] The present invention provides a method for initiating sleep measurement via a user device, wherein, in the start determination information generation step, if information indicating that the user is using the user device cannot be sensed during a predetermined period of time while the sensing step is set to a mode in which the output of information indicating the occurrence of events related to an application running on the user device is restricted, the end of the predetermined period is determined as the start time for sleep measurement.

[0110] The present invention provides a method for initiating sleep measurement via a user device, wherein the information that the user is using the user device is at least one of the following: the user's movement or motion sensed by the user device, a signal indicating that the user device's display is lit, the user's voice input to the user device, or the user's tapping sensed by the user device's display.

[0111] The present invention provides an electronic device for initiating sleep measurement of a user, comprising: a sensing unit for sensing a sleep measurement start trigger; a memory unit on which an application can be recorded; and a processor unit on which the application can be executed, wherein the processor unit starts sleep measurement of the electronic device when the trigger is sensed by the sensing unit.

[0112] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, comprising: a sleep information reception step of receiving user sleep information, including the user's sleep acoustic information, from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring the user's sleep state information based on the received user sleep information; a step of receiving average sleep data of other people; a step of generating a graphic user interface that includes a graph comparing the received average sleep data of other people with the acquired user's sleep state information; and a step of displaying the graphic user interface.

[0113] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, characterized in that the graph provided for comparing the received average sleep data of others with the acquired sleep state information of the user has a length proportional to the data value.

[0114] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about a user's sleep, characterized in that the evaluation obtained by comparing the received average sleep data of others with the acquired sleep state information of the user is represented by "emojis".

[0115] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, characterized in that the average sleep data of others is an average value calculated based on medically obtained information.

[0116] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, characterized in that the average sleep data of others is an average value calculated based on medically recommended information.

[0117] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, characterized in that the average sleep data of other people is obtained statistically.

[0118] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, characterized in that the average sleep data of others is obtained by acquiring sleep state information of others and analyzing said sleep state information of others.

[0119] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, characterized in that the average sleep data of other people is classified based on the age range of those other people.

[0120] To solve this problem, the present invention provides a method for providing a graphic user interface that displays information about the user's sleep, characterized in that the average sleep data of other people is classified based on the gender of those other people.

[0121] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, characterized in that the average sleep data of others is classified based on the occupation of those others.

[0122] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, characterized in that the average sleep data of others is classified based on the activity records of others for that day.

[0123] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, characterized in that the average sleep data of others is classified based on the financial data information of others.

[0124] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, characterized in that the average sleep data of others is classified based on two or more combinations of the other person's age group, gender, occupation, daily activity record, and financial data.

[0125] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, where the received average sleep data of others is the average number of wakes during sleep, and the acquired user sleep state information is the number of wakes during sleep for the user.

[0126] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about the user's sleep, wherein the received average sleep data of others is average light sleep proportion information, and the acquired user sleep state information is the user's light sleep proportion information.

[0127] To solve this problem, the present invention provides a method for providing a graphical user interface that displays information about a user's sleep, wherein the received average sleep data of others is a combination of two or more of the following: average sleep onset delay information, average deep sleep proportion information, average REM sleep proportion information, average number of awakenings during sleep, and light sleep proportion information, and the acquired user sleep state information is a combination of two or more of the following: average sleep onset delay information, average deep sleep proportion information, average REM sleep proportion information, average number of awakenings during sleep, and light sleep proportion information.

[0128] To achieve the objectives of the present invention, a method for providing a graphic user interface that shows an evaluation of a user's sleep is provided, which is a method for providing a graphic user interface that shows an evaluation of a user's sleep using text, and includes a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices - the sleep information includes the user's sleep acoustic information - a sleep phrase generation step of generating a sleep phrase that includes at least two words that show an evaluation of the user's sleep based on the acquired sleep information - and a step of displaying a graphic user interface that includes the generated phrase.

[0129] Furthermore, the phrase provides a method for generating one or more graphic user interfaces that represent an evaluation of the user's sleep in a non-numerical manner.

[0130] Furthermore, the present invention provides a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including sleep environment information and user lifestyle information, based on the sleep information.

[0131] The present invention provides a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, including a sleep log information storage step in which sleep log information associated with an account assigned to the user is stored in memory.

[0132] Furthermore, the sleep information acquisition step can provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, further including a sleep information conversion step that converts the user's sleep acoustic information in the time domain into information in the frequency domain.

[0133] Furthermore, the method for generating one or more graphic user interfaces that show an evaluation of the user's sleep is characterized in that the sleep information acquisition step further includes a sleep information inference step, in which sleep acoustic information is used as input to a sleep information inference deep learning model to infer information about sleep.

[0134] The sleep phrase generation step provides a method for generating one or more graphic user interfaces that represent the user's sleep evaluation, including a first sleep phrase generation step that generates text that is of high importance among the user's sleep evaluations, and a second sleep phrase generation step that generates text that constitutes the user's sleep evaluation.

[0135] Furthermore, the sleep phrase generation step provides a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including a third sleep phrase generation step that generates advice text based on the evaluation of the user's sleep.

[0136] According to one embodiment of the present invention, the sleep phrase generation step can provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, further comprising a lookup table sleep phrase generation step which generates the sleep phrases based on a lookup table.

[0137] Furthermore, according to one embodiment of the present invention, the lookup table sleep phrase generation step can provide a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, which includes a lookup table user sleep characteristic classification step that classifies the user's sleep characteristics based on the sleep information, and a lookup table sleep phrase extraction step that extracts sleep phrases based on the lookup table corresponding to the user's sleep characteristics.

[0138] According to one embodiment of the present invention, the sleep phrase generation step can provide a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, further comprising a large-scale language model sleep phrase generation step that generates the sleep phrases based on a large-scale language model.

[0139] Furthermore, according to one embodiment of the present invention, a method can be provided for generating one or more graphic user interfaces that show an evaluation of a user's sleep, which includes a large-scale language model sleep phrase generation step, a large-scale language model user sleep characteristic classification step, which classifies the user's sleep characteristics based on the sleep information, and a large-scale language model sleep phrase extraction step, which extracts sleep phrases using the user's sleep characteristics as input to the large-scale language model.

[0140] According to one embodiment of the present invention, the user graphic user interface display stage can provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including a user sleep graph generation stage that generates a graph of sleep stages within the user's sleep time based on the sleep information.

[0141] Furthermore, according to one embodiment of the present invention, a method can be provided for generating one or more graphic user interfaces that show an evaluation of a user's sleep, characterized in that the third sleep phrase is arranged to be spaced apart from the generated first sleep phrase and the generated second sleep phrase, and is positioned at the lower end of the generated user sleep graph.

[0142] Furthermore, in order to achieve the objectives of the present invention, a non-temporary computer-readable storage medium containing a program that provides a graphic user interface showing an evaluation of a user's sleep is provided as a non-temporary computer-readable storage medium storing one or more programs configured to be executed by one or more processors to generate one or more graphic user interfaces showing an evaluation of a user's sleep, wherein the one or more programs include instructions to carry out the method according to an embodiment of the present invention.

[0143] Furthermore, in order to achieve the objectives of the present invention, the present invention provides a device that generates one or more graphic user interfaces that show an evaluation of a user's sleep, comprising: a display unit; one or more processors; and a memory that stores one or more programs configured to be executed by the one or more processors - the one or more programs include instructions to carry out the method according to the embodiment of the present invention.

[0144] To achieve the objectives of the present invention, the present invention provides a method for providing a graphic user interface that shows an evaluation of a user's sleep, utilizing a non-numerical method and a numerical method, comprising: a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices - the sleep information includes the user's sleep acoustic information; a sleep evaluation generation step of generating a sleep evaluation that shows an evaluation of the user's sleep based on the acquired sleep information - the sleep evaluation includes a numerical method and a non-numerical method; and a step of displaying a graphic user interface that includes the generated phrases, wherein the method provides one or more graphic user interfaces that show an evaluation of a user's sleep.

[0145] Furthermore, the present invention provides a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including sleep environment information and user lifestyle information, based on the sleep information.

[0146] The present invention also provides a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, including a sleep log information storage step in which sleep log information associated with an account assigned to the user is stored in memory.

[0147] The sleep information acquisition step further includes a sleep information conversion step, which converts the user's sleep acoustic information in the time domain into information in the frequency domain, to provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep.

[0148] Furthermore, the method for generating one or more graphic user interfaces that show an evaluation of a user's sleep is characterized in that the sleep information acquisition step further includes a sleep information inference step, in which sleep acoustic information is used as input to a sleep information inference deep learning model to infer information about sleep.

[0149] The present invention provides a method for generating one or more graphic user interfaces that represent an evaluation of a user's sleep, including a first sleep phrase generation stage that generates text that is of high importance among the evaluation of the user's sleep, and a second sleep phrase generation stage that generates text that constitutes the evaluation of the user's sleep.

[0150] Furthermore, the sleep evaluation generation step provides a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including a third sleep phrase generation step that generates advice text based on the evaluation of the user's sleep.

[0151] Furthermore, the sleep evaluation generation step provides a method for generating one or more graphic user interfaces that show the user's sleep evaluation, including a sleep numerical information generation step that provides the user with a numerical evaluation of their sleep based on the user's sleep evaluation.

[0152] According to one embodiment of the present invention, the sleep evaluation generation step can provide a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, further comprising a lookup table sleep phrase evaluation step that generates the sleep evaluation based on a lookup table.

[0153] Furthermore, according to one embodiment of the present invention, the lookup table sleep evaluation generation step can provide a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, which includes a lookup table user sleep characteristic classification step that classifies the user's sleep characteristics based on the sleep information, and a lookup table sleep evaluation extraction step that extracts a sleep evaluation based on the lookup table corresponding to the user's sleep characteristics.

[0154] According to one embodiment of the present invention, the sleep evaluation generation step can further include a large-scale language model sleep phrase generation step, which generates the sleep evaluation based on a large-scale language model, to provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep.

[0155] Furthermore, according to one embodiment of the present invention, the large-scale language model sleep evaluation generation step provides a method for generating one or more graphic user interfaces that show an evaluation of a user's sleep, including a large-scale language model user sleep characteristic classification step that classifies the user's sleep characteristics based on the sleep information, and a large-scale language model sleep evaluation extraction step that extracts sleep phrases using the user's sleep characteristics as input to the large-scale language model.

[0156] According to one embodiment of the present invention, the user graphic user interface display stage can provide a method for generating one or more graphic user interfaces that show an evaluation of the user's sleep, including a user sleep graph generation stage that generates a graph of sleep stages within the user's sleep time based on the sleep information.

[0157] Furthermore, in order to achieve the objectives of the present invention, a non-temporary computer-readable storage medium is provided which contains a program that provides a graphic user interface showing an evaluation of the user's sleep, and which stores one or more programs configured to be executed by one or more processors to generate one or more graphic user interfaces showing an evaluation of the user's sleep, wherein the one or more programs include instructions to carry out the method according to the embodiment of the present invention.

[0158] Furthermore, in order to achieve the objectives of the present invention, the present invention provides a device that generates one or more graphic user interfaces that show an evaluation of a user's sleep, comprising: a display unit; one or more processors; and a memory that stores one or more programs configured to be executed by the one or more processors - the one or more programs include instructions to carry out the method according to the embodiment of the present invention.

[0159] To achieve the objectives of the present invention, a method for collecting user sleep feedback and a method for providing a graphic user interface can be provided, which includes: a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring user sleep state information - the user sleep state information includes at least one of the user's sleep stage information and user's sleep event information - based on the acquired sleep information; a sleep service provision step of providing sleep services based on the acquired sleep state information or the acquired sleep information; and a step of displaying a user feedback graphic user interface to collect user feedback on the provided sleep services.

[0160] Furthermore, according to one embodiment of the present invention, the sleep information may include one or more of the following: sleep environment information, sleep acoustic information, and sleep lifestyle information.

[0161] Furthermore, according to one embodiment of the present invention, the sleep information acquisition step may include a preprocessing step that converts the sleep acoustic information into information that includes changes in frequency components over time.

[0162] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep content service based on the acquired sleep state information or the acquired sleep information.

[0163] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep analysis information provision service based on the acquired sleep state information or the acquired sleep information.

[0164] Furthermore, the sleep service provision stage can provide sleep environment adjustment services based on the acquired sleep state information or the acquired sleep information.

[0165] Furthermore, the step of displaying the user feedback graphic user interface may include the steps of collecting the user's response to the provided sleep service via the displayed user feedback graphic user interface, and storing the collected user response.

[0166] To achieve the objectives of the present invention, a method for collecting user sleep feedback and a method for providing a graphic user interface may include: a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring user sleep state information—the user sleep state information includes at least one of the user's sleep stages and user sleep events—based on the already acquired sleep information; a sleep service provision step of providing sleep services based on the acquired sleep state information or the acquired sleep information; and a user behavior feedback sensing step of collecting user feedback on the provided sleep services.

[0167] Furthermore, according to one embodiment of the present invention, the sleep information may include one or more of the following: sleep environment information, sleep acoustic information, and sleep lifestyle information.

[0168] Furthermore, according to one embodiment of the present invention, the sleep information acquisition step may include a preprocessing step that converts the sleep acoustic information into information that includes changes in frequency components over time.

[0169] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide sleep content services based on the acquired sleep state information or the acquired sleep information.

[0170] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep analysis information provision service based on the acquired sleep state information or the acquired sleep information.

[0171] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep environment adjustment service based on the acquired sleep state information or the acquired sleep information.

[0172] Furthermore, according to one embodiment of the present invention, the user action feedback sensing step may include a step of sensing the user's actions in response to the provided sleep service, and a step of converting the sensed user actions into information for feedback.

[0173] Furthermore, according to one embodiment of the present invention, the user motion feedback sensing step may include one or more of the following: voice motion feedback, physical motion feedback, and sleep environment adjustment feedback.

[0174] A method for collecting user sleep feedback and providing a graphic user interface to achieve the objectives of the present invention may include: a sleep information acquisition step of acquiring sleep information from one or more sleep information sensor devices; a sleep state information acquisition step of acquiring user sleep state information - the user sleep state information includes at least one of the user's sleep stages and user sleep events - based on the acquired sleep information; a sleep service provision step of providing sleep services based on the acquired sleep state information or the acquired sleep information; and a step of sensing changes in the acquired sleep state information in order to collect user feedback on the provided sleep services.

[0175] Furthermore, the sleep information may include one or more of the following: sleep environment information, sleep acoustic information, and sleep lifestyle information.

[0176] Furthermore, according to one embodiment of the present invention, the sleep information acquisition step may include a preprocessing step that converts the sleep acoustic information into information that includes changes in frequency components over time.

[0177] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide sleep content services based on the acquired sleep state information or the acquired sleep information.

[0178] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep analysis information provision service based on the acquired sleep state information or the acquired sleep information.

[0179] Furthermore, according to one embodiment of the present invention, the sleep service provision stage can provide a sleep environment adjustment service based on the acquired sleep state information or the acquired sleep information.

[0180] Furthermore, according to one embodiment of the present invention, the step of obtaining sleep state information feedback regarding sleep services based on the changes in the sensed sleep state information may be further included.

[0181] Furthermore, according to one embodiment of the present invention, the step of sensing changes in the acquired sleep state information may include the step of acquiring correspondence information between the acquired sleep state information and the provided sleep service, and the step of collecting changes in the sleep state information as feedback based on the acquired correspondence information.

[0182] Furthermore, according to one embodiment of the present invention, the step of sensing the change in the acquired sleep state information may include the steps of determining whether the change in the acquired sleep state information falls within the range of positive sleep criteria, and determining that if it falls within the range of positive sleep criteria, it is positive sleep feedback, and if it does not fall within the range of positive sleep criteria, it is negative sleep feedback.

[0183] Furthermore, according to one embodiment of the present invention, the sleep state information may include information regarding sleep apnea.

[0184] Furthermore, according to one embodiment of the present invention, the sleep state information may include information regarding the time of delayed sleep onset.

[0185] Furthermore, according to one embodiment of the present invention, the sleep state information may include information regarding the delay time of REM sleep.

[0186] Furthermore, according to one embodiment of the present invention, the sleep state information may include information on the proportion of deep sleep.

[0187] Furthermore, according to one embodiment of the present invention, the sleep state information may include information on the total amount of sleep.

[0188] To achieve the objectives of the present invention, a non-temporary computer-readable storage medium can be provided that stores one or more programs configured to be executed by one or more processors to collect user sleep feedback and to provide a graphical user interface, wherein the one or more programs include instructions to perform one or more of the methods described above.

[0189] To achieve the objectives of the present invention, the present invention provides a device for collecting user sleep feedback and providing a graphical user interface, comprising a display unit, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors - the one or more programs include instruction words for performing any one of the methods described above. [Effects of the Invention]

[0190] According to the present invention, by generating and providing an evaluation of the user's sleep using a graphical user interface, it is possible to provide the user with an evaluation of their sleep and contribute to improving the quality of their sleep.

[0191] Furthermore, by collecting users' sleep information, it is possible to accurately understand their individual sleep patterns, quality, and health status. By acquiring and visually representing sleep information from a diverse range of users, it becomes possible to perform a more accurate sleep assessment.

[0192] Furthermore, by collecting users' sleep information or feedback on sleep services, this data can be used for AI learning and other purposes, enabling the development of more accurate sleep analysis models.

[0193] According to the present invention, by generating and providing information about the user's sleep through a graphical user interface, it is possible to provide the user with information about their sleep and contribute to improving the quality of their sleep.

[0194] According to the present invention, by generating and providing information about the user's sleep through a graphical user interface, it is possible to provide the user with information about their sleep and contribute to improving the quality of their sleep. Furthermore, by intuitively indicating when the user's sleep information was acquired, it is possible to use this information to aid in the analysis of sleep data.

[0195] The effects of this invention are to accurately capture the timing of REM sleep and provide an alarm to the user; to provide the scheduled wake-up time and estimated sleep time on a single screen; to provide the estimated sleep time, which is updated in real time when the scheduled wake-up time is changed; to provide minimal time-related information on the screen after the alarm is set, so as not to create anxiety in the user about the wake-up time; to display a hypnogram on the screen after the alarm is set so that it can be accurately determined whether the user woke up at the REM sleep time; and to provide the user with information on how much more sleep they will have before the alarm sounds.

[0196] This invention provides a graphical user interface that analyzes the user's sleep acoustic information to provide sleep state information, and also provides information that can be easily understood at a glance by comparing the user's sleep results with the sleep of others.

[0197] This invention allows for easy triggering of sleep measurement on the user's device to provide information about sleep states.

[0198] According to the present invention, by generating and providing an evaluation of the user's sleep using a graphical user interface, it is possible to provide the user with an evaluation of their sleep and contribute to improving the quality of their sleep.

[0199] Furthermore, by collecting users' sleep information, it is possible to accurately understand their individual sleep patterns, quality, and health status. By acquiring and visually representing sleep information from a diverse range of users, it becomes possible to perform a more accurate sleep assessment.

[0200] Furthermore, by collecting users' sleep information or feedback on sleep services, this data can be used for AI learning and other purposes to train more accurate sleep analysis models. [Brief explanation of the drawing]

[0201] Figure 1a is a conceptual diagram showing a system in which various forms of a device that generates one or more graphic user interfaces showing information about a user's sleep, according to one embodiment of the present invention, can be realized.

[0202] Figure 1b is a conceptual diagram showing a system in which various forms of a device that provides one or more graphic user interfaces showing information about a user's sleep, according to one embodiment of the present invention, can be realized.

[0203] Figure 2a is a conceptual diagram showing a system in which the generation and / or provision of one or more graphic user interfaces showing information about the user's sleep, according to one embodiment of the present invention, is implemented on the user's terminal.

[0204] Figure 2b is a conceptual diagram showing a system in which various forms of diverse electronic devices according to the present invention can be realized.

[0205] Figure 3 is a block diagram showing the configuration of an apparatus / device that generates / provides one or more graphic user interfaces showing information about a user's sleep according to one embodiment of the present invention.

[0206] Figures 4a to 4g are diagrams showing a graphic user interface that displays information about the date and / or time when information about the user's sleep was obtained based on the time of waking up, according to an embodiment of the present invention.

[0207] Figures 5a to 5e are diagrams showing a graphic user interface that displays information about the date and / or time when information about the user's sleep was obtained based on the time of falling asleep, according to an embodiment of the present invention.

[0208] Figures 6a to 6e are diagrams showing a graphic user interface that displays information about the date and / or time of acquisition of information about the user's sleep based on the time of falling asleep and the time of waking up, according to an embodiment of the present invention.

[0209] Figures 7a to 7g are diagrams showing a graphic user interface that, according to an embodiment of the present invention, displays information regarding the date and / or time to which information about the user's sleep was obtained based on the definition of the time period to which the user woke up belonged.

[0210] Figure 8 is a diagram illustrating the process of acquiring sleep acoustic information in the sleep analysis method according to the present invention.

[0211] Figure 9 is a diagram illustrating a method for obtaining a spectrogram corresponding to sleep acoustic information in the sleep analysis method according to the present invention.

[0212] Figure 10 is a flowchart of a method for generating and providing one or more graphic user interfaces that show information about a user's sleep, according to one embodiment of the present invention.

[0213] Figure 11 is a schematic diagram showing one or more network functions for carrying out the sleep analysis method according to the present invention.

[0214] Figure 12 is a diagram illustrating sleep stage analysis using a spectrogram in the sleep analysis method according to the present invention.

[0215] Figure 13 is a diagram illustrating the sleep disorder assessment method using a spectrogram in the sleep analysis method according to the present invention.

[0216] Figure 14 is a diagram showing the experimental process for verifying the performance of the sleep analysis method according to the present invention.

[0217] Figure 15 is a diagram illustrating the overall structure of a sleep analysis model according to one embodiment of the present invention.

[0218] Figure 16 is a diagram illustrating a feature extraction model and a feature classification model according to one embodiment of the present invention.

[0219] Figures 17a and 17b are graphs that verify the performance of the sleep analysis method according to the present invention, comparing the results of polysomnography (PSG) and the analysis results using the AI ​​algorithm according to the present invention.

[0220] Figure 18 is a graph verifying the performance of the sleep analysis method according to the present invention, comparing the results of polysomnography (PSG) and analysis results using the AI ​​algorithm according to the present invention (AI result) in relation to sleep apnea and hypopnea.

[0221] Figure 19a is a diagram showing a graphic user interface that displays information about the sleep of a user whose alarm function according to the present invention has been activated.

[0222] Figure 19b is a diagram showing a graphic user interface that displays information about the sleep of a user in whom the alarm function according to the present invention was not activated.

[0223] Figure 20a is a diagram showing a graphic user interface indicating that sleep measurement is in progress with the alarm function according to the present invention activated.

[0224] Figure 20b is a diagram showing a graphic user interface indicating that sleep measurement is in progress with the alarm function according to the present invention deactivated.

[0225] Figure 21 is a diagram showing a graphic user interface according to the present invention that provides information to allow users to check their predicted sleep time if their predicted sleep time is longer than a predetermined time.

[0226] Figure 22 is a diagram showing a user interface that provides information corresponding to sleep measurement when the user's predicted sleep time according to the present invention is less than a predetermined time.

[0227] Figure 23 is a diagram illustrating a method for providing an alarm based on the user's sleep state information according to the present invention.

[0228] Figure 24a is a diagram illustrating the case where the AI ​​alarm of the present invention is not set, via a hypnogram.

[0229] Figure 24b is a diagram illustrating the case where the AI ​​alarm of the present invention is set, via a hypnogram.

[0230] Figure 25c is a diagram showing a respiratory stability graph according to an embodiment of the present invention.

[0231] Figure 25e is a diagram showing a graphic user interface including an explanatory display for respiratory instability according to one embodiment of the present invention.

[0232] Figures 26a and 26b are diagrams showing a graphic user interface including statistical information on sleep state according to an embodiment of the present invention.

[0233] Figures 27a and 27b are diagrams showing a graphic user interface that includes sleep state information acquired over one week, according to an embodiment of the present invention.

[0234] Figures 28a and 28b are diagrams showing a graphic user interface that includes sleep state information acquired over a predetermined period of time, according to an embodiment of the present invention.

[0235] Figure 29 is a flowchart of a method for generating and providing one or more graphic user interfaces that show information about a user's sleep, according to one embodiment of the present invention.

[0236] Figures 31a to 31e are diagrams showing hypnogram graphs of sleep stage information represented by a conventional sleep measurement interface.

[0237] Figure 32 is a diagram illustrating that, in one embodiment of the present invention, when a finger swipe input is sensed on the display of a user device, the moment the finger swipe is sensed is recognized as the sensing point for the sleep measurement start trigger.

[0238] Figure 33 is a diagram illustrating that, in one embodiment of the present invention, when a palm swipe input is sensed on the display of a user device, the moment the palm swipe is sensed is recognized as the sensing point for the sleep measurement start trigger.

[0239] Figure 34 is a diagram showing that, according to one embodiment of the present invention, when a user device senses the start of wired charging, it recognizes the time of the charging start sensing as the sensing time for the sleep measurement start trigger.

[0240] Figure 35 is a diagram showing that, according to one embodiment of the present invention, when a user device senses the start of wireless charging, it recognizes the time of the charging start sensing as the sensing time for the sleep measurement start trigger.

[0241] Figure 36 is a diagram illustrating that, according to one embodiment of the present invention, when movement or motion is sensed by a user device, the moment the movement or motion is sensed is recognized as the sensing point for the sleep measurement start trigger.

[0242] Figure 37 is a diagram illustrating that, in one embodiment of the present invention, when a user's voice is sensed by a user device, the moment the user's voice is sensed is recognized as the sensing point for the sleep measurement start trigger.

[0243] Figure 38 is a diagram illustrating that, according to one embodiment of the present invention, when another device connected to the network of a user device senses sleep measurement start trigger information, the sensing time is recognized as the sensing time of the sleep measurement start trigger.

[0244] Figure 39 is a diagram illustrating that, in one embodiment of the present invention, when sleep acoustic information is sensed by a user device, the moment the sleep acoustic information is sensed is recognized as the sensing point of the sleep measurement start trigger.

[0245] Figure 40 is a diagram illustrating one embodiment of the sleep mode in which, if sleep acoustic information is sensed by the user device, the moment when the sleep acoustic information is sensed is recognized as the sensing point of the sleep measurement start trigger.

[0246] Figure 41 is a diagram illustrating that, according to one embodiment of the present invention, when the unlocking of the user device is sensed, the moment the unlocking is sensed is recognized as the sensing point of the sleep measurement start trigger.

[0247] Figure 42 is a diagram illustrating one embodiment of the sleep mode in which, if finger tapping is not sensed by the user device, the sensing time for the sleep measurement start trigger is recognized as the time after a predetermined period of time has elapsed since the finger tapping was not sensed.

[0248] Figure 43 is a diagram illustrating one embodiment of the sleep mode in which, if the user's movement or motion is not sensed by the user device, the sensing time for the sleep measurement start trigger is recognized as the time after a predetermined period of time has elapsed since the time when the user's movement or motion is not sensed.

[0249] Figure 44 is a diagram illustrating one embodiment of the sleep mode, in which, when the user device's display turns off, the sensing time for the sleep measurement start trigger is recognized as the time after a predetermined period of time has elapsed since the user device's display turned off.

[0250] Figure 45 is a diagram illustrating one embodiment of the sleep mode in which, if the user's voice is not recognized by the user device, the sensing time for the sleep measurement start trigger is recognized as the time after a predetermined period of time has elapsed since the user's voice was no longer recognized by the user device.

[0251] Figure 46a is a diagram showing a graphic user interface illustrating a comparison of the average sleep onset delay time for the general population and the average sleep onset delay time for a user, according to one embodiment of the present invention.

[0252] Figure 46b is a diagram showing a graphic user interface illustrating a comparison of the average sleep onset delay time for a specific age group and gender with the average sleep onset delay time for a user, according to one embodiment of the present invention.

[0253] Figure 46c is a diagram showing a graphic user interface illustrating a comparison of the average sleep onset delay time for a specific occupation with the average sleep onset delay time for a user, according to one embodiment of the present invention.

[0254] Figure 46d is a diagram showing a graphic user interface according to one embodiment of the present invention, which compares the average sleep onset delay time for a specific age group, specific gender, and specific occupation with the average sleep onset delay time for the user.

[0255] Figure 47a is a diagram showing a graphic user interface illustrating a comparison between the average deep sleep of a general person and the average deep sleep of a user, according to one embodiment of the present invention.

[0256] Figure 47b is a diagram showing a graphic user interface according to one embodiment of the present invention, which compares the average deep sleep of a specific age group and gender with the average deep sleep of a user.

[0257] Figure 47c is a diagram showing a graphic user interface illustrating a comparison of deep sleep in a specific occupation with that of the user, according to one embodiment of the present invention.

[0258] Figure 47d is a diagram showing a graphic user interface illustrating a comparison of deep sleep levels in a specific age group, gender, and occupation with those of the user, according to one embodiment of the present invention.

[0259] Figure 48a is a diagram showing a graphic user interface illustrating a comparison of the average REM sleep of a general person with the REM sleep of a user, according to one embodiment of the present invention.

[0260] Figure 48b is a diagram showing a graphic user interface according to one embodiment of the present invention, which compares the average REM sleep of a specific age group and gender with the REM sleep of a user.

[0261] Figure 48c is a diagram showing a graphic user interface illustrating a comparison of REM sleep in a specific occupation with that of the user, according to one embodiment of the present invention.

[0262] Figure 48d is a diagram showing a graphic user interface illustrating a comparison of REM sleep in a specific age group, gender, and occupation with that of the user, according to one embodiment of the present invention.

[0263] Figure 49a is a flowchart of a method for generating one or more graphic user interfaces that show a non-numerical evaluation of a user's sleep according to one embodiment of the present invention.

[0264] Figure 49b is a flowchart of a method for generating one or more graphic user interfaces that show a user's sleep evaluation in both non-numerical and numerical ways, according to one embodiment of the present invention.

[0265] Figures 50a to 50c are diagrams illustrating one or more graphic user interfaces that demonstrate a non-numerical evaluation of a user's sleep according to one embodiment of the present invention.

[0266] Figure 51a is a diagram illustrating one or more graphic user interfaces that show a numerical evaluation of a user's sleep, according to one embodiment of the present invention.

[0267] Figure 51b is a diagram illustrating one or more graphic user interfaces that show a numerical evaluation of a user's sleep, according to one embodiment of the present invention.

[0268] Figure 51c is a diagram illustrating the factors influencing sleep quality, based on a survey of 36 psychiatrists conducted to generate a numerical evaluation formula for users' sleep, according to one embodiment of the present invention.

[0269] Figure 51d is a diagram illustrating a sleep keyword and a Likert Scale for calculating a numerical evaluation of a user's sleep, according to one embodiment of the present invention.

[0270] Figure 52 is a diagram illustrating the structure of a sleep analysis model that utilizes deep learning to analyze a user's sleep, according to one embodiment of the present invention.

[0271] Figure 53 is a diagram illustrating a method for generating sleep phrases based on a lookup table in a method for providing one or more graphic user interfaces that show an evaluation of a user's sleep, according to one embodiment of the present invention.

[0272] Figure 54 is a diagram illustrating a method for generating sleep phrases based on a large-scale language model in a method for providing one or more graphic user interfaces that show an evaluation of a user's sleep, according to one embodiment of the present invention.

[0273] Figure 55 is a diagram illustrating a user sleep graph, which generates a graph of sleep stages within a user's sleep time, in a method for providing one or more graphic user interfaces that show an evaluation of a user's sleep according to one embodiment of the present invention.

[0274] Figure 56 is a diagram illustrating a hypnogram showing sleep stages within a user's sleep time according to one embodiment of the present invention.

[0275] Figure 57 is a diagram illustrating a hypnodensity graph that displays sleep stages within a user's sleep time according to one embodiment of the present invention.

[0276] Figure 58 is a diagram illustrating an embodiment of the present invention that provides a sleep content service to a user in order to collect feedback.

[0277] Figure 59 is a diagram illustrating an embodiment of the present invention that provides sleep analysis information to a user in order to collect feedback.

[0278] Figure 60 is a diagram illustrating one embodiment of the invention, which provides a sleep environment adjustment service to a user in order to collect feedback.

[0279] Figure 61 is a diagram illustrating a user graphic display for collecting user feedback on sleep content, according to one embodiment of the present invention.

[0280] Figure 62 is a diagram illustrating a user graphic display for collecting user feedback on sleep analysis information, according to one embodiment of the present invention.

[0281] Figure 63 is a diagram illustrating a user graphic display for collecting user feedback on a sleep service, according to one embodiment of the present invention.

[0282] Figure 64 is a diagram illustrating user behavior feedback for collecting user feedback on a sleep environment adjustment service according to one embodiment of the present invention.

[0283] Figure 65 is a diagram illustrating user physical motion feedback for collecting user feedback on sleep services or information, according to one embodiment of the present invention.

[0284] Figure 66 is a diagram illustrating a user voice action feedback mechanism for collecting user feedback on sleep services or information, according to one embodiment of the present invention.

[0285] Figure 67 is a diagram illustrating a sleep state information feedback mechanism for collecting user feedback on sleep services or information, according to one embodiment of the present invention.

[0286] Figure 68 is a flowchart illustrating a method for collecting user feedback on sleep services or information utilizing user interface feedback, according to one embodiment of the present invention.

[0287] Figure 69 is a flowchart illustrating a method for collecting user feedback on sleep services or information utilizing user motion feedback, according to one embodiment of the present invention.

[0288] FIG. 70 is a flowchart for explaining a method for collecting user feedback on a sleep service or information utilizing user sleep state feedback according to an embodiment of the present invention.

[0289] FIG. 71 is a drawing for explaining the structure of a Transformer model that forms the basis of a Large Language Model.

[0290] FIG. 72 is a drawing for explaining an Inverter Model of a DIFFUSION model in content generation artificial intelligence according to an embodiment of the present invention.

[0291] FIG. 73 is a drawing for explaining a Generator and a Discriminator of a GAN (Generative Adversarial Network) in content generation artificial intelligence according to an embodiment of the present invention.

MODE FOR CARRYING OUT THE INVENTION

[0292] Overall structure

[0293] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0294] Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a part of each step may be performed by different devices from each other according to the embodiment.

[0295] FIG. 1a is a conceptual diagram showing a system in which various aspects of an apparatus 100 for generating one or more graphic user interfaces indicating information related to a user's sleep according to an embodiment of the present invention can be embodied. As shown in FIG. 1a, the system according to an embodiment of the present invention may include an apparatus 100 for generating a graphic user interface, a user terminal 10, an external server 20, and a network.

[0296] Here, a system in which an apparatus 100 for generating one or more graphic user interfaces indicating information regarding a user's sleep shown in FIG. 1a is implemented is according to one embodiment, and its components are not limited to the embodiment shown in FIG. 1a, and may be added, changed, or deleted as necessary.

[0297] FIG. 1b is a conceptual diagram showing a system in which various aspects of an apparatus 200 for providing one or more graphic user interfaces indicating information regarding a user's sleep according to an embodiment of the present invention can be implemented. As shown in FIG. 1b, a system according to an embodiment of the present invention may include an apparatus 200 for providing a graphic user interface, a user terminal 10, an external server 20, and a network.

[0298] Here, a system in which an apparatus 200 for providing one or more graphic user interfaces indicating information regarding a user's sleep shown in FIG. 1b is implemented is according to one embodiment, and its components are not limited to the embodiment shown in FIG. 1b, and may be added, changed, or deleted as necessary.

[0299] On the other hand, FIG. 2a shows a conceptual diagram showing a system in the case where generation and / or provision of one or more graphic user interfaces indicating information regarding a user's sleep is implemented in the user terminal 10 according to still another embodiment of the present invention. As shown in FIG. 2a, generation and / or provision of one or more graphic user interfaces indicating information regarding a user's sleep may be performed in the user terminal 10 without a separate generation apparatus 100 and / or a separate provision apparatus 200.

[0300] Also, FIG. 2b shows a conceptual diagram showing a system in which various aspects of various electronic devices related to still another embodiment of the present invention can be implemented.

[0301] The electronic device shown in Figure 2b can perform at least one of the operations performed by various devices according to embodiments of the present invention.

[0302] For example, the operations performed by various electronic devices according to embodiments of the present invention may include operations to acquire environmental sensing information, operations to perform learning for sleep analysis, operations to perform inference for sleep analysis, and operations to acquire sleep state information.

[0303] Alternatively, the system may include, for example, receiving information related to the user's sleep, transmitting or receiving environmental sensing information, discriminating environmental sensing information, extracting acoustic information from environmental sensing information, processing or manipulating data, processing services, providing services, constructing a learning dataset based on environmental sensing information or information related to the user's sleep, storing acquired data or multiple data that serve as input to a neural network, transmitting or receiving diverse information, mutually transmitting and receiving data for the system according to embodiments of the present invention via a network, generating one or more graphic user interfaces that display information related to the user's sleep, and providing one or more graphic user interfaces that display information related to the user's sleep.

[0304] The electronic device shown in Figure 2b can perform the various operations performed by the different electronic devices according to the embodiments of the present invention individually, but it can also perform one or more operations simultaneously or in a time series.

[0305] Referring to Figure 2b, the electronic devices 1a to 1d shown in Figure 2b may be electronic devices located within the region 11a from which object state information or environmental sensing information can be acquired. Hereinafter, for convenience, the region 11a from which object state information or environmental sensing information can be acquired will be referred to as "region 11a".

[0306] On the other hand, referring to Figure 2b, electronic devices 10a and 10d may be devices consisting of a combination of two or more electronic devices. Furthermore, electronic devices 10a and 10b may be electronic devices connected to a network within region 11a. Furthermore, electronic devices 10c and 10d may be electronic devices not connected to a network within region 11a.

[0307] On the other hand, referring to Figure 2b, electronic devices 20a to 20b may be electronic devices located outside the range of region 11a. Furthermore, they may be networks that interact with electronic devices within the range of region 11a, or networks that interact with electronic devices outside the range of region 11a.

[0308] Here, the network interacting with the electronic device within region 11a can perform the role of sending and receiving information for controlling smart home appliances.

[0309] Furthermore, the network interacting with the electronic device within the scope of region 11a may be, for example, a short-range network or a local network. Here, the network interacting with the electronic device within the scope of region 11a may be, for example, a long-range network or a global network.

[0310] On the other hand, referring to Figure 2b, there may be one or more electronic devices connected via the network outside the scope of region 11a, and these electronic devices may distribute data processing among themselves or perform one or more operations separately. Here, the electronic devices connected via the network outside the scope of region 11a may include server devices.

[0311] Alternatively, if there is one or more electronic devices connected via a network outside the range of region 11a, these electronic devices can perform a variety of operations independently of each other.

[0312] As shown in Figures 1a and 1b, the device 100 that generates one or more graphic user interfaces showing information about the user's sleep and the device 200 that provides one or more graphic user interfaces showing information about the user's sleep can mutually send and receive data for the system according to the embodiment of the present invention via a network with the user terminal 10.

[0313] As shown in Figure 2a, even if one or more graphic user interface generating devices 100 that display information about the user's sleep and one or more graphic user interface providing devices 200 that display information about the user's sleep are not separately provided, the user terminal 10 can perform the roles of one or more graphic user interface generating devices 100 that display information about the user's sleep and one or more graphic user interface providing devices 200 that display information about the user's sleep via a network, and can mutually send and receive data for the system according to the embodiment of the present invention.

[0314] As shown in Figure 2b, various electronic devices according to the present invention can mutually send and receive data for the system according to the embodiment of the present invention via a network.

[0315] The network according to embodiments of the present invention can utilize a variety of wired communication systems such as Public Switched Telephone Network (PSTN), xDSL (x Digital Subscriber Line), RADSL (Rate Adaptive DSL), MDSL (Multi Rate DSL), VDSL (Very High Speed ​​DSL), UADSL (Universal Asymmetric DSL), HDSL (High Bit Rate DSL), and Local Area Network (LAN). Furthermore, the network presented herein can utilize a variety of wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.

[0316] The network according to the embodiment of the present invention may be configured without distinguishing between wired and wireless communication modes, and may consist of various communication networks such as Personal Area Networks (PANs) and Wide Area Networks (WANs). Furthermore, the network may be the well-known World Wide Web (WWW), and may also utilize wireless transmission technologies used for short-range communication in conjunction with infrared (IrDA) or Bluetooth. The technologies described herein may be used not only in the networks mentioned above but also in other networks.

[0317] Figure 3 is a block diagram showing the configuration of a device 100 / providing a device 200 that generates one or more graphic user interfaces showing information about a user's sleep according to one embodiment of the present invention.

[0318] According to one embodiment of the present invention, the apparatus 100 for generating / providing one or more graphic user interfaces indicating information related to the user's sleep may include a display 120, a memory 140 storing one or more programs configured to be executed by one or more processors, and one or more processors 160.

[0319] According to one embodiment of the present invention, the memory 140 storing one or more programs includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Further, the memory can store instruction words for performing a method of providing one or more graphic user interfaces indicating information related to the user's sleep.

[0320] According to one embodiment of the present invention, the processor 160 may be composed of one or more. Further, the processor can execute a memory storing one or more programs.

[0321] Acquisition of environmental sensing information

[0322] The environmental sensing information or sleep information according to an embodiment of the present invention may be obtained from one or more sensor devices. Further, the sensor device according to one embodiment of the present invention may be embodied in the form of the user terminal 10.

[0323] The environmental sensing information may mean sensing information obtained in the space where the user is located. The environmental sensing information may be sensing information obtained in the space where the user is located by a non-contact method.

[0324] For example, the environmental sensing information may be acoustic information acquired in the bedroom where the user sleeps. According to the embodiment, the environmental sensing information acquired via the user terminal 10 may be the basis for acquiring the user's sleep state information in the present invention. To give a specific example, sleep state information related to whether the user is before sleep, during sleep, or after sleep may be acquired via environmental sensing information acquired in relation to the user's activities.

[0325] Furthermore, the environmental sensing information may consist of at least one of the following: noise information that frequently occurs in daily life (such as sound information related to cleaning, sound information related to cooking food, sound information related to watching TV, cat sounds, puppy sounds, bird sounds, car sounds, wind sounds, rain sounds, etc.) or other biometric information (such as electrocardiogram, electroencephalogram, pulse information, information on muscle movement, etc.).

[0326] Such a user terminal 10 may mean any form of entity in a system having a mechanism for communication with a device 100 / device 200 that generates a graphic user interface. For example, such a user terminal 10 may include a personal computer (PC), notebook computer, mobile terminal, smartphone, tablet PC, artificial intelligence (AI) speaker and AI TV, and wearable device, and may include all types of terminals that can connect to a wired / wireless network. The user terminal 10 may also include any server embodied by at least one of an agent, an API (Application Programming Interface), and a plug-in. The user terminal 10 may also include an application source and / or client application.

[0327] According to one embodiment of the present invention, the system may further include an external server that stores information on multiple training data for learning a neural network. The multiple training data may include, for example, health checkup information or sleep examination information. For example, the external server may be at least one of a hospital server and an information server, and may be a server that stores information on multiple sleep multi-referenced examination records, electronic health records and electronic medical records, etc. For example, the sleep multi-referenced examination record may include information on the breathing and movements of the sleep examination subject during sleep, and information on sleep diagnosis results (e.g., sleep stages) corresponding to said information. Information stored on the external server (not shown) may be used as training data, verification data and test data for learning a neural network in the present invention.

[0328] Furthermore, an external server according to one embodiment of the present invention may record an artificial intelligence model for analyzing sleep state information. In this case, by acquiring environmental sensing information from the user terminal 10 or the like and sending it to the external server, sleep state information can be generated based on the environmental sensing information via the artificial intelligence model implemented on the external server.

[0329] Alternatively, according to one embodiment of the present invention, environmental sensing information can be acquired at the user terminal 10, and sleep acoustic information can be acquired at the user terminal 10 through preprocessing of the environmental sensing information. The acquired sleep acoustic information can then be sent to an external server, and the external server can generate sleep state information based on the received sleep acoustic information.

[0330] A device 100 / device 200 that generates / provides a graphic user interface according to one embodiment of the present invention can receive health checkup information or sleep checkup information from an external server and construct a learning dataset based on that information.

[0331] An apparatus 100 / 200 that generates a graphic user interface according to one embodiment of the present invention can generate a sleep analysis model for acquiring sleep state information based on environmental sensing information by performing training on one or more network functions via a training dataset. A specific description of the configuration for constructing the training dataset for neural network learning and the learning method utilizing the training dataset of the present invention will be described later.

[0332] The external server may be a digital device equipped with a processor and memory, such as a laptop computer, notebook computer, desktop computer, webpad, or mobile phone, possessing computing power. The external server may be a web server that processes services. The types of servers described above are merely examples, and the present invention is not limited thereto.

[0333] According to one embodiment of the present invention, a device 100 / 200 that generates and provides a graphic user interface can acquire information about the user's sleep state and generate and / or provide one or more graphic user interfaces that show information about the user's sleep based on the information about the user's sleep state.

[0334] Specifically, the apparatus 100 / apparatus 200 for generating / providing a graphic user interface according to one embodiment of the present invention can acquire sleep state information related to whether the user is before sleep, during sleep, or after sleep based on environmental sensing information, and can generate and / or provide one or more graphic user interfaces that show information about the user's sleep based on the acquired sleep state information of the user.

[0335] On the other hand, the specific descriptions related to sleep state information mentioned above are merely illustrative, and the present invention is not limited thereto.

[0336] According to one embodiment of the present invention, one or more sleep sensor devices may include a microphone module, a camera, and an illuminance sensor provided in the user terminal 10.

[0337] For example, information related to the user's activities may be acquired in a single space via a microphone module provided in the user terminal 10.

[0338] Furthermore, when sensing information via a microphone module installed in the user terminal 10, the microphone module must be installed in the user terminal 10, which is relatively small in size, and therefore may be composed of a MEMC (Micro-electro Mechanical System).

[0339] The microphone module according to the embodiment of the present invention can be manufactured to be very small, but may have a lower signal-to-noise ratio (SNR) compared to condenser microphones and dynamic microphones. A low signal-to-noise ratio means that the proportion of noise, which is the sound that is not identified, is high compared to the sound that is to be identified, which may mean that sound identification is not easy (i.e., unclear).

[0340] Furthermore, the environmental sensing information to be analyzed in this invention may include acoustic information related to the user's breathing and movements acquired during sleep. Such acoustic information is information about very small sounds (i.e., sounds that are difficult to distinguish), such as the user's breathing and movements, and is acquired along with other sounds during sleep. Therefore, when acquired through a microphone module with a low signal-to-noise ratio as described above, detection and analysis can be quite difficult.

[0341] In such cases, an electronic device according to one embodiment of the present invention can convert and / or adjust environmental sensing information, which is acquired in an unclear manner containing a lot of noise, into analyzable data, and can utilize the converted and / or adjusted data to perform training on an artificial neural network. Once pre-training of the artificial neural network is complete, the trained neural network (e.g., an acoustic analysis model) can acquire information on the user's sleep state based on data (e.g., a spectrogram) acquired (e.g., converted and / or adjusted) in response to sleep acoustic information.

[0342] In this embodiment, the sleep state information may include not only information related to whether or not the user is sleeping, but also sleep stage information related to changes in the user's sleep stage during sleep. For example, the sleep state information may include sleep stage information indicating that the user was in REM sleep at a first time point, and was in light sleep at a second time point that differed from the first time point. In this case, through this sleep state information, information may be obtained indicating that the user fell into relatively deep sleep at the first time point and was in lighter sleep at the second time point.

[0343] In other words, the apparatus 100 / apparatus 200 for generating / providing a graphic user interface according to an embodiment of the present invention can acquire sleep acoustic information with a low signal-to-noise ratio via a commonly used user terminal (e.g., an artificial intelligence speaker, bedroom IoT device, mobile phone, wearable device, etc.) to collect sound, process this information into data suitable for analysis, and then process the processed data to provide sleep state information related to changes in sleep stages. This eliminates the need to equip the user with a microphone that is in contact with their body for clear sound acquisition, and also increases convenience by allowing sleep state monitoring in a typical home environment simply through a software update, without the need to purchase a separate additional device with a high signal-to-noise ratio.

[0344] In Figure 1a, the device 100 that generates the graphic user interface is represented as a separate entity from the user terminal 10. However, according to an embodiment of the present invention, as shown in Figure 2a, the device 100 that generates the graphic user interface is included within the user terminal 10, and sleep state measurement and sleep-related product recommendation functions can be performed by a single integrated device.

[0345] Similarly, although the device 200 providing the graphic user interface is represented as a separate entity from the user terminal 10 in Figure 1b, according to embodiments of the present invention, as shown in Figure 2a, the device 200 providing the graphic user interface can be included within the user terminal 10, and the functions of measuring sleep state and verifying sleep-related products can be performed in a single integrated device.

[0346] In embodiments, the device 100 that generates / provides a graphic user interface may be a terminal or a server, and may include any form of device. The device 100 that generates / provides a graphic user interface may be a digital device equipped with a processor and memory, such as a laptop computer, notebook computer, desktop computer, webpad, or mobile phone, and may have computing power. It may be a web server that processes the services of the device 100 that generates / provides a graphic user interface. The types of servers described above are merely examples, and the present invention is not limited thereto.

[0347] According to one embodiment of the present invention, the device 100 that generates a graphic user interface / the device 200 that provides it may be a server that provides cloud computing services. More specifically, the device 100 that generates a graphic user interface / the device 200 that provides it may be a server that provides cloud computing services, which is a type of internet-based computing, in which information is processed on other computers connected to the internet that are not the user's computer.

[0348] The aforementioned cloud computing service may be a service that stores materials on the internet and allows users to access necessary materials and programs anytime, anywhere via an internet connection without having to install them on their own computers, and allows for easy sharing and transmission of materials stored on the internet with simple operations and clicks. Furthermore, the cloud computing service may not only simply store materials on an internet server, but also allow users to perform desired tasks using the functions of applications provided on the web without having to install separate programs, and may be a service that allows various people to work on documents simultaneously.

[0349] Furthermore, the cloud computing service may be embodied in at least one form from among IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), virtual machine-based cloud server, and container-based cloud server. That is, the device 100 that generates / provides the graphic user interface of the present invention may be embodied in at least one form from among the cloud computing services described above. The specific description of the cloud computing service described above is merely illustrative and may include any platform for constructing the cloud computing environment of the present invention.

[0350] Information in the Time Domain

[0351] On the other hand, in the present invention, environmental sensing information may include sleep environment information and user lifestyle information. The sleep environment information may be acoustic information related to the user's sleep. One or more sleep information sensor devices can collect raw data related to the sounds generated during sleep for analysis of sleep. The raw data related to the sounds generated during sleep may be in the time domain.

[0352] Specifically, sleep acoustic information may be related to the user's breathing and movement patterns during sleep. For example, when awake, all nervous systems are activated, so breathing patterns may be irregular and there may be a lot of body movement. Also, because the throat muscles are not relaxed, breathing sounds may be very quiet. On the other hand, when the user falls asleep, the autonomic nervous system stabilizes, breathing becomes more regular, body movement decreases, and breathing sounds may become louder. Furthermore, if apnea occurs during sleep, a compensation mechanism may cause loud breathing sounds to occur immediately after the apnea. In other words, by collecting raw data related to sleep, it is possible to advance the analysis of sleep.

[0353] Preprocessing of information in the time domain

[0354] According to embodiments of the present invention, the acquired environmental sensing information and acoustic information can undergo a noise reduction preprocessing step as information in the time domain.

[0355] In the noise reduction process, noise (e.g., white noise) contained in the raw data is removed. The noise reduction process may be performed using algorithms such as spectral gating and spectral subtraction to remove background noise. Furthermore, in this invention, the noise reduction process can be carried out using a deep learning-based noise reduction algorithm. The deep learning-based noise reduction algorithm can be a noise reduction algorithm that is specialized for the user's breath and breathing sounds, in other words, a noise reduction algorithm that has been learned through the user's breath and breathing sounds.

[0356] The preprocessing described above may be performed during the learning process of sleep state information, or during the inference process.

[0357] Spectral noise gating

[0358] Spectral gating, or spectral noise gating, is a preprocessing method for acoustic information. While noise reduction can be performed on the entire acquired acoustic information, it is also possible to perform a split at regular time intervals (e.g., 5 minutes) and then perform noise reduction on each of the split acoustic pieces. To perform noise reduction on acoustic information split at regular time intervals, the method may include first calculating the spectrum for each frame.

[0359] This allows us to identify the frame with the lowest energy frequency spectrum among the calculated spectral frames.

[0360] The method may include assuming that the frame with the lowest energy frequency spectrum among each spectral frame is static noise, and then reducing the frequencies of the frequency spectrum frame assumed to be static noise from the spectral frame.

[0361] Deep learning-based noise reduction

[0362] To perform noise reduction preprocessing, a deep learning-based noise reduction method can be used, which is applied to raw acoustic information in the time domain rather than the frequency domain. For deep learning-based noise reduction, a method may be used that preserves information necessary for input to a sleep analysis model, such as sleep acoustic information, while reducing other sounds.

[0363] Noise reduction can be performed not only on acoustic information obtained through PSG test results, but also on acoustic information obtained through microphones built into user devices such as smartphones.

[0364] Generation / transformation and preprocessing of transformed information or spectrograms

[0365] In one embodiment of the present invention, raw acoustic information can be converted into information that includes the time-axis changes of the frequency components of the time-domain sleep acoustic information in order to analyze sleep acoustic information. On the other hand, the information in the frequency domain according to one embodiment of the present invention may mean that the raw sleep acoustic information has been converted into information that includes the time-axis changes of the frequency components.

[0366] Furthermore, according to the embodiments of the present invention, it is possible to visualize the time-axis changes of frequency components contained in low-resolution acoustic information from which noise has been removed, and convert them into information represented as an image.

[0367] According to one embodiment of the present invention, low acoustic information, which includes information on the change of frequency components over time, may be converted into a spectrogram. In this case, a method can be used to convert the low acoustic information into a spectrogram based only on the amplitude, excluding the phase. This method not only protects privacy but also reduces data size and improves processing speed. However, in other embodiments, it is also possible to generate a spectrogram using both phase and amplitude.

[0368] One embodiment of the present invention can generate a sleep analysis model using a spectrogram SP converted based on sleep acoustic information SS.

[0369] One embodiment of the present invention removes noise from sleep acoustic information using the method described above, converts it into a spectrogram, and generates a sleep analysis model by training on the spectrogram. This reduces the amount of computation and computation time, and also protects individual privacy.

[0370] For example, in acoustic information acquired via a microphone, sleep acoustic information necessary for analyzing sleep stages (e.g., the user's breathing) may be relatively small compared to other noises. However, when converted to a spectrogram, the identification of sleep acoustic information may be relatively superior compared to other ambient noises.

[0371] On the other hand, when converting to a spectrogram according to an embodiment of the present invention, personal information cannot be identified by converting to a low resolution in the frequency domain. However, if the frequency resolution (frequency bins) is set to a certain number (for example, 20) or less, personal information cannot be identified from the reconstructed signal.

[0372] Furthermore, embodiments of the present invention may include a method for converting acquired acoustic information into a spectrogram in real time.

[0373] Furthermore, by enabling the compression of the spectrogram's frequency resolution on the user's smartphone, rather than on a server or cloud, it is possible to prevent the leakage of personal information.

[0374] On the other hand, the spectrogram according to the embodiment of the present invention may be a Mel spectrogram to which the Mel scale is applied.

[0375] Method for converting raw sleep acoustic information

[0376] Figure 8 is a diagram illustrating the process of acquiring sleep acoustic information in the sleep analysis method according to the present invention.

[0377] Figure 9 is a diagram illustrating a method for obtaining a spectrogram corresponding to sleep acoustic information in the sleep analysis method according to the present invention.

[0378] As shown in Figure 9, the processor 130 can generate a spectrogram SP in response to sleep acoustic information SS. It can receive raw data (raw acoustic information in the time domain) which forms the basis for generating the spectrogram SP. The raw data according to the present invention may be collected via polysomnography (PSG) in a hospital environment, or it may be collected by the user in a home environment via a microphone built into a wearable device or user terminal such as a smartphone.

[0379] Furthermore, raw data may be acquired via the user terminal 10 from the start time entered by the user to the end time, or from the time the user performs a device operation (e.g., setting an alarm) to the time corresponding to the device operation (e.g., the alarm setting time), or the time may be automatically selected and acquired based on the user's sleep pattern, or the time of the user's intended sleep may be automatically determined and acquired based on sounds (user's voice, breathing sounds, sounds from peripheral devices (TV, washing machine), etc.) or changes in illumination. On the other hand, the intended sleep time according to one embodiment of the present invention may be calculated from the user's intended sleep information.

[0380] The processor 130 can perform a Fast Fourier Transform on the sleep acoustic information SS to generate a sleep spectrogram SP. The spectrogram SP is for visualizing and understanding sound and waves, and may be a combination of waveform and spectral features. The spectrogram SP may represent the difference in amplitude due to changes in the time axis and frequency axis as a difference in print density or display color.

[0381] The preprocessed acoustic raw data may be cut into 30-second segments and converted into spectrograms. This results in a 30-second spectrogram with dimensions of 20 frequency bins × 1201 time steps. In this invention, rectangular spectrograms can be transformed into a shape close to a square by employing various methods such as reshaping, resizing, and split-cutting. Alternatively, by using such methods, a relative amount of information can be stored.

[0382] On the other hand, the present invention can be used to simulate breath measured in various home environments by adding various noises generated in the home environment to clean breath. Sound has additive properties, so it can be added to each other. However, adding the original acoustic signals such as mp3 or pcm and converting them into a spectrogram would consume a lot of computing resources. Therefore, the present invention presents a method of adding breath and noise after converting them into spectrograms. Through this, by simulating breath measured in various home environments and utilizing this for AI model training, it becomes possible to ensure the robustness of the AI ​​model to information from various home environments.

[0383] Preprocessing of transformed information or spectrograms

[0384] The purpose of converting data into a spectrogram according to one embodiment of the present invention is to infer, through a learned model, which patterns in the spectrogram correspond to which sleep states or sleep stages, as input to a sleep analysis model. However, some preprocessing steps may be necessary before the data is used as input to the sleep analysis model. Such preprocessing steps may be performed only during the learning process, or both during the learning and inference processes, or only during the inference process.

[0385] The spectrogram preprocessing process according to embodiments of the present invention may include data augmentation preprocessing techniques such as adding Gaussian noise to the data to increase the amount of data, pitch shifting preprocessing methods that gradually raise or lower the overall acoustic pitch, and Tile UnTile (TUT) augmentation methods in which the spectrogram or Mel spectrogram is converted into a vector during the learning process, the converted vector is randomly tiled at the input stage of a node (neuron), and then rejoined after the output of the node (neuron).

[0386] Furthermore, the data augmentation preprocessing method according to the embodiment of the present invention may include a noise addition augmentation method that adds noise that occurs in various environments other than Gaussian noise (for example, external sounds, natural sounds, the sound of a fan turning, the sound of a door opening and closing, sounds made by animals, sounds of people talking, sounds of movement, etc.).

[0387] The noise augmentation according to an embodiment of the present invention may include a method of artificially adding noise information to the spectrogram after converting the noise information into a spectrogram, in order to shorten the training time when the spectrogram is used as input to a learning model. In this case, there may be little difference between the spectrogram obtained by converting the entire domain, which includes the original acoustic information and the noise information added to the sleep acoustic information, and the spectrogram obtained by converting the sleep acoustic information and the noise information separately into spectrograms and then adding them on the domain.

[0388] Furthermore, in order to protect the user's privacy by making it difficult to convert the spectrogram back to the original signal, the noise augmentation according to the embodiment of the present invention maintains only the amplitude of the sleep acoustic information and noise information spectrograms, while creating and adding a phase of an arbitrary type, thereby making it difficult to convert the spectrogram back to the original signal.

[0389] Alternatively, the noise augmentation according to embodiments of the present invention may include not only a method of adding noise on a domain converted from acoustic information into a spectrogram, but also a method of adding noise on a domain converted from a Mel spectrogram to which the Mel scale has been applied.

[0390] Furthermore, according to embodiments of the present invention, the method of adding data using the Mel scale can reduce the time required for hardware to process the data.

[0391] On the other hand, the specific descriptions of the types of noise mentioned above are merely illustrative examples for explaining the noise-adding augmentation of the present invention, and the present invention is not limited thereto.

[0392] Furthermore, according to embodiments of the present invention, a preprocessing method may be performed to convert information or a spectrogram in the frequency domain into a shape close to a square.

[0393] Sleep status information

[0394] In one embodiment, the sleep state information may include information related to whether or not the user is sleeping. Specifically, the sleep state information may include at least one of the following: first sleep state information indicating the user is in a sleep transition state; second sleep state information indicating the user is asleep; and third sleep state information indicating the user is post-sleep. In other words, if the first sleep state information is inferred in relation to the user, the processor 130 can determine that the user is in a pre-sleep state (i.e., before going to bed); if the second sleep state information is inferred, the processor can determine that the user is asleep; and if the third sleep state information is obtained, the processor can determine that the user is post-sleep (i.e., awake).

[0395] Furthermore, sleep state information may include information on at least one of the following: sleep apnea, snoring, tossing and turning, coughing, sneezing, or teeth grinding, in addition to information related to the user's sleep stage.

[0396] Learning or inferring sleep stage information according to embodiments of the present invention may require acoustic information acquired over long time intervals.

[0397] On the other hand, in order to learn or predict sleep state information other than sleep stage information according to embodiments of the present invention (for example, snoring or sleep apnea information), acoustic information acquired during a relatively short time interval (for example, 1 minute) before and after the occurrence of the sleep state may be required.

[0398] Such sleep state information can be characterized by being acquired based on environmental sensing information. Environmental sensing information may include sensing information acquired in the space where the user is located using a non-contact method.

[0399] According to one embodiment, the processor 130 can acquire environmental sensing information. Specifically, the environmental sensing information may be acquired via a user terminal 10 held by the user. For example, environmental sensing information related to the space in which the user is active may be acquired via the user terminal 10 held by the user, and the processor 130 can receive said environmental sensing information from the user terminal 10.

[0400] The processor 130 can acquire sleep state information based on acoustic information, actigraphy, and biometric information acquired from the user terminal 10. Specifically, the processor 130 can identify singularities from the acoustic information. Here, the singularities in the acoustic information may be related to breathing and movement patterns associated with sleep. For example, in a awake state, all nervous systems are activated, so breathing patterns may be irregular and there may be a lot of body movement. Also, because the throat muscles are not relaxed, the sound of breathing may be very quiet. On the other hand, when the user falls asleep, the autonomic nervous system stabilizes, breathing becomes regular, body movement also decreases, and the breathing sound may become louder. In other words, the processor 130 can identify singularities as points in time when acoustic information with patterns related to regular breathing, little body movement, or little breathing sound is detected from the acoustic information. Furthermore, the processor 130 can acquire sleep acoustic information based on the acoustic information acquired using the identified singularities as a reference. The processor 130 can identify singularities related to the user's sleep time from acoustic information acquired in a time series, and acquire sleep acoustic information based on those singularities.

[0401] Figure 8 is a diagram illustrating the process of acquiring sleep acoustic information in the sleep analysis method according to the present invention. Referring to Figure 8, the processor 130 can identify a singularity P from the acoustic information E that corresponds to the time when acoustic information of a pattern related to regular breathing, little body movement, or little breathing sound is detected. The processor 130 can acquire sleep acoustic information SS based on the acoustic information acquired after the identified singularity P, with the singularity P as the reference point. The waveforms and singularities related to acoustics in Figure 4 are merely illustrative for understanding the present invention, and the present invention is not limited thereto.

[0402] In other words, the processor 130 can identify singularities P related to the user's sleep from the acoustic information, and based on these singularities P, extract and acquire only the sleep acoustic information SS from a vast amount of environmental sensing information (i.e., acoustic information). This provides convenience by automating the process of the user recording their sleep time, while also contributing to improving the accuracy of the acquired sleep acoustic information.

[0403] Furthermore, in this embodiment, the processor 130 can acquire sleep state information related to whether the user is before or during sleep, based on a singularity P identified from the acoustic information E. Specifically, if the singularity P is not identified, the processor 130 can determine that the user is before sleep, and if the singularity P is identified, the processor 130 can determine that the user is asleep from that singularity P onward. The processor 130 can also identify a point in time (e.g., the time of waking) after the singularity P is identified when the identified pattern is no longer observed, and if that point in time is identified, the processor 130 can determine that the user has finished sleeping, i.e., has woken up.

[0404] In other words, the processor 130 can acquire sleep state information related to whether the user is before sleep, during sleep, or after sleep, based on whether a singularity P is identified from the acoustic information E and whether a previously set pattern is continuously perceived after the singularity is identified.

[0405] On the other hand, the processor 130 can acquire sleep state information based on actigraphy and biometric information, rather than acoustic information E. It is sometimes advantageous to acquire user movement information via a sensor unit that is in contact with the body. In this invention, since the user's sleep state information is grasped in advance using actigraphy and biometric information during primary sleep analysis, the reliability of the sleep state analysis can be further improved.

[0406] Sleep stage information

[0407] According to one embodiment, the processor 130 can extract sleep stage information. The sleep stage information may be extracted based on the user's environmental sensing information. Sleep stages may be divided into NREM (non-REM) sleep and REM (Rapid eye movement) sleep, and NREM sleep may be further divided into multiple stages (e.g., two stages: Light and Deep, or four stages: N1 to N4). The setting of sleep stages may be defined by general sleep stages, but may also be arbitrarily set by the designer using a variety of sleep stages. Through the analysis of sleep stages, it is possible to predict not only the quality of sleep related to sleep, but also sleep disorders (e.g., sleep apnea) and their underlying causes (e.g., snoring). The processor 130 can generate sleep-related product recommendation information and verification information based on the sleep stage information.

[0408] According to one embodiment of the present invention, when a word indicating a light sleep stage is displayed in Korean, it may be displayed as "light sleep" or "normal sleep." Users who are not professionally familiar with sleep stages may be misled by the word "light sleep" into believing that they did not sleep properly during their sleep period. To address this, displaying it as "normal sleep" may reduce such misunderstandings.

[0409] Sleep analysis model

[0410] Figure 15 is a diagram illustrating the overall structure of a sleep analysis model according to one embodiment of the present invention.

[0411] In this invention, sleep state information may be obtained through a sleep analysis model that analyzes the user's sleep stage based on acoustic information (sleep acoustic information).

[0412] In this invention, since the sleep acoustic information SS is sound related to breathing and body movements acquired during the user's sleep time, it may be very quiet. As a result, as described above, this invention allows for the conversion of the sleep acoustic information SS into a spectrogram SP to perform acoustic analysis. In this case, since the spectrogram SP contains information showing how the frequency spectrum of the sound changes over time, breathing or movement patterns associated with relatively quiet sounds can be easily identified, potentially improving the efficiency of the analysis. Specifically, while changes in the energy level of the sleep acoustic information alone may not be sufficient to predict whether the user is awake, in REM sleep, in light sleep, or in deep sleep, converting the sleep acoustic information into a spectrogram makes it easy to perceive changes in the spectrum of each frequency, potentially enabling analysis that corresponds to quiet sounds (e.g., breathing and body movements).

[0413] The processor 130 can obtain sleep state information by processing the spectrogram SP as input to a sleep analysis model. Here, the sleep analysis model is a model for obtaining sleep state information related to changes in the user's sleep stages, and can output sleep state information by taking sleep acoustic information acquired during the user's sleep as input. In an embodiment, the sleep analysis model may include a neural network model configured via one or more network functions.

[0414] Network functions and neural networks

[0415] Figure 11 is a schematic diagram showing one or more network functions for carrying out the sleep analysis method according to the present invention.

[0416] A sleep analysis model consists of one or more network functions, which may consist of a set of interconnected computational units that can generally be referred to as "nodes." Such "nodes" may be referred to as "neurons." One or more network functions consist of at least one node. The nodes (or neurons) that make up one or more network functions may be interconnected by one or more "links."

[0417] Within a neural network, one or more nodes connected via links can form a relative input-output node relationship. The concepts of input and output nodes are relative; any node that is an output node to one other node may be an input node to another node, and vice versa. As mentioned above, the input-output node relationship may be generated around links. One input node may be connected to one or more output nodes via links, and vice versa.

[0418] In an input-output node relationship connected via a single link, the value of the output node may be determined based on the data input to the input node. Here, the nodes that interconnect the input and output nodes may have weights. The weights may be variable and can be changed by the user or algorithm in order for the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by their respective links, the output node's value can be determined based on the values ​​input to the input nodes connected to the output node and the weights set for the links corresponding to each input node.

[0419] As mentioned above, a neural network consists of one or more nodes interconnected via one or more links, forming an input-output node relationship within the network. The characteristics of a neural network may be determined by the number of nodes and links within the network, the correlation between nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​between links, the two neural networks may be perceived as different from each other.

[0420] Some of the nodes that make up a neural network can form a layer based on their distance from the initial input node. For example, a set of nodes that are n in distance from the initial input node can form an n-layer. The distance from the initial input node may be defined by the minimum number of links that must be traversed to reach that node. However, such a layer definition is arbitrary for illustrative purposes, and the order of layers in a neural network may be defined in a different way than described above. For example, the layer of nodes may be defined by their distance from the final output node.

[0421] The initial input node may mean one or more nodes in the neural network that receive data directly without going through links in relation to other nodes. Alternatively, it may mean a node in the neural network that does not have other input nodes connected to it in relation to links. Similarly, the final output node may mean one or more nodes in the neural network that do not have an output node in relation to other nodes. Furthermore, hidden nodes may mean nodes that constitute the neural network that are neither the initial input node nor the final output node. In one embodiment of the present invention, the neural network may have more nodes in the input layer than in the hidden layer which is closer to the output layer, and the number of nodes may decrease as one moves from the input layer to the hidden layer.

[0422] A neural network may include one or more hidden layers. Hidden nodes in a hidden layer can take the output of the previous layer and the output of surrounding hidden nodes as input. The number of hidden nodes in each hidden layer may be the same or different. The number of nodes in an input layer may be determined based on the number of data fields in the input data, and may be the same as or different from the number of hidden nodes. Input data input to an input layer may be processed by the hidden nodes of the hidden layers and output by a fully connected layer (FCL), which is the output layer.

[0423] A deep neural network (DNN) can refer to a neural network that includes multiple hidden layers in addition to the input and output layers. Using a deep neural network, it is possible to grasp the latent structures of data. That is, it is possible to grasp the latent structures of photographs, texts, videos, audios, and music (for example, what objects are in a photograph, what is the content and emotion of a text, what is the content and emotion of an audio, etc.). Deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, Generative Adversarial Networks (GANs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Siam networks, Transformers, Vision Transformers (ViTs), and Mobile Vision Transformers (Mobile ViTs). The description of the deep neural network mentioned above is merely illustrative, and the present invention is not limited thereto.

[0424] In this invention, the network function may include an autoencoder. The autoencoder may be a type of artificial neural network for outputting output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be placed between the input and output layers. The number of nodes in each layer may be reduced to the number of nodes in the input layer, into an intermediate layer called a bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetric to the input layer). The nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetric. The autoencoder can perform nonlinear dimensionality reduction.

[0425] The number of input and output layers can correspond to the number of sensors remaining after preprocessing of the input data. In an autoencoder structure, the number of nodes in the hidden layers included in the encoder can decrease as it moves further away from the input layer. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is very small, a sufficient amount of information may not be transmitted, so it may be maintained at a certain number or more (for example, more than half of the input layer).

[0426] A neural network may be trained using at least one of the following methods: supervised learning, unsupervised learning, or semi-supervised learning. The purpose of training a neural network is to minimize the error in its output. In neural network training, iterative training data is input to the neural network, the output of the neural network and the target error for the training data are calculated, and the error of the neural network is backpropagated from the output layer to the input layer of the neural network in a direction that reduces the error, thereby updating the weight values ​​of each node in the neural network.

[0427] In supervised learning, training data is used in which each training data point is labeled with the correct answer (i.e., labeled training data), whereas in unsupervised learning, each training data point may not be labeled with the correct answer. For example, in supervised learning of data classification, the training data may be data in which each training data point is labeled with a category. The labeled training data may be input into a neural network, and the error may be calculated by comparing the output (category) of the neural network with the labels of the training data. As another example, in unsupervised learning of data classification, the error may be calculated by comparing the input training data with the output of the neural network.

[0428] The calculated error may be backpropagated from the neural network in the reverse direction (i.e., from the output layer to the input layer), and the backpropagation may update the connection weights of each node in each layer of the neural network. The amount of change in the connection weight of each node to be updated may be determined according to the learning rate. The calculation of the neural network on the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate may be applied differently depending on the number of iterations of the neural network's learning cycle. For example, a high learning rate may be used in the early stages of learning to increase efficiency by allowing the neural network to quickly achieve a certain level of performance, while a low learning rate may be used in the later stages of learning to increase accuracy.

[0429] In neural network training, training data is generally a subset of real-world data (i.e., data that the trained neural network intends to process). Therefore, there can be training cycles where errors on training data decrease, but errors on real-world data increase. Overfitting is the phenomenon where the training data is over-trained, leading to increased errors on real-world data. For example, a neural network trained to recognize cats by being shown yellow cats may fail to recognize cats that are not yellow; this is a type of overfitting. Overfitting can act as a cause of increased errors in AI algorithms. Various optimization methods can be used to prevent such overfitting. To prevent overfitting, methods such as increasing the amount of training data, regularization or normalization, or dropout (omitting some of the network nodes during the training process) may be applied.

[0430] Throughout this specification, the terms computational model, neural network, network function, and neural network may be used interchangeably. (Hereafter, the term neural network will be used consistently.) A data structure may include a neural network. A data structure including a neural network may be stored on a computer-readable medium. A data structure including a neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and a loss function for training the neural network. A data structure including a neural network may include any of the components of the configuration disclosed above. That is, a data structure including a neural network may consist of all or any combination thereof of data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and a loss function for training the neural network. In addition to the configuration described above, a data structure including a neural network may include any other information that determines the properties of the neural network. Furthermore, the data structure may include, but is not limited to, all forms of data used or generated in the computational processes of the neural network. Computer-readable media may include computer-readable recording media and / or computer-readable transmission media. The neural network may consist of a collection of interconnected computational units that may generally be referred to as nodes. Such nodes may be referred to as neurons. The neural network consists of at least one node.

[0431] Feature extraction models and feature classification models

[0432] Figure 16 is a diagram illustrating a feature extraction model and a feature classification model according to one embodiment of the present invention.

[0433] The sleep analysis model used in the present invention may include a feature extraction model that extracts one or more features for each predetermined epoch, and a feature classification model that classifies each of the features extracted via the feature extraction model into one or more sleep stages to generate sleep state information. The feature extraction model can extract features related to respiratory sounds, respiratory patterns, and movement patterns by analyzing the time-series frequency pattern of a spectrogram SP. In one embodiment, the feature extraction model may be configured via a portion of a neural network model pre-trained via a training dataset.

[0434] The sleep analysis model used in the present invention may include a feature extraction model and a feature classification model. The feature extraction model may be a deep learning model based on a natural language processing model that can learn the time-series relationships of given data. The feature classification model may be a deep learning model based on a natural language processing model that can learn the time-series relationships of given data. Here, the deep learning model based on a natural language processing model that can learn time-series relationships may include, but is not limited to, Tarnsformer, ViT, MobileViT, and MobileViT2.

[0435] A training dataset according to one embodiment of the present invention may consist of data in the frequency domain and multiple sleep state information corresponding to each data point.

[0436] Alternatively, a training dataset according to one embodiment of the present invention may consist of multiple spectrograms and multiple sleep state information corresponding to each spectrogram.

[0437] Alternatively, a training dataset according to one embodiment of the present invention may consist of multiple Mel spectrograms and multiple sleep state information corresponding to each Mel spectrogram.

[0438] For the sake of clarity, the configuration and execution of a sleep analysis model according to one embodiment of the present invention will be described in detail below based on a spectrogram dataset. However, the training data used in the sleep analysis model of the present invention is not limited to spectrograms; information in the frequency domain, spectrograms, or Mel spectrograms may be used as training data.

[0439] The feature extraction model in the sleep analysis model according to the embodiment of the present invention may be pre-trained by a one-to-one proxy task in which a single spectrogram is input and the model is trained to predict the sleep state information corresponding to that single spectrogram. When a CNN deep learning model is used in the feature extraction model according to the embodiment of the present invention, the learning can also be performed by adopting an FC (Fully Connected Layer) or FCN (Fully Connected Neural Network) structure. When a MobileViTV2 deep learning model is used in the feature extraction model according to the embodiment of the present invention, the learning can also be performed by adopting an Intermediate Layer structure.

[0440] Among the sleep analysis models according to embodiments of the present invention, the feature classification model can receive multiple consecutive spectrograms as input, predict sleep state information for each spectrogram, and learn to predict or classify overall sleep state information by analyzing the sequence of multiple consecutive spectrograms.

[0441] Furthermore, according to embodiments of the present invention, after performing pre-training on a feature extraction model via a one-to-one proxy task, fine-tuning can be performed on the pre-trained feature extraction model and feature classification model via a many-to-many task. For example, sleep stages can be inferred by inputting a sequence of 40 consecutive spectrograms into multiple feature extraction models trained on a one-to-one proxy task and outputting 20 pieces of sleep state information. The specific numerical descriptions relating the number of spectrograms, the number of feature extraction models, and the number of pieces of sleep state information mentioned above are merely illustrative, and the present invention is not limited thereto.

[0442] As described above, an inference model is generated to extract the user's sleep state and sleep stage through deep learning of environmental sensing information. To briefly explain again, environmental sensing information, including acoustic information, is converted into a spectrogram, and an inference model is generated based on the spectrogram.

[0443] The inference model may be built into the device 100 that generates / provides the graphical user interface, as described above.

[0444] Subsequently, environmental sensing information, including user acoustic information, acquired via the user terminal 10, is input to the inference model, and sleep state information and / or sleep stage information is output as result values. At this time, learning and inference may be performed by the same entity, or learning and inference may be performed by separate entities. That is, both learning and inference may be performed by the device 100 that generates the graphic user interface of Figure 1a or the device 200 that provides the graphic user interface of Figure 1b, while learning may be performed by the device 100 that generates the graphic user interface of Figure 1a or the device 200 that provides the graphic user interface of Figure 1b, but inference may be performed by the user terminal 10.

[0445] Alternatively, as shown in Figure 2a, if the device 100 for generating a graphic user interface and the device 200 for providing a graphic user interface are integrated into the user terminal 10, both learning and inference may be performed by the user terminal 10.

[0446] Alternatively, if the present invention is embodied through a system composed of various electronic devices, as shown in Figure 2b, learning or reasoning may be performed by at least one of the electronic devices shown in Figure 2b.

[0447] The feature extraction model and feature classification model according to embodiments of the present invention will be described in detail below.

[0448] Feature extraction model

[0449] The feature extraction model may consist of a proprietary deep learning model trained through a training dataset. The feature extraction model may be trained via supervised or unsupervised learning methods. The feature extraction model may be trained to output output data similar to the input data through the training dataset. More specifically, only the core feature data (or features) of the input spectrogram can be trained via a hidden layer. In this case, the output data of the hidden layer during the decoding process via the decoder may be an approximation of the input data (i.e., the spectrogram) that is not a perfect copy.

[0450] Each of the multiple spectrograms included in the training dataset may be tagged with sleep state information. Each of the multiple spectrograms may be input to a feature extraction model, and the output corresponding to each spectrogram may be stored after matching with the tagged sleep state information. Specifically, when a first training dataset (i.e., multiple spectrograms) tagged with first sleep state information (e.g., light sleep) is input, the features associated with the output for that input may be stored after matching with the first sleep state information. In this embodiment, one or more features associated with the output may be displayed in a vector space. In this case, the feature data output for each of the first training datasets can be located relatively close to each other in the vector space because they are outputs via spectrograms associated with the first sleep stage. That is, learning may be performed so that multiple spectrograms output similar features corresponding to each sleep stage.

[0451] The feature extraction model described above, which utilizes a learning process, can extract features corresponding to a spectrogram (for example, a spectrogram transformed to correspond to sleep acoustic information) when given a spectrogram as input.

[0452] In one embodiment, the processor 130 can process the spectrogram SP generated in response to the sleep acoustic information SS as input to a feature extraction model to extract features. Here, since the sleep acoustic information SS is time-series data acquired over time during the user's sleep, the processor 130 can divide the spectrogram SP into predetermined epochs. For example, the processor 130 can divide the spectrogram SP corresponding to the sleep acoustic information SS into 30-second units to acquire multiple spectrograms. For example, if sleep acoustic information is acquired during the user's 7 hours (i.e., 420 minutes) of sleep, the processor 130 can divide the spectrogram into 30-second units to acquire 840 spectrograms. The specific numerical descriptions of sleep duration, spectrogram division time units, and number of divisions mentioned above are merely illustrative, and the present invention is not limited thereto.

[0453] The processor 130 can process each of the divided spectrograms as input to a feature extraction model and extract multiple features corresponding to each of the spectrograms. For example, if the number of spectrograms is 840, the number of features extracted by the feature extraction model may also be 840. The specific numerical descriptions relating to the number of spectrograms and multiple features mentioned above are merely illustrative, and the present invention is not limited thereto.

[0454] On the other hand, the feature extraction model according to the embodiment of the present invention can also be trained using a one-to-one proxy task. Furthermore, in the process of training to extract sleep state information from a single spectrogram, the feature extraction model may be combined with other neural networks (NNs) to train it to extract sleep state information.

[0455] According to embodiments of the present invention, if training is performed via a pre-trained, simple neural network, the training time of the feature extraction model can be shortened or the training efficiency can be increased.

[0456] For example, one spectrogram divided into 30-second units according to one embodiment of the present invention may be trained to output sleep state information as input to a different neural network, where the output vector is the input to a feature extraction model.

[0457] Feature classification model

[0458] Furthermore, the processor 130 can obtain sleep state information by processing multiple features output via the feature extraction model as input to a feature classification model. In this embodiment, the feature classification model may be a neural network model modeled to predict sleep stages in relation to features. For example, the feature classification model may be a model comprising a fully connected layer that classifies features into at least one of the sleep stages. For example, when the feature classification model receives a first feature corresponding to a first spectrogram as input, it can classify the first feature as light sleep.

[0459] Feature classification models can perform multi-epoch classification, which predicts sleep stages for various epochs by taking spectrograms associated with various epochs as input. Multi-epoch classification does not provide one sleep stage analysis for a single epoch spectrogram (i.e., one spectrogram corresponding to 30 seconds), but rather takes spectrograms corresponding to multiple epochs (i.e., combinations of spectrograms each corresponding to 30 seconds) as input to estimate various sleep stages (e.g., changes in sleep stages due to changes in time) at once. For example, since breathing patterns or movement patterns change more slowly than electroencephalograms or other biological signals, accurate sleep stage estimation may only be possible by observing how the patterns change at past and future points in time. To give a specific example, a feature classification model can take 40 spectrograms (e.g., 40 spectrograms each corresponding to 30 seconds) as input and perform predictions for the 20 spectrograms in the middle. In other words, by examining all 1 to 40 spectrograms in detail, sleep stages can be predicted through classifications corresponding to 10 to 20 spectrograms. The specific numerical descriptions of the number of spectrograms mentioned above are merely illustrative, and the present invention is not limited thereto.

[0460] In other words, instead of predicting sleep stages by corresponding to each individual spectrogram in the sleep stage estimation process, the accuracy of the output can be improved by utilizing spectrograms corresponding to multiple epochs as input, so as to take into account all information related to the past and future.

[0461] Figure 12 is a diagram illustrating sleep stage analysis using a spectrogram in the sleep analysis method according to the present invention.

[0462] According to one embodiment of the present invention, following a primary sleep analysis based on Actigraphy and / or HRV, a secondary analysis based on sleep acoustic information will use the sleep analysis model described above. As shown in Figure 12, once the user's sleep acoustic information is input, the corresponding sleep stage (Wake, REM, Light, Deep) can be immediately inferred. In addition, the secondary analysis based on sleep acoustic information can extract the time when sleep disorders (sleep apnea, hyperventilation) or snoring occurred via singularities in the Mel spectrogram corresponding to the sleep stage.

[0463] Figure 13 is a diagram illustrating the determination of sleep disorders using a spectrogram in the sleep analysis method according to the present invention.

[0464] As shown in Figure 13, by analyzing the breathing pattern in a single mel spectrogram, if characteristics corresponding to sleep apnea or hyperpnea events are detected, that point in time can be determined to be the time when the sleep disorder occurred. At this time, the process may further include classifying the snoring as not being sleep apnea or hyperpnea through frequency analysis.

[0465] Figure 14 is a diagram showing the experimental process for verifying the performance of the sleep analysis method according to the present invention.

[0466] As shown in Figure 14, the user's sleep video and sleep acoustics are acquired in real time, and the acquired sleep acoustic information can be immediately converted into information in the frequency domain. In one embodiment of the present invention, the user's sleep acoustic information may be immediately converted into a spectrogram. At this time, a preprocessing process for the sleep acoustic information may be performed. The converted information in the frequency domain or spectrogram may be input into a sleep analysis model to immediately analyze the sleep stage.

[0467] Furthermore, when a CNN or Transformer-based deep learning model is used in the feature classification model according to one embodiment of the present invention, the operation may be carried out as follows.

[0468] One embodiment of the present invention allows a spectrogram containing time-series information to be used as input to a CNN-based deep learning model, outputting a vector with reduced dimensionality. This reduced-dimensional vector can then be used as input to a transformer-based deep learning model, outputting a vector containing time-series information.

[0469] According to embodiments of the present invention, the output vector of a transformer-based deep learning model can be input to a 1d CNN (1d Convolutional Neural Network) so that the average pooling technique can be applied to it, and the process of converting it into an N-dimensional vector containing time-series information through an averaging operation on the time-series information can be performed. In this case, the N-dimensional vector containing time-series information still corresponds to data that contains time-series information, only differing in resolution from the input data.

[0470] By performing multi-epoch classification on a combination of N-dimensional vectors containing time-series information output by embodiments of the present invention, predictions for various sleep stages can be made. In this case, the output vectors of a Transformer-based deep learning model can also be used as inputs to multiple Fully Connected Layers (FCs) to perform predictions of continuous sleep state information.

[0471] Furthermore, if a ViT or Mobile ViT-based deep learning model is used in the feature classification model according to one embodiment of the present invention, the operation may be carried out as follows.

[0472] A processor according to one embodiment of the present invention can output a low-dimensional vector when a spectrogram containing time-series information is used as input to a Mobile ViT-based deep learning model.

[0473] Furthermore, according to the embodiment of the present invention, features can be extracted from each spectrogram in the output of the deep learning model on the Mobile ViT platform.

[0474] In embodiments of the present invention, a vector with reduced dimensionality may be used as input to an intermediate layer, and a vector containing time-series information may be output. The intermediate layer model may include at least one of the following steps: a linearization step that stores information in the vector, a layer normalization step for inputting the mean and variance, or a dropout step that deactivates some nodes.

[0475] According to the embodiment of the present invention, overfitting can be prevented by performing a process in which a vector with reduced dimensionality is used as input to an intermediate layer and a vector containing time-series information is output.

[0476] According to an embodiment of the present invention, sleep state information can be output by using the output vector of the intermediate layer as input to a ViT-based deep learning model. In this case, sleep state information corresponding to information in the frequency domain containing time-series information, a spectrogram, or a Mel spectrogram can be output.

[0477] Furthermore, according to one embodiment of the present invention, sleep state information corresponding to a series of configurations of information in the frequency domain that includes time-series information, a spectrogram, or a Mel spectrogram can be output.

[0478] On the other hand, the feature extraction model or feature classification model according to the embodiments of the present invention can employ a variety of deep learning models in addition to the AI ​​models mentioned above to perform learning or inference. The specific descriptions related to the types of deep learning models mentioned above are merely illustrative, and the present invention is not limited thereto.

[0479] Performance and graph of the sleep analysis method according to the present invention

[0480] Figures 17a and 17b are graphs that verify the performance of the sleep analysis method according to the present invention, comparing the results of polysomnography (PSG) and the analysis results using the AI ​​algorithm according to the present invention.

[0481] As shown in Figures 17a and 17b, when compared with the results of polysomnography (PSG), it was confirmed that the results of the sleep analysis model using sleep acoustic information as input were very accurate.

[0482] Existing sleep analysis models predict sleep stages using ECG (Electrocardiogram) or HRV (Heart Rate Variability) as input. However, the present invention converts sleep acoustic information into information, a spectrogram, or a Mel-spectrogram that includes the time-axis changes in the frequency components of the sleep acoustic information in the time domain, and uses this as input to proceed with the analysis and inference of sleep stages. Therefore, unlike existing sleep analysis models, because sleep acoustic information is converted into information, a spectrogram, or a Mel-spectrogram in the frequency domain as input, it is possible to sense or acquire sleep stages in real time through the analysis of the specificity of sleep patterns.

[0483] According to the present invention, a graphic user interface that displays information about the user's sleep may be provided.

[0484] The following describes the performance of the sleep analysis method according to the present invention and the method for obtaining graphs showing information about the user's sleep.

[0485] A hypnogram showing sleep stage information.

[0486] Figures 17a and 17b are graphs that verify the performance of the sleep analysis method according to the present invention, comparing the results of polysomnography (PSG) and the analysis results using the AI ​​algorithm according to the present invention.

[0487] The hypnodensity graph shown at the bottom of Figure 17a is a graph that shows the probability of belonging to one of the four sleep stage classes.

[0488] The hypnogram, shown in the center of Figure 17a, can be obtained by determining the sleep stage with the highest probability from the hypnodensity graph.

[0489] At this time, according to one embodiment of the present invention, when receiving input of user sleep acoustic information and predicting the sleep stage, it is possible to show the probability of which of the four classes (Wake, Light, Deep, REM) the user belongs to in 30-second intervals. Here, the four classes represent the awake state, the lightly asleep state, the deeply asleep state, and the REM sleep state, respectively.

[0490] As shown in Figure 17b, the sleep analysis results obtained by the present invention showed excellent agreement when compared with labeling data obtained through multi-source sleep analysis.

[0491] Furthermore, it demonstrated the ability to include more precise and meaningful information related to sleep stages (Wake, Light, Deep, REM).

[0492] Graph showing sleep breathing information

[0493] Figure 18 is a graph verifying the performance of the sleep analysis method according to the present invention, comparing the results of polysomnography (PSG) and analysis results using the AI ​​algorithm according to the present invention (AI result) in relation to sleep apnea and hypopnea.

[0494] The probability graph at the bottom of Figure 18 shows the probability of which of two disorders (sleep apnea or respiratory depression) a user is classified into in 30-second intervals when predicting sleep disorders based on input sleep acoustic information.

[0495] Of the three graphs shown in Figure 18, the central graph can be obtained by determining the disease with the highest probability from the probability graph shown below it.

[0496] As shown in Figure 18, the sleep analysis method according to the present invention yielded sleep state information that closely matched that obtained from multi-factor sleep studies. Furthermore, it demonstrated the ability to include more precise analytical information related to sleep apnea and respiratory depression.

[0497] According to the present invention, it is possible to analyze the user's sleep in real time and identify the point at which sleep disorders (sleep apnea, hyperventilation, hypopnea) occur. If the user is given a stimulus (tactile, auditory, olfactory, etc.) at the moment a sleep disorder occurs, the sleep disorder may be temporarily alleviated. In other words, according to the present invention, it is possible to interrupt the user's sleep disorder based on accurate detection of events associated with sleep disorders and reduce the frequency of sleep disorders.

[0498] Date information and graphic user interface related to the acquisition of user sleep information

[0499] In the field of sleep analysis and related technologies, services existed that acquired and provided information about users' sleep. However, there was difficulty in intuitively indicating when the user's sleep information was acquired, and research has been conducted to address this issue.

[0500] The present invention may provide a method for generating information to display the date on which information regarding a user's sleep was acquired. The method for providing information on the date on which sleep information was acquired according to an embodiment of the present invention has the advantage of making it easier to analyze sleep by allowing the user to intuitively understand the date on which the sleep information was acquired.

[0501] Furthermore, according to the present invention, a graphic user interface may be provided that displays information about the user's sleep. Additionally, a graphic user interface may be provided that displays information about the date and / or time when the user's sleep information was acquired.

[0502] According to embodiments of the present invention, the graphic user interface showing information about the user's sleep may be displayed on an electronic device or user terminal 10 that is embodied in various forms of displays.

[0503] The following describes in detail, through embodiments, a method for generating information to display the date on which user sleep information was acquired, according to the present invention. Furthermore, a graphic user interface for displaying sleep information according to embodiments of the present invention will be described using drawings.

[0504] Based on the time of waking, date information is generated and displayed for sleep-related data.

[0505] Figures 4a to 4f are diagrams showing a graphic user interface that displays information about the date and / or time when information about the user's sleep was obtained based on the time of waking up, according to an embodiment of the present invention.

[0506] According to one embodiment of the present invention, a method is provided for generating date information for sleep-related information based on wake-up time information included in acquired sleep state information.

[0507] Alternatively, according to one embodiment of the present invention, a method may be provided to display the sleep state information at the time of waking up, based on the date the sleep state information at that time of waking up was acquired, using the waking time information included in the acquired sleep state information.

[0508] For example, as shown in Figure 4a, if sleep state information is obtained that includes information that the person fell asleep on April 5th and woke up on April 6th, the date on which this sleep state information was obtained can be indicated as April 6th, which is the date of waking up.

[0509] Alternatively, as shown in Figure 4d, if sleep state information is obtained that includes information that the person fell asleep on June 7th and woke up on the same day, June 7th, the date of waking up can be indicated as the date the sleep state information was obtained.

[0510] Thus, when expressing the date on which sleep information was obtained based on the time of waking, it has the effect of emphasizing that the sleep experience has been analyzed, by giving the user the impression that they are being provided with a report analyzing their past sleep. Furthermore, when obtaining and providing sleep state information that includes information such as falling asleep the previous night and waking up the following morning, it has the effect of further emphasizing that the sleep experience during the nighttime has been analyzed.

[0511] The numerical descriptions related to the dates mentioned above, as well as the specific descriptions regarding the time of falling asleep and waking up, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0512] In one embodiment of the present invention, when generating date information obtained based on the time of waking up, a graphical user interface including a figure for displaying the date may be provided.

[0513] Furthermore, according to one embodiment of the present invention, sleep may occur several times on the same day. As shown in Figure 4e, if several sleeps occur and the user wakes up N times (where N is a non-negative integer) on the same day, then N pieces of information indicating the day on which sleep information was acquired may be generated on that same day, based on the information regarding the time of waking up. In such cases, when the information indicating the date is displayed in the graphic user interface, it may be displayed to include N shapes.

[0514] For example, as shown in reference number 101b in Figure 4b, if two sleep sessions resulted in two awakenings on April 8th, then two figures indicating that sleep-related information was obtained on April 8th may be generated.

[0515] Alternatively, according to one embodiment of the present invention, as shown in reference numeral 101d in Figure 4g, if three sleep sessions result in three awakenings on July 1st, then three graphics indicating that sleep-related information was acquired on July 1st may be generated. In this case, when the area of ​​the graphic user display corresponding to the date July 1st is checked, it may be displayed as three graphics as shown in reference numeral 101d.

[0516] On the other hand, according to one embodiment of the present invention, a graphic user interface showing sleep state information obtained for a specific session out of three sleep sessions measured on that date may be displayed at its lower end by swiping left / right on the graphic user interface or by touching each of the shapes shown in 101d. Here, the sleep state information displayed at the lower end may include at least one of the following: the time of falling asleep, the time of waking up, the duration of sleep, and text information indicating information about sleep, which are included in the user's sleep information obtained on that date, as shown in Figure 4g.

[0517] The numerical descriptions related to the dates mentioned above, as well as specific descriptions regarding the number of times one wakes up, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0518] Furthermore, the figure used to display the date according to the embodiment of the present invention may be represented by at least one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of straight and curved lines. For example, as shown in reference numeral 101c of Figure 4c, in one embodiment of the present invention, the figure used to display the date may be represented by a star shape, but this is merely an example, and as described above, it may be represented by one or more of the various figures.

[0519] Furthermore, the graphic user interface according to the embodiment of the present invention may display at least one of the following text information indicating information about sleep, which is included in the sleep information of the user acquired on that date, as shown in reference numerals 102 to 104 in Figures 4a to 4c: the time of falling asleep, the time of waking up, the duration of sleep, and information about sleep. In this case, whether or not the sleep information is acquired on a particular date may be indicated using graphics, as shown in reference numeral 101a, to distinguish it from other dates. Alternatively, it may be indicated by using a different color or brightness from other dates.

[0520] For example, if an analysis of sleep stages included in the user's sleep information obtained on April 6th shows that the proportion of deep sleep was relatively high, the phrase "You slept soundly last night! You'll be able to start the day in good condition." may be displayed along with the phrase "You slept soundly last night! You'll be able to start the day in good condition." (References 102 and 103). Furthermore, according to embodiments of the present invention, the actual time spent sleeping and the time it took to fall asleep can also be displayed in parallel (Reference 104). The phrases shown in the sleep evaluation text information described above are merely examples to illustrate embodiments of the present invention, and the present invention is not limited thereto.

[0521] On the other hand, the graphic user interface according to the embodiment of the present invention, which was described in detail earlier, is not limited to being displayed in Korean, but may be displayed in a variety of languages. For example, when the date is to be displayed in English, it may be represented as shown in Figure 4f.

[0522] Based on the time of falling asleep, date information related to sleep is generated and displayed.

[0523] Figures 5a to 5d are diagrams showing a graphic user interface that displays information about the date and / or time when information about the user's sleep was obtained based on the time of falling asleep, according to an embodiment of the present invention.

[0524] According to one embodiment of the present invention, a method is provided for generating date information for sleep-related information based on sleep onset time information included in acquired sleep state information.

[0525] Alternatively, according to one embodiment of the present invention, a method is provided for displaying the time of falling asleep based on the date on which the sleep state information was acquired, after acquiring sleep state information.

[0526] For example, as shown in Figure 5a, if sleep state information is obtained that includes information that the person fell asleep on April 5th and woke up on April 6th, the date on which this sleep state information was obtained can be indicated as April 5th, which is the date the person fell asleep.

[0527] Alternatively, as shown in Figure 5d, if sleep state information is obtained that includes information that the person fell asleep on April 8th and woke up on the same day, April 8th, the date of falling asleep (April 8th) can be indicated as the date the sleep state information was obtained.

[0528] When sleep information is obtained based on the time of falling asleep, this method has the effect of accurately providing users with the time they fell asleep.

[0529] The numerical descriptions related to the dates mentioned above, as well as the specific descriptions regarding the time of falling asleep and waking up, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0530] In one embodiment of the present invention, when generating date information obtained based on the time of falling asleep, a graphical user interface including a figure for displaying the date may be provided.

[0531] Furthermore, according to one embodiment of the present invention, sleep may occur several times on the same day. For example, if sleep occurs several times, resulting in N instances of falling asleep on the same day (where N is a non-negative integer), then N pieces of information displaying the date on which the sleep-related information was acquired may be generated on that same day, based on the information regarding the time of falling asleep. In such cases, when the information displaying the date is displayed in the graphic user interface, it may be displayed to include N shapes.

[0532] For example, as shown in reference number 201b in Figure 5b, if two sleep sessions resulted in two instances of falling asleep on April 7th, then two figures indicating that sleep-related information was obtained on April 7th may be generated.

[0533] Alternatively, according to one embodiment of the present invention, as shown in reference numeral 201d in Figure 5e, if three sleep sessions occurred on July 1st, then three graphics indicating that information regarding sleep was acquired on July 1st may be generated. In this case, when the area of ​​the graphic user display corresponding to the date July 1st is checked, it may be displayed as three graphics as shown in reference numeral 201d.

[0534] On the other hand, according to one embodiment of the present invention, by swiping left / right on the graphic user interface or touching each of the shapes shown in 201d, a graphic user interface showing sleep state information acquired for a specific session out of three sleep sessions measured on that date may be displayed at its lower end. Here, the sleep state information displayed at the lower end may include at least one of the following: the time of falling asleep, the time of waking up, the duration of sleep, and text information indicating information about sleep, which are included in the user's sleep information acquired on that date, as shown in Figure 5e.

[0535] The numerical descriptions related to the dates mentioned above, as well as specific descriptions regarding the number of times one falls asleep, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0536] Furthermore, the figure used to display the date according to the embodiment of the present invention may be represented by at least one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of straight and curved lines. For example, as shown in Figure 5c, according to one embodiment of the present invention, the figure used to display the date may be represented by a star shape, but this is merely an example, and as described above, it may be represented by one or more of the following diverse figures.

[0537] Furthermore, the graphic user interface according to the embodiment of the present invention may display at least one of the following text information indicating information about sleep, which is included in the sleep information of the user acquired on that date, as shown in reference numerals 202 to 204 in Figures 5a to 5c: the time of falling asleep, the time of waking up, the duration of sleep, and information about sleep. In this case, whether or not the sleep information is acquired on a particular date may be indicated using graphics, as shown in reference numeral 201a, to distinguish it from other dates. Alternatively, it may be indicated by using a different color or brightness from other dates.

[0538] For example, if an analysis of sleep stages included in the sleep information of a user obtained on April 5th shows that the proportion of deep sleep was relatively high, the phrase "You slept soundly last night! You'll be able to start the day in good condition." may be displayed along with the phrase "You slept soundly last night! You'll be able to start the day in good condition." (References 202 and 203). Furthermore, according to embodiments of the present invention, the actual time spent sleeping and the time it took to fall asleep can also be displayed in parallel (Reference 204). The phrases that appear in the sleep evaluation text information described above are merely examples to illustrate embodiments of the present invention, and the present invention is not limited thereto.

[0539] Date information is generated and displayed based on the time of falling asleep and waking up, providing information about sleep.

[0540] Figures 6a to 6c are diagrams showing a graphic user interface that displays information about the date and / or time of acquisition of information about the user's sleep based on the time of falling asleep and the time of waking up, according to an embodiment of the present invention.

[0541] According to one embodiment of the present invention, a method is provided for generating date information for sleep-related information based on sleep onset time information and wake-up time information included in acquired sleep state information.

[0542] Alternatively, according to one embodiment of the present invention, a method is provided for displaying the period from the time of falling asleep to the time of waking up, based on the date the sleep state information was acquired, after acquiring sleep state information, using the information on the time of falling asleep and the time of waking up included in the acquired sleep state information.

[0543] For example, as shown in Figure 6a, if sleep state information is obtained that includes information that the person fell asleep on April 5th and woke up on April 6th, the date on which this sleep state information was obtained can be indicated as being between April 5th and April 6th.

[0544] Alternatively, as shown in reference number 301b in Figure 6b, if sleep state information is obtained that includes information that the person fell asleep on April 8th and woke up on the same day, April 8th, the date on which the sleep state information was obtained can be indicated as April 8th, which is the date of falling asleep and waking up.

[0545] Thus, when expressing the date on which sleep information was obtained based on the time of falling asleep and waking up, it has the effect of intuitively informing the user of the period during which sleep occurred by showing them the date on which sleep was sustained.

[0546] The numerical descriptions related to the dates mentioned above, as well as the specific descriptions regarding the time of falling asleep and waking up, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0547] In one embodiment of the present invention, when generating date information obtained based on the time of falling asleep and the time of waking up, a graphical user interface including a figure that displays the sleep duration may be provided.

[0548] Furthermore, referring to reference no. 301b, etc., the figure for displaying the date according to the embodiment of the present invention may be a figure having a continuous form that displays from the date of falling asleep to the date of waking up. The figure according to the embodiment of the present invention may be displayed using at least one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of straight lines and curves.

[0549] Furthermore, according to one embodiment of the present invention, sleep may occur several times on the same day. For example, if sleep occurs several times on the same day, and as a result, the person falls asleep and wakes up N times each on the same day (N is a non-negative integer), then N pieces of information indicating the date on which the sleep information was acquired may be generated on that same day, based on the information regarding the time of falling asleep and the time of waking up. In such cases, when the information indicating the date is displayed in the graphic user interface, it may be displayed to include N figures. Here, the N figures may be generated as figures having a continuous form connecting two different points, and each of the N figures according to the embodiment of the present invention may be represented by at least one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of a line and a curve.

[0550] For example, as shown in reference number 301c in Figure 6c, if two sleep sessions occurred on April 8th, both falling asleep and waking up, then two figures indicating that sleep-related information was obtained on April 8th may be generated.

[0551] Alternatively, according to one embodiment of the present invention, as shown in reference numeral 301d in Figure 6d, if three sleep sessions occur, with three instances of falling asleep and three instances of waking up occurring on July 1st, then three graphics indicating that information regarding sleep was acquired on July 1st may be generated. In this case, when the area of ​​the graphic user display corresponding to the date July 1st is checked, it may be displayed as three graphics as shown in reference numeral 301d.

[0552] On the other hand, according to one embodiment of the present invention, a graphic user interface showing sleep state information obtained for a specific session out of three sleep sessions measured on that date may be displayed at its lower end by swiping left / right on the graphic user interface or by touching each of the shapes shown in 301d. Here, the sleep state information displayed at the lower end may include at least one of the following: the time of falling asleep, the time of waking up, the duration of sleep, and text information indicating information about sleep, which are included in the user's sleep information obtained on that date, as shown in Figure 6d.

[0553] Alternatively, according to one embodiment of the present invention, a graphic user interface may be generated as shown in reference numerals 305 and 301d in Figure 6e. Reference numeral 305 is a code indicating an area for displaying the date. Reference numeral 301e is a graphic that displays date information obtained regarding sleep, which is displayed in the area assigned by reference numeral 305.

[0554] When displaying date information obtained based on sleep onset time information and wake-up time information included in sleep state information acquired by one embodiment of the present invention, a graphic user interface may be generated that displays the portion corresponding to the sleep onset time and wake-up time on the horizontal axis showing the time-series information shown in reference number 305, as in reference number 301e.

[0555] For example, as shown in Figure 6e, if a person falls asleep (or starts sleep measurement) at 11:42 p.m. on the 5th and wakes up (or ends sleep measurement) at 7:01 a.m. on the 6th, the date information can be displayed in the area shown in reference number 305, using a figure that represents the period from the 5th to the 6th. The starting point of the figure may be displayed in the area allocated to the 5th, but close to the boundary line between the 5th and the 6th, and the ending point of the figure may be displayed in the area allocated to the 6th, but in the area allocated to the 6th, which is the 7th of 24 equal parts drawn on the horizontal axis.

[0556] On the other hand, according to one embodiment of the present invention, a graphic user interface showing sleep state information acquired on the given date may be displayed at its lower end. The sleep state information displayed at the lower end may include at least one of the following, as shown in Figure 6e: the time of falling asleep, the time of waking up, the duration of sleep, and text information indicating information about sleep, all included in the user's sleep information acquired on the given date. According to one embodiment of the present invention, when sleep is measured on various dates, the information displayed at the lower end can be distinguished by using different colors or different brightness levels for the graphic shown in reference number 301e, thereby indicating which day's sleep measurements the information displayed at the lower end is based on.

[0557] The numerical descriptions related to the dates mentioned above, as well as specific descriptions regarding the number of times one falls asleep, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0558] Furthermore, the graphic user interface according to the embodiment of the present invention may also display text information indicating the time of falling asleep, time of waking up, sleep duration, and information about sleep, which are included in the sleep information of the user acquired on that date, as shown in reference numerals 302 to 304 in Figures 6a to 6c. In this case, whether or not the sleep information is acquired on a particular date may be indicated using graphics, as shown in reference numeral 301a, to distinguish it from other dates. Alternatively, it may be indicated by using a different color or brightness from other dates.

[0559] For example, if an analysis of sleep stages included in the sleep information of a user obtained from April 5th to April 6th shows that the proportion of REM sleep was relatively high, the phrase "Stress-relieving sleep" may be displayed along with the phrase "You had a high proportion of REM sleep! This type of sleep not only relieves stress but also helps to unleash creativity." (References 302 and 303). Furthermore, according to embodiments of the present invention, the actual time spent sleeping and the time it took to fall asleep can also be displayed in parallel (Reference 304). The phrases shown in the sleep evaluation text information described above are merely examples to illustrate embodiments of the present invention, and the present invention is not limited thereto.

[0560] The acquisition time information is generated and displayed based on the definition of the time period to which the wake-up time belonged.

[0561] Figures 7a to 7e are diagrams showing a graphic user interface that, according to an embodiment of the present invention, displays information regarding the date and / or time to which information about the user's sleep was obtained based on the definition of the time period to which the user woke up belonged.

[0562] According to one embodiment of the present invention, a method is provided for generating date information for sleep-related information based on wake-up time information included in acquired sleep state information.

[0563] Alternatively, according to one embodiment of the present invention, a method is provided to display the wake-up time on the date the sleep state information was acquired, based on the wake-up time information included in the acquired sleep state information after acquiring sleep state information. Here, date information for acquiring sleep information can be generated based on a definition of the time period to which the wake-up time belongs. The definition of the time period may already be set, for example, "dawn" may be set for 12:00 a.m. (midnight) or more but less than 5:00 a.m., "morning" for 5:00 a.m. or more but less than 9:00 a.m., "daytime" for 9:00 a.m. or more but less than 5:00 p.m., "evening" for 5:00 p.m. or more but less than 9:00 p.m., and "night" for 9:00 p.m. or more but less than 12:00 a.m. (midnight).

[0564] For example, as shown in Figure 7a, if sleep state information is obtained that includes information that the person fell asleep on April 5th and woke up on April 6th, the date on which this sleep state information was obtained can be indicated as April 6th.

[0565] Alternatively, if sleep state information is obtained that includes information that the person fell asleep on April 8th and woke up on the same day, April 8th, the date of waking up (April 8th) can be indicated as the date the sleep state information was obtained, as shown in reference number 401d in Figure 7c.

[0566] In one embodiment of the present invention, when generating date information obtained based on the time of waking up, a graphical user interface including a figure for displaying that date may be provided.

[0567] On the other hand, according to an embodiment of the present invention, the color of the figure displaying the date information on which sleep information was acquired can be the same color (for example, white), but as shown in Figure 7d, the color of the figure displaying the date information may be displayed differently depending on the time period on which the user's sleep information was acquired.

[0568] For example, if the sleep time included in the user's sleep information mainly falls within the "early morning" time slot, the shape indicating the date the sleep information was acquired can be displayed in a dark blue color. Alternatively, if the sleep time included in the user's sleep information mainly falls within the "daytime" time slot, the shape indicating the date the sleep information was acquired can be displayed in yellow.

[0569] The figure used to display the date according to an embodiment of the present invention may be represented by at least one of the following: a point, a polygon, a circle, an ellipse, a sector, or a figure consisting of a combination of straight and curved lines.

[0570] On the other hand, according to embodiments of the present invention, the type of figure used to display the date information on sleep is the same type (for example, a dot), but as shown in Figure 7c, the type of figure may be displayed differently depending on the time period during which the user's sleep information was acquired.

[0571] For example, if the sleep time included in the user's sleep information mainly falls within the "early morning" time period, the date the sleep information was acquired can be displayed as a star shape, as shown in reference number 401c in Figure 7c. Alternatively, if the sleep time included in the user's sleep information mainly falls within the "daytime" time period, the date the sleep information was acquired can be displayed as a rectangle (for example, a square), as shown in reference number 401d.

[0572] On the other hand, the specific descriptions of the colors and types of the figures mentioned above are merely illustrative examples to illustrate the present invention, and the present invention is not limited thereto. It can be extended to the extent that a person of ordinary skill can easily adopt the figures and colors according to the present invention, as long as they can clearly and intuitively present information about sleep to the user.

[0573] Furthermore, according to one embodiment of the present invention, sleep may occur several times on the same day. For example, if sleep occurs several times on the same day, and as a result, the user wakes up N times (where N is a non-negative integer) on the same day, then N pieces of information displaying the date on which the sleep information was acquired may be generated on that same day, based on the information regarding the time of waking up. In such cases, when the information displaying the date is displayed in the graphic user interface, it may be displayed to include N shapes.

[0574] For example, as shown in reference number 401b in Figure 7b, if two sleep sessions resulted in two awakenings on April 7th, then two figures indicating that sleep-related information was obtained on April 7th may be generated.

[0575] Alternatively, according to one embodiment of the present invention, as shown in reference numeral 401f in Figure 7f, if three sleep sessions result in three awakenings on July 1st, then three graphics indicating that sleep-related information was acquired on July 1st may be generated. In this case, when checking the area of ​​the graphic user display corresponding to the day July 1st, it may be displayed as three graphics as shown in reference numeral 401f.

[0576] On the other hand, according to one embodiment of the present invention, by swiping left / right on the graphic user interface or touching each of the shapes shown in 401f, a graphic user interface showing sleep state information acquired for a specific session out of three sleep sessions measured on that date may be displayed at its lower end. Here, the sleep state information displayed at the lower end may include at least one of the following: the time of falling asleep, the time of waking up, the duration of sleep, and text information indicating information about sleep, which are included in the user's sleep information acquired on that date, as shown in Figure 7f.

[0577] On the other hand, as described in detail earlier, according to the embodiment of the present invention, the type of figure used to display the date information on sleep is the same type (for example, a dot), but as shown in reference numeral 401g in Figure 7g, the type of figure may be displayed differently depending on the time period during which the user's sleep information was acquired.

[0578] The numerical descriptions related to the time periods mentioned above, as well as the specific descriptions regarding the time of falling asleep and waking up, are merely illustrative examples for explaining the present invention, and the present invention is not limited thereto.

[0579] Furthermore, the graphic user interface according to the embodiment of the present invention may also display, as shown in reference numerals 402 to 404 in Figures 7a to 7e, the time the user fell asleep, the time they woke up, the duration of sleep, and text information indicating information about sleep, which are included in the sleep information of the user acquired on that date. In this case, whether or not the sleep information was acquired on a particular day may be indicated using graphics, as shown in reference numeral 401a, to distinguish it from other dates. Alternatively, it may be indicated by using a different color or brightness from other dates.

[0580] For example, if an analysis of sleep stages included in the sleep information of a user obtained on April 6 shows that the proportion of REM sleep was relatively high, the phrase "Stress-relieving sleep" may be displayed along with the phrase "You had a high proportion of REM sleep! This type of sleep not only relieves stress but also helps to unleash creativity." (References 402 and 403). Furthermore, according to embodiments of the present invention, the actual time spent sleeping and the time it took to fall asleep can also be displayed in parallel (Reference 404). The phrases shown in the sleep evaluation text information described above are merely examples to illustrate embodiments of the present invention, and the present invention is not limited thereto.

[0581] Furthermore, according to embodiments of the present invention, a graphic user interface may be provided that further includes text displaying time-of-day information for which sleep-related information has been acquired, as shown in reference numeral 405 of Figure 7e.

[0582] For example, if the time period to which the sleep data included in the user's sleep information belongs is primarily the "morning" time, the text "Morning Sleep Report" can be displayed along with it. Similarly, if the time period to which the sleep data included in the user's sleep information belongs is primarily the "daytime" time, the text "Daytime Sleep Report" can be displayed along with it; if it belongs to the "evening" time, the text "Evening Sleep Report" can be displayed along with it; if it belongs to the "night" time, the text "Nighttime Sleep Report" can be displayed along with it; and if it belongs to the "dawn" time, the text "Dawn Sleep Report" can be displayed along with it.

[0583] On the other hand, the text displaying the time period information from which the user's sleep information was obtained is not limited to the examples above. It may include keywords such as "dawn" if the time period to which the user woke up was early morning, "morning" if it was during the daytime, "daytime" if it was during the evening, and "night" if it was during the nighttime.

[0584] Explanation of the flowchart in Figure 10

[0585] Figure 10 is a flowchart of a method for generating and providing one or more graphic user interfaces that show information about a user's sleep, according to one embodiment of the present invention.

[0586] As shown in Figure 10, according to one embodiment of the present invention, a method for generating one or more graphic user interfaces that show information about sleep may include the steps of: acquiring sleep information from a user (S100); converting the acquired sleep information into information that includes changes in frequency components over time (S120); acquiring sleep state information (S140); generating a graphic user interface (S160); and providing a graphic user interface (S180).

[0587] The sleep information acquired at this stage may include environmental sensing information or sleep acoustic information.

[0588] Furthermore, according to one embodiment of the present invention, a method for generating one or more graphic user interfaces that display information about sleep may further include a sleep log storage step (not shown) of storing sleep log information associated with an account assigned to the user in memory.

[0589] Furthermore, according to one embodiment of the present invention, the step (S120) of converting the acquired sleep information into information that includes changes in the frequency components over time may include a step of performing preprocessing on the raw acoustic information in the time domain or the information in the frequency domain.

[0590] Alternatively, according to one embodiment of the present invention, the step of converting acquired sleep information into information in the frequency domain (S120) may include a step of converting acoustic information into spectrogram information. In this case, the step of applying a Mel scale to the spectrogram to convert it into a Mel spectrogram may be further included.

[0591] Furthermore, the step of acquiring sleep state information according to one embodiment of the present invention (S140) may include the step of extracting sleep state information corresponding to each piece of information obtained by dividing the frequency domain information, spectrogram, or mel spectrogram into 30-second units.

[0592] Furthermore, according to one embodiment of the present invention, the step of generating a graphic user interface (S160) may include the step of generating date information obtained based on the user's sleep information, which includes the user's time of falling asleep and / or waking up.

[0593] In embodiments of the present invention, the step of generating date information for acquiring sleep information may further include the step of generating time zone information for acquiring sleep information based on the time zone information to which the wake-up time belonged. Here, the time zone information may be determined by a previously established definition.

[0594] AI alarm screen

[0595] Figure 19a is a diagram showing a graphic user interface that displays information about the sleep of a user whose alarm function according to the present invention has been activated.

[0596] Figure 19b is a diagram showing a graphic user interface that displays information about the sleep of a user whose alarm function according to the present invention is not activated.

[0597] As illustrated, the user can trigger the alarm function, and the user may be provided with an alarm function trigger activation state user interface 19100 and an alarm function trigger deactivation state user interface 19200.

[0598] For example, a user can trigger an alarm function, and if the user triggers the alarm function in a first direction, the user may be provided with an alarm function trigger activation state user interface 19100, and if the user triggers the deactivation of the alarm function in a second direction, the user may be provided with an alarm function trigger deactivation state user interface 19200.

[0599] To give a more specific example, if an activated state user interface is provided, when a trigger ring is input, it switches to the alarm function state deactivated user interface; if no trigger ring is input, the user may be provided with the alarm function trigger activated state user interface 19100; and if the user triggers the alarm function deactivated state, the user may be provided with the alarm function trigger deactivated state user interface 19200, but is not limited to this.

[0600] As illustrated, the graphic user interface generation stage may include a graphic user interface generation stage to start sleep measurement along with the activation of the alarm function if the trigger area for activating the notification function is activated, and a graphic user interface generation stage to start sleep measurement along with the deactivation of the alarm function if the trigger area for activating the notification function is not activated.

[0601] In other words, when a user triggers and activates the alarm function, an alarm window user interface 19101 may be provided in the state where the alarm function is triggered and activated, and the alarm window user interface 19101 in the state where the alarm function is triggered and activated can provide a wake-up time input notification phrase user interface 19110 in the state where the alarm function is triggered and activated.

[0602] For example, the wake-up time input notification phrase user interface 19110 in the alarm function trigger activated state may be in the form of providing the phrase "When do you want to wake up?". To give yet another example, the wake-up time input notification phrase user interface 19110 in the alarm function trigger activated state may be in the form of providing the phrase "What time do you want to wake up?", or the phrase "What time do you want to wake up?", or even a form like "When should I wake you up?", but this is just one example among many and is not limited to these.

[0603] As shown in Figure 19a, when a user triggers and activates the alarm function, an alarm window user interface 19101 may be provided in the alarm function trigger activation state, and the alarm window user interface 19101 in the alarm function trigger activation state may provide a wake-up schedule range user interface 19120 that includes the desired wake-up time information in the alarm function trigger activation state, or a slide area user interface 19130 that allows input of the desired wake-up time information in the alarm function trigger activation state.

[0604] According to one embodiment of the present invention, a slide area user interface 19130 provided for inputting desired wake-up time information when the alarm function trigger is activated may include a slide area bar user interface 19131 provided for inputting desired wake-up time information when the alarm function trigger is activated, and the user can slide the slide area user interface 19130 provided for inputting desired wake-up time information when the alarm function trigger is activated to select the user's desired wake-up time.

[0605] According to one embodiment of the present invention, a graphic user interface generation step including a slide area for selecting desired wake-up time information may include a graphic user interface generation step in which, when the slide area is slid in a first direction, the time of the desired wake-up time information is delayed, and when the slide area is slid in a second direction, the time of the desired wake-up time information is advanced.

[0606] To give a specific example, if the user slides down the slide area user interface 19130, which is provided to allow the user to input information about their desired wake-up time when the alarm function trigger is activated, the desired wake-up time may be set to an earlier time, and if the user slides up the slide area user interface 19130, which is provided to allow the user to input information about their desired wake-up time when the alarm function trigger is activated, the desired wake-up time may be set to a later time, but is not limited to this.

[0607] To give a more specific example, as shown in Figure 19a, if the user's desired wake-up time is 8:36 a.m., the slide area bar user interface 19131, which is provided to allow input of the desired wake-up time information when the alarm function trigger is activated, may be provided to point to 8:36 a.m. The wake-up range user interface 19120, which includes the desired wake-up time information when the alarm function trigger is activated, may indicate a range from 8:06 a.m. to 8:36 a.m. if it is set to a predetermined range from 8:36 a.m., for example, 30 minutes.

[0608] According to one embodiment of the present invention, a wake-up schedule range user interface 19120, which includes information on the desired wake-up time when the alarm function trigger is activated, can indicate the interval to which the user's wake-up time is predicted to belong.

[0609] According to one embodiment of the present invention, when a user triggers and activates an alarm function, an alarm window user interface 19101 may be provided in the state in which the alarm function is triggered and activated, and the alarm window user interface 19101 in the state in which the alarm function is triggered and activated may include a user interface 19140 that provides sleep prediction time information in the state in which the alarm function is triggered and activated.

[0610] Specifically, the graphic user interface generation stage may include a stage that, upon receiving desired wake-up time information via a slide area, displays an estimated sleep duration based on the difference between the received desired wake-up time and the current time. For example, as shown in Figure 19a, if the desired wake-up time is 8:36 AM and the current time is 12:39 AM, a user interface 19140 may be provided that displays estimated sleep duration information in the alarm function trigger activated state, with a difference of 8 hours and 57 minutes between the desired wake-up time of 8:36 AM and the current time of 12:39 AM.

[0611] According to yet another embodiment of the present invention, the graphic user interface generation step may include, upon receiving wake-up time information via a slide area user interface 19130 provided to enable the user to input wake-up time information when the alarm function trigger is activated, a graphic user interface generation step that indicates the predicted sleep time based on any one of the following: predicted wake-up time information or predicted sleep time or a combination of predicted sleep time and sleep efficiency, inferred from the received user sleep state information.

[0612] Specifically, sleep efficiency may be (the user's actual sleep time / the time from the start of sleep measurement until the user wakes up) * 100, and more specifically, it may be {the time calculated by subtracting the wake phase from the sum of the NREM phase and REM phase of the user's sleep) / the time from the start of sleep measurement until the user wakes up} * 100.

[0613] According to one embodiment of the present invention, the sleep prediction time provided to the slide area user interface 19130, which is provided to allow the user to input information about the desired wake-up time when the alarm function trigger is activated, may include the time from the current time to the time when REM sleep is predicted to occur, or to a time after a predetermined period of time has elapsed, based on the user's sleep state information, within the wake-up range which includes the desired wake-up time information when the alarm function trigger is activated.

[0614] According to yet another embodiment of the present invention, the sleep prediction time provided to the slide area user interface 19130, which is provided so that the user can input information about the desired wake-up time when the alarm function trigger is activated, may be a time calculated by multiplying the sleep efficiency by the time from the current time to the time when REM sleep is predicted to occur or to a predetermined time after that time, based on the user's sleep state information within the planned wake-up range which includes the information about the desired wake-up time when the alarm function trigger is activated.

[0615] As illustrated, when the user triggers and activates the alarm function, an alarm window user interface 19101 may be provided in the state where the alarm function is triggered and activated, and simultaneously, a user interface 19150 may be provided to start sleep measurement in the state where the alarm function is triggered and activated.

[0616] To give a specific example, the user interface 19150 that initiates sleep measurement when the alarm function trigger is activated may provide the phrase "Go to sleep." It may also provide the phrase "Start sleep measurement," or the phrase "Shall we go to sleep?", but this is merely an example and not limited to these.

[0617] As shown in Figure 19b, the user can trigger the alarm function, and the user may be provided with an alarm function trigger activation state user interface 19100 or an alarm function trigger deactivation state user interface 19200.

[0618] For example, a user can trigger an alarm function, and if the user triggers the alarm function in a first direction, the user may be provided with an alarm function trigger activation state user interface 19100, and if the user triggers the deactivation of the alarm function in a second direction, the user may be provided with an alarm function trigger deactivation state user interface 19200.

[0619] If an alarm function trigger deactivation state user interface 19200 is provided, the alarm function is not triggered, and an information provision user interface 19201 corresponding to sleep measurement in a state where the wake-up time is not determined may be provided.

[0620] To give a specific example, the information-providing user interface 19201 corresponding to sleep measurement when the wake-up time is not set may be a phrase that provides information that sleep measurement results can only be provided after sleeping for a certain amount of time or more, for example, a phrase such as "You can receive the report if you sleep for 30 minutes or more."

[0621] Another example would be a phrase like, "Why not try using an AI alarm for your next sleep?" This is merely an example and not the only one that is allowed.

[0622] In one embodiment of the present invention, if an alarm function trigger deactivation state user interface 19200 is provided, a user interface 19250 for initiating sleep measurement in the alarm function trigger deactivation state may also be provided.

[0623] To give a specific example, the user interface 19250 for initiating sleep measurement when the alarm function trigger is deactivated may be provided in the form of a phrase such as "Go to sleep without an alarm," or it may be provided in the form of a phrase such as "Turn off the alarm and go to sleep." Alternatively, the user interface 19250 for initiating sleep measurement when the alarm function trigger is deactivated may be provided in the form of a phrase such as "Turn off the alarm and start sleep measurement," and this is merely an example and not limited to this.

[0624] Figure 20a is a diagram showing a graphic user interface indicating that sleep measurement is in progress with the alarm function according to the present invention activated. Figure 20b is a diagram showing a graphic user interface indicating that sleep measurement is in progress with the alarm function according to the present invention deactivated.

[0625] According to one embodiment of the present invention, when a user triggers and activates the alarm function, an alarm window user interface 19101 in the alarm function trigger activation state may be provided, and at the same time, a user interface 19150 for starting sleep measurement in the alarm function trigger activation state may be provided.

[0626] As a result, when an input signal is received from the user interface 19150 to initiate sleep measurement in the alarm function trigger activated state, a user interface 20310 indicating that the alarm function trigger is activated during sleep measurement in the alarm function trigger activated state shown in Figure 20a may be provided.

[0627] To give a specific example, user interface 20310 indicating that the alarm function trigger is activated during sleep measurement may be provided in the form of a phrase that informs the user that the alarm function is activated, such as "The AI ​​will definitely wake you up!". To give yet another example, user interface 20310 indicating that the alarm function trigger is activated during sleep measurement may be provided in the form of a phrase such as "The AI ​​alarm function is activated!", or "The AI ​​will help you wake up refreshed!", or "I'll wake you up, so sleep soundly!", and this is merely an example and is not limited to these.

[0628] According to one embodiment of the present invention, when a user triggers and activates the alarm function, an alarm window user interface 19101 in the state where the alarm function is triggered and activated may be provided, and at the same time, a user interface 19150 for starting sleep measurement in the state where the alarm function is triggered and activated may be provided, and at the same time, a user interface 20320 for indicating a planned wake-up range including desired wake-up time information during sleep measurement in the state where the alarm function is triggered and activated may be provided.

[0629] According to the present invention, the user interface 20320, which indicates a wake-up schedule range including desired wake-up time information when the alarm function trigger is activated, can provide alarm schedule information within the same range as the wake-up schedule range user interface 19120, which includes desired wake-up time information when the alarm function trigger is activated during sleep measurement.

[0630] To give a specific example, if the wake-up schedule range user interface 19120, which includes information on the desired wake-up time when the alarm function trigger is activated, provides the range "between 8:06 AM and 8:36 AM", then the wake-up schedule range user interface 20320, which indicates the wake-up schedule range including information on the desired wake-up time during sleep measurement when the alarm function trigger is activated, can provide the phrase "Alarm between 8:06 AM and 8:36 AM".

[0631] To give yet another example, if the wake-up range user interface 19120, which includes information on the desired wake-up time when the alarm function trigger is activated, provides a range of "between 8:06 AM and 8:36 AM", then the wake-up range user interface 20320, which shows the wake-up range including information on the desired wake-up time during sleep measurement when the alarm function trigger is activated, can provide the phrase "The alarm will sound between 8:06 AM and 8:36 AM!".

[0632] According to one embodiment of the present invention, when a user triggers and activates the alarm function, an alarm window user interface 19101 in the alarm function trigger activation state may be provided, simultaneously a user interface 19150 for starting sleep measurement in the alarm function trigger activation state may be provided, simultaneously a user interface 20320 indicating a planned wake-up range including desired wake-up time information during sleep measurement in the alarm function trigger activation state may be provided, and simultaneously a user interface 20340a indicating the first wave of the user's sleep acoustic information during sleep measurement in the alarm function trigger activation state may be provided.

[0633] According to one embodiment of the present invention, the step of generating a graphic user interface including a screen indicating that sleep measurement is in progress includes a step of generating a graphic user interface indicating that the trigger area for activating the alarm function is not activated, further includes a step of generating a graphic user interface indicating a range of scheduled wake-up times, and may further include a step of generating a graphic user interface indicating the wave patterns of the user's sleep acoustic information.

[0634] Specifically, the user interface 20340a, which displays the first wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is activated, may change amplitude in real time depending on the intensity of the user's sleep acoustics. To give a specific example, the user interface 20340a, which displays the second wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is activated, can move in any direction in real time, and its amplitude can change depending on the intensity of the user's sleep acoustics.

[0635] According to one embodiment of the present invention, when a user triggers and activates the alarm function, an alarm window user interface 19101 may be provided in the state where the alarm function is triggered and activated, and at the same time, a user interface 19150 may be provided to start sleep measurement in the state where the alarm function is triggered and activated, and at the same time, a user interface 20320 may be provided that shows a planned wake-up range including desired wake-up time information during sleep measurement in the state where the alarm function is triggered and activated, and at the same time, a user interface 20340a may be provided that shows the first wave of the user's sleep acoustic information during sleep measurement in the state where the alarm function is triggered and activated, and a user interface 20330 may be provided to interrupt sleep measurement during sleep measurement in the state where the alarm function is triggered and activated.

[0636] As shown in Figure 20a, Figure 20a(a) is a diagram showing that a user interface 20340a is provided that displays the first wave of the user's sleep acoustic information during sleep measurement in the state where the alarm function trigger is activated, and Figure 20a(b) is a diagram showing a user interface 20340b that displays the second wave of the user's sleep acoustic information during sleep measurement in the state where the alarm function trigger is activated.

[0637] User interface 20340a, which shows the first wave of the user's sleep acoustic information during sleep measurement with the alarm function trigger activated, and user interface 20340b, which shows the second wave of the user's sleep acoustic information during sleep measurement with the alarm function trigger activated, may be provided differently depending on the model of the user's terminal. User interface 20340b, which shows the second wave of the user's sleep acoustic information during sleep measurement with the alarm function trigger activated, is characterized in that the amplitude of all multiple waves changes in the same way and decreases depending on the magnitude of the user's sleep acoustics, and may be provided differently depending on the model of the user's terminal, specifically the model of the smartphone.

[0638] According to the present invention, the step of generating a graphic user interface including a screen indicating that sleep measurement is in progress may further include a step of generating a graphic user interface that can terminate sleep measurement.

[0639] To give a specific example, the user interface 20330 for interrupting sleep measurement while the alarm function trigger is activated may be provided in the form of indicating the phrase "Wake up," or the user interface 20330 for interrupting sleep measurement while the alarm function trigger is activated may be provided in the form of indicating the phrase "Get up." It may also be provided in the form of indicating the phrase "Go get up," or the phrase "End sleep measurement," but this is merely an example and is not limited to these.

[0640] As shown in Figure 20b, according to one embodiment of the present invention, if the user does not trigger and activate the alarm function, a user interface 20250 may be provided to start sleep measurement in the state where the alarm function trigger is deactivated. If an input to start sleep measurement is received via the user interface 20250, a user interface 20610 may be provided that displays a phrase or emoticon, or a phrase and emoticon, indicating that sleep measurement is in progress while the alarm function trigger is not activated.

[0641] According to one embodiment of the present invention, the user interface 20610 that displays a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress may be provided in the form of a sleep-inducing phrase such as "Close your eyes and get some rest ^_^". Alternatively, the user interface 20610 that displays a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress may be provided in the form of the phrase "You will fall into a peaceful sleep", or the user interface 20610 that displays a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress may be provided in the form of the phrase "Tonight will be a peaceful night", but this is merely an example and is not limited to these.

[0642] As shown in Figure 20b, according to one embodiment of the present invention, if the user does not trigger and activate the alarm function, a user interface 19250 may be provided to start sleep measurement in the state where the alarm function trigger is deactivated. If an input to start sleep measurement is received via the user interface 19250, a user interface 20610 may be provided that shows a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress while the alarm function trigger is not activated. Simultaneously, a user interface 20640a may be provided that shows the first wave of the user's sleep acoustic information while sleep measurement is in progress while the alarm function trigger is not activated.

[0643] According to one embodiment of the present invention, the user interface 20640a, which displays the first wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is not activated, may change amplitude in real time depending on the intensity of the user's sleep acoustics. Specifically, the user interface 20640a, which displays the first wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is not activated, may move in any direction in real time, and its amplitude may change depending on the intensity of the user's sleep acoustics.

[0644] As shown in Figure 20b, according to one embodiment of the present invention, if the user does not trigger and activate the alarm function, a user interface 19250 may be provided to start sleep measurement in the state where the alarm function trigger is deactivated. If an input to start sleep measurement is received via the user interface 19250, a user interface 20610 may be provided that shows a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress while the alarm function trigger is not activated. Simultaneously, a user interface 20640a may be provided that shows the first wave of the user's sleep acoustic information during sleep measurement while the alarm function trigger is not activated. An information provision user interface 20620 may be provided that corresponds to sleep measurement when the wake-up time has not been determined.

[0645] According to the present invention, the graphic user interface generation step may include a graphic user interface generation step that provides information corresponding to sleep where the wake-up time has not been determined, if the trigger area for activating the notification function is not activated.

[0646] To give a specific example, the information provision user interface 20620 corresponding to sleep measurement when the wake-up time is not set may be provided in the form of a phrase that provides information for receiving the sleep measurement results, such as "You can receive the report if you sleep for 30 minutes or more." Alternatively, the information provision user interface 20620 corresponding to sleep measurement when the wake-up time is not set may be provided in the form of information necessary when the wake-up time is not set, such as "The best amount of sleep is 7 hours!" Or, it may be provided in the form of a phrase such as "Tomorrow is the weekend! You can sleep even more than on weekdays!", but this is merely an example and is not limited to these.

[0647] As shown in Figure 20b, Figure 20b(a) is a diagram showing that a user interface 20640a is provided that displays the first wave of sleep acoustic information of the user during sleep measurement when the alarm function trigger is not activated, and Figure 20a(b) is a diagram showing the user interface 20640b that displays the second wave of sleep acoustic information of the user during sleep measurement when the alarm function trigger is not activated.

[0648] User interface 20640a, which shows the first wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is not activated, and user interface 20640b, which shows the second wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is not activated, may be provided differently depending on the model of the user's terminal. User interface 20640b, which shows the second wave of the user's sleep acoustic information during sleep measurement when the alarm function trigger is not activated, is characterized in that the amplitude of all multiple waves changes in the same way and decreases depending on the magnitude of the user's sleep acoustics, and may be provided differently depending on the model of the user's terminal, specifically the model of the smartphone.

[0649] As shown in Figure 20b, according to one embodiment of the present invention, if the user does not trigger and activate the alarm function, a user interface 19250 may be provided to start sleep measurement in the state where the alarm function trigger is deactivated. If an input to start sleep measurement is received via the user interface 19250 to start sleep measurement in the state where the alarm function trigger is deactivated, a user interface 20610 may be provided that shows a phrase or emoticon or a phrase and emoticon indicating that sleep measurement is in progress while the alarm function trigger is not activated. Simultaneously, a user interface 20640a may be provided that shows the first wave of the user's sleep acoustic information during sleep measurement while the alarm function trigger is not activated. An information provision user interface 20620 corresponding to sleep measurement when the wake-up time has not been determined may be provided. A user interface 20630 may be provided for interrupting sleep measurement during sleep measurement while the alarm function trigger is not activated.

[0650] According to the present invention, the user interface 20630 for interrupting sleep measurement during sleep measurement when the alarm function trigger is not activated may be provided in a form indicating the phrase "Wake up", or the user interface 20630 for interrupting sleep measurement during sleep measurement when the alarm function trigger is not activated may be provided in a form indicating the phrase "Get up". It may also be provided in a form indicating the phrase "Go get up", or in a form indicating the phrase "End sleep measurement", but this is merely an example and is not limited thereto.

[0651] Embodiments related to Figure 21

[0652] Figure 21 is a diagram showing a graphic user interface according to the present invention that provides information to allow users to check their predicted sleep time if their predicted sleep time is longer than a predetermined time.

[0653] According to the present invention, the graphic user interface generation step may include a step in which a pop-up window is displayed so that the predicted sleep time can be confirmed if the predicted sleep time exceeds a predetermined time.

[0654] To give a specific example, if the time shown in the user interface 19140, which provides sleep prediction time information when the alarm function trigger is activated, exceeds 12 hours, and the sleep prediction time is greater than or equal to a predetermined time, a pop-up window user interface 21700 for confirming the sleep prediction time can be provided. However, this is not limited to cases where the time exceeds 12 hours; even if the time exceeds a predetermined time, if the sleep prediction time is greater than or equal to a predetermined time, the pop-up window user interface 21700 for confirming the sleep prediction time can be provided.

[0655] According to one embodiment of the present invention, the pop-up window user interface 21700 for checking the predicted sleep time may include a user interface 21710 that explains the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time. For example, if the predicted sleep time is longer than a predetermined time, the user interface 21710 that explains the pop-up window for checking the predicted sleep time may provide the phrase "Check sleep time". Alternatively, if the predicted sleep time is longer than a predetermined time, the user interface 21710 that explains the pop-up window for checking the predicted sleep time may provide information that guides the user to check the set desired wake-up time again, such as providing a phrase like "Your predicted sleep time is a little long!".

[0656] According to one embodiment of the present invention, the pop-up window user interface 21700 for checking the predicted sleep time may include a user interface 21710 that explains the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time, and may include a user interface 21720 that displays a phrase asking about the desired wake-up time in the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time.

[0657] According to the present invention, the user interface 21720 that displays a phrase asking about the desired wake-up time in a pop-up window for checking the predicted sleep time may be provided in the form of the phrase, "Your predicted sleep time is 00:00. Is the time you set correct?" Alternatively, the user interface 21720 that displays a phrase asking about the desired wake-up time in a pop-up window for checking the predicted sleep time may be provided in the form of the phrase, "Your predicted sleep time is a little long! Is the desired alarm time 00:00 AM correct?" Alternatively, it may be provided in the form of the phrase, "Your predicted sleep time exceeds 00 hours. It looks like you want to get plenty of sleep today!" However, these are merely examples and the invention is not limited to these.

[0658] According to one embodiment of the present invention, the pop-up window user interface 21700 for checking the predicted sleep time may include a user interface 21710 that explains the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time, and may further include a user interface 21730 for resetting the desired wake-up time in the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time, or a user interface 21740 for maintaining the existing desired wake-up time in the pop-up window for checking the predicted sleep time if the predicted sleep time is longer than a predetermined time.

[0659] According to the present invention, the user interface 21730 for resetting the desired wake-up time in the pop-up window for checking the predicted sleep time may be provided in the form of the phrase "reset," or it may be provided in the form of the phrase "go reset." It may also be provided in the form of the phrase "reset alarm," but this is merely an example.

[0660] According to the present invention, the user interface 21740 for maintaining the desired wake-up time already set in the pop-up window for confirming the predicted sleep time may be provided in the form of the phrase "Yes, that's correct," or it may be provided in the form of the phrase "The time is correct!" or it may be provided in the form of the phrase "Yes, I'm going to sleep," but it is not limited to these.

[0661] As shown in Figure 22, when the user triggers and activates the alarm function, an alarm window user interface 19101 may be provided in the alarm function trigger activation state, and the alarm window user interface 19101 in the alarm function trigger activation state may include a user interface 22140 that provides sleep prediction time information in the alarm function trigger activation state, and if the sleep prediction time is less than a predetermined time, an information provision user interface 22202 corresponding to sleep measurement may be provided.

[0662] To give a specific example, if the user's estimated sleep time is less than 30 minutes, for example, 3 minutes, the information provision user interface 22202 corresponding to sleep measurement may be provided in the form of the phrase, "You can receive the report if you sleep for just 27 more minutes."

[0663] Alarm delivery method

[0664] Figure 23 is a diagram illustrating a method for providing an alarm based on the user's sleep state information according to the present invention.

[0665] As shown in Figure 23, a method for providing an alarm based on a user's sleep state information may include a sleep information reception step (S2310) in which user sleep information including the user's sleep acoustic information is received from one or more sleep information sensor devices; a sleep state information acquisition step (S2320) in which user sleep state information including the user's average sleep time information is obtained based on the received user sleep information; a wake-up time information reception step (S2330) in which wake-up time information is received; an alarm time information generation step (S2340) in which alarm time information is generated based on the acquired sleep state information and the received wake-up time information; and an alarm sound provision step (S2350) in which an alarm sound is provided based on the generated alarm time information.

[0666] The following describes in detail the alarm time information generation step (S2340), which generates alarm time information based on the acquired sleep state information and the received desired wake-up time information.

[0667] According to the present invention, if sleep state information indicating that the user is in REM sleep is acquired in the sleep state information acquisition step (S2320) within a time range that includes the desired wake-up time information, the alarm time information generation step (S2340) can generate alarm time information based on the sleep state information.

[0668] Specifically, in a method for providing an alarm based on the user's sleep state information, the time range including the desired wake-up time information includes the time from a predetermined time before the desired wake-up time to the desired wake-up time, and if sleep state information indicating that the user is in REM sleep cannot be obtained within the time range, the alarm time information can be generated at the desired wake-up time.

[0669] For example, if the desired wake-up time is 8:36 AM, the range refers to the time from 8:36 AM (the desired wake-up time) to 30 minutes before a predetermined time, i.e., the range from 8:06 AM to 8:36 AM. If the user's REM sleep is not measured during the time from 8:06 AM to 8:36 AM, the alarm time information can be generated at 8:36 AM.

[0670] According to the present invention, a method for providing an alarm based on the user's sleep state information includes a time range that includes the time from a predetermined time before the desired wake-up time to the desired wake-up time, and if sleep state information indicating that the user is in REM sleep is acquired within the time range, the alarm time information can be generated at the time the sleep state information indicating that the user is in REM sleep is acquired.

[0671] For example, if the desired wake-up time is 8:36 AM, the range refers to the time from 8:36 AM (the desired wake-up time) to 30 minutes before a predetermined time, i.e., the range from 8:06 AM to 8:36 AM. If the user's REM sleep is measured at 8:10 AM within the time period from 8:06 AM to 8:36 AM, the alarm time information can be generated at 8:10 AM or after a predetermined time has elapsed from 8:10 AM.

[0672] Figure 24a is a diagram illustrating the case where the AI ​​alarm of the present invention is not set, via a hypnogram. Figure 24b is a diagram illustrating the case where the AI ​​alarm of the present invention is set, via a hypnogram.

[0673] As shown in Figure 24a, if an AI alarm is not set, even if the user's sleep status information indicates that the user is in REM sleep 30 minutes before the user's desired wake-up time, the wake-up will occur when the user is in normal sleep, i.e., during the NREM stage, even if the user's sleep status information indicates that the user is in REM sleep, because the wake-up process has passed the point in time when the REM sleep was indicated.

[0674] As shown in Figure 24b, when an AI alarm is set, if the user's sleep state information indicates that the user is in REM sleep 30 minutes before the user's desired wake-up time, the alarm time can be set to the user's REM sleep time, even if the user does not reach their desired wake-up time. However, it is also possible to set the alarm time to a time after a predetermined period of time from the REM sleep time.

[0675] According to yet another embodiment of the present invention, if the user's REM sleep persists 30 minutes before the time of the user's desired wake-up time information, and a point in time appears indicating that the user's sleep stage is a sleep stage other than REM sleep, that point in time can also be set as the alarm time information. To give a specific example, if the time of the user's desired wake-up time information is 8:30 a.m., and REM sleep is detected at 8:10 a.m., which is between 8:00 a.m. and 8:30 a.m., then if the user's sleep state information indicates normal sleep at 8:20 a.m., then the alarm time information can also be set to 8:20 a.m.

[0676] According to yet another embodiment of the present invention, if the user's sleep state information indicates that the user is in REM sleep 30 minutes before the user's desired wake-up time, and the confidence level of the sleep state information indicating that the user is in REM sleep becomes low, the alarm time information can be set at the point when the confidence level of the sleep state information becomes low.

[0677] Figure 25a Graphical user interface including a hypnogram

[0678] Figure 25a is a diagram showing a graphic user interface including a hypnogram according to one embodiment of the present invention.

[0679] A graphic user interface including a hypnogram according to an embodiment of the present invention may also display the phrase “sleep stages,” as shown in reference no. 25101.

[0680] According to one embodiment of the present invention, the hypnogram may include figures corresponding to each sleep stage, with the x-axis representing time (reference number 25114) and the y-axis representing sleep stage information.

[0681] According to Figure 25a, the sleep stage information according to one embodiment of the present invention comprises a total of four stages and may include multiple regions assigned to each sleep stage. For example, it may be represented by a region assigned to the wakefulness stage (reference number 25102), a region assigned to the REM sleep stage (reference number 25103), a region assigned to the light sleep (or "normal sleep") stage (reference number 25104), and a region assigned to the deep sleep stage (reference number 25105).

[0682] According to one embodiment of the present invention, the colors of the figures corresponding to each sleep stage information may be different from each other. For example, the color of the figure corresponding to the deep sleep stage may be a color in a series that is relatively darker than the other figures (reference number 25113). Alternatively, the color of the figure corresponding to the REM sleep stage may be a color that is relatively brighter than the other figures (reference number 25111).

[0683] Alternatively, according to embodiments of the present invention, the colors of the figures and backgrounds corresponding to each sleep stage information may be colors that are represented at different positions in the color space.

[0684] For example, according to one embodiment of the present invention, the background may be represented by a black color series, the figures corresponding to the deep sleep stage may be represented by a relatively dark blue color series, the figures corresponding to the light sleep or general sleep stage may be represented by a relatively light blue color series (reference number 25112), the figures corresponding to the REM sleep stage may be represented by a purple color series, and the figures corresponding to the wakefulness stage may be represented by a yellow color series.

[0685] According to embodiments of the present invention, the figures corresponding to sleep stage information may be figures that are represented discretely from one another.

[0686] In this invention, the term "discretely represented figures" may mean that each figure is not connected to the others by solid lines, dotted lines, or the like.

[0687] Furthermore, in one embodiment of the present invention, the discrete representation of figures means that, unlike the prior art, figures corresponding to different sleep stages may share only one intersection point.

[0688] Furthermore, the representation of figures discretely in one embodiment of the present invention may include the meaning that although each figure includes multiple figures corresponding to multiple sleep stages, at least one of the multiple figures is isolated from the rest.

[0689] A graphic user interface that displays information about sleep.

[0690] According to the present invention, a graphic user interface that displays information about the user's sleep may be provided.

[0691] Figures 30a to 30c are diagrams illustrating how a graphic user interface according to an embodiment of the present invention is displayed on various display units.

[0692] As shown in Figures 30a to 30c, according to embodiments of the present invention, the graphic user interface showing information about the user's sleep may be displayed on an electronic device or user terminal embodied in various forms of displays.

[0693] Hereinafter, a graphic user interface including a graph showing sleep-related information according to an embodiment of the present invention will be described with reference to the drawings.

[0694] As shown in Figures 31a to 31e, in conventional sleep measurement interfaces, hypnogram graphs sometimes displayed sleep stages as being connected to each other by lines such as dotted or solid lines, even when the shapes representing sleep stages were far apart.

[0695] For example, as can be seen from Figures 31a to 31e, in conventional techniques, when representing graphs showing sleep stages, changes between the first and second sleep stages were sometimes represented continuously rather than discretely.

[0696] Furthermore, as shown in Figure 31e, in order to represent the existence of continuity between figures even if there are no intersection points between them, a line (dotted or solid) was sometimes used to connect the point representing the end of the first figure with the point representing the beginning of the second figure.

[0697] However, representing hypnograms through such continuous graphs risked creating the misconception that the transition from the first to the second sleep stage required passing through other sleep stages. While the transition patterns between sleep stages are crucial in sleep stage analysis, continuous hypnogram representations can sometimes fail to accurately depict these transition patterns.

[0698] According to the present invention, unlike the prior art, when the figures corresponding to each sleep stage information are represented discretely by one embodiment of the present invention, it is possible to reduce the misconception that one must pass through other sleep stages in the process of transitioning from the first sleep stage to the second sleep stage.

[0699] Furthermore, since the transition from the first to the second sleep stage does not necessarily require passing through other sleep stages, a discrete representation of the shapes corresponding to each sleep stage has the advantage of allowing for an intuitive understanding of the transitions between sleep stages.

[0700] For example, referring to the figure shown in reference number 25110 in Figure 25a (the figure corresponding to the wakefulness stage), the sleep transitions from the light sleep (normal sleep) stage to the wakefulness stage and then back to the light sleep (normal sleep) stage. However, when such transitions or changes in sleep stages are represented in a "continuous" hypnogram graph, it can lead to the misconception that the REM sleep stage must be passed through when the sleep stages are transitioned. With the present invention, however, when represented in a "discrete" hypnogram graph, such misconceptions can be reduced. Furthermore, when hypnograms discretely represent the information of each sleep stage, they clearly show the frequency of each sleep stage. This allows for direct confirmation of the frequency of sleep stages belonging to a specific interval, making it easier to grasp the relative importance of each sleep stage. This can be useful in understanding the characteristics and abnormalities of sleep patterns.

[0701] On the other hand, according to one embodiment of the present invention, the figure corresponding to the sleep stage information may be represented as at least one of the following shapes: trapezoid, isosceles trapezoid, kite, parallelogram, rhombus, rectangle, square, or other common quadrilateral. Preferably, it may be represented as a rectangle.

[0702] As shown in Figure 25a, multiple sleep stage information may be represented by multiple rectangles. In this case, at least one of the multiple rectangles may be isolated from the other rectangles, as shown by the rectangle reference number 25110 in Figure 25a. In the embodiment shown in Figure 25a, the rectangle 25110, which corresponds to the wakefulness stage of the sleep stage information, is isolated from the other rectangles.

[0703] When performing sleep analysis, the wakefulness phase may occur at least once during sleep. When representing sleep stage information in a hypnogram according to an embodiment of the present invention, the rectangle representing the wakefulness phase can be displayed separately from the rectangles representing other sleep stages. This display has the advantage of allowing sleep analysis by clearly distinguishing the occurrence of the wakefulness phase during sleep. According to the interface of the prior art, when wakefulness stage information is frequently acquired during sleep, each sleep stage and the wakefulness phase during sleep are connected by a line (solid or dotted line), which may result in the hypnogram being represented in a relatively unclear manner, and the occurrence of the wakefulness phase may not be clearly distinguishable.

[0704] However, when sleep stage information is shown using discrete and / or separated figures according to the present invention, the frequency of the awakening stage during sleep can be clearly confirmed, thus eliminating the unclear errors of the prior art.

[0705] Furthermore, according to one embodiment of the present invention, the regions assigned to the wakefulness stage, REM sleep stage, light sleep stage, and deep sleep stage may be arranged and displayed in order from top to bottom (see reference numbers 25102 to 25105).

[0706] On the other hand, as shown in Figure 31c or Figure 31d, even if the figure indicating the wakefulness stage was displayed separately from the figures indicating other sleep stages, the figures corresponding to the sleep stages were displayed continuously in the area allocated to other sleep stages, making it difficult to clearly understand which sleep stage was occurring at that time.

[0707] For example, referring to Figure 31c, while multiple figures indicating the wakefulness stage are displayed separately from figures corresponding to different sleep stages during the overall sleep time, the graph also displays areas indicating the REM sleep stage, Light sleep stage, or Deep sleep stage at that point in time. This presents a problem in that it is not clear whether the person is asleep or awake at that point in time.

[0708] On the other hand, in the interface according to the embodiment of the present invention, if at least one of the figures corresponding to multiple sleep stages is isolated from the remaining figures, it is possible to clearly understand which sleep stage has occurred at that time by ensuring that no figures exist in the areas assigned to other sleep stages.

[0709] Furthermore, in the interface according to the embodiment of the present invention, the figures corresponding to multiple sleep stages are represented by figures of the same shape (for example, rectangles), and at least one of these figures may be isolated from the other figures. In such an embodiment, by ensuring that no figures exist in the areas assigned to other sleep stages represented by figures of the same shape, it is possible to clearly understand which sleep stage has occurred at that time.

[0710] Furthermore, among the multiple figures corresponding to sleep stages included in the interface according to the embodiment of the present invention, at least one figure corresponding to a sleep stage may have the same form as the figures corresponding to other sleep stages.

[0711] Furthermore, according to embodiments of the present invention, the figure corresponding to at least one sleep stage may be displayed separately from other figures corresponding to the same sleep stage.

[0712] According to an embodiment of the present invention, as shown in Figure 25a, when an awakening stage occurs, the figure corresponding to the sleep stage is displayed only in the area of ​​reference number 25102, and not in the areas allocated to the remaining sleep stages (reference numbers 25103 to 25105). Therefore, the user can clearly understand which sleep stage (specifically, the awakening stage) occurred at that time simply by looking at the hypnogram graph according to the present invention.

[0713] Furthermore, as shown in Figure 25a, according to one embodiment of the present invention, when the word indicating the light sleep stage is displayed in Korean, it may be displayed as "light sleep" or "normal sleep." Users who are not professionally familiar with sleep stages may be misled by the word "light sleep" into believing that they did not sleep properly during their sleep period. To address this, displaying it as "normal sleep" may reduce such misunderstandings (reference number 25104).

[0714] According to one embodiment of the present invention, the figures corresponding to each sleep stage may be displayed only in the multiple areas assigned to each piece of sleep stage information (see reference numbers 25102 to 25105). Through this display method, sleep stage information during sleep time can be clearly distinguished and represented, allowing the user to distinguish the sleep stage information during sleep time and accurately understand the graph of sleep stage information.

[0715] According to one embodiment of the present invention, there may be boundaries between multiple regions assigned to each sleep stage information, and such boundaries may be represented by dots or solid lines (reference no. 25108). The lines indicating the boundaries may be straight or curved.

[0716] Furthermore, according to one embodiment of the present invention, the lines indicating the boundaries of multiple regions assigned to each sleep stage information may be displayed with different colors or brightness levels. Specifically, the boundary line separating the wakefulness stage region from the remaining sleep stage regions may be displayed in a relatively brighter color.

[0717] According to one embodiment of the present invention, each of the multiple regions assigned to sleep stage information may have a height assigned to each region.

[0718] Furthermore, according to one embodiment of the present invention, the heights assigned to multiple regions, each assigned to sleep stage information, may be the same or different from each other. Specifically, the height assigned to the region assigned to wakefulness stage information may be different from the heights assigned to other regions (see reference no. 25102).

[0719] According to one embodiment of the present invention, there may be a pitch assigned between a plurality of regions assigned to each of the sleep stage information.

[0720] According to one embodiment of the present invention, the figures corresponding to each sleep stage information may be represented discretely based on the pitch assigned between multiple regions assigned to each sleep stage information. Such discrete representation allows the user to divide the sleep stage information during sleep time and accurately understand the graph of sleep stage information.

[0721] As shown in Figure 25a, according to one embodiment of the present invention, at least one piece of information from the time of falling asleep and the time of waking up may be displayed together in the hypnogram. In this case, at least one piece of information from the time of falling asleep and the time of waking up may be displayed as time information on the x-axis (reference numbers 25106 and 25107). In this way, when at least one piece of information from the time of falling asleep and the time of waking up is displayed together in the hypnogram, the time interval from the time of falling asleep to the time of waking up can be easily grasped by looking only at the hypnogram. According to one embodiment of the present invention, when at least one piece of information from the time of falling asleep and the time of waking up is displayed as time information on the x-axis, it may be displayed connected to a figure and line corresponding to each piece of sleep stage information. According to the embodiment of the present invention, the line connecting at least one piece of information from the time of falling asleep and the time of waking up to the figure corresponding to the sleep stage information may be a solid line or a dotted line, a straight line or a curved line.

[0722] Furthermore, according to one embodiment of the present invention, as shown in Figure 25a, comment information based on sleep stage information or sleep state information may be displayed together with a hypnogram graph showing sleep stages (reference number 25109).

[0723] Figure 25d is a drawing showing a graphic user interface including a hypnogram according to yet another embodiment of the present invention.

[0724] According to one embodiment of the present invention, if a person is unable to sleep at all after starting sleep measurement, the hypnogram may only display shapes corresponding to the wakefulness stage, as shown in Figure 25d.

[0725] According to one embodiment of the present invention, the hypnogram graph may be generated in real time during sleep.

[0726] According to the present invention, sleep information, including the user's sleep acoustics, can be acquired in real time, and the acquired sleep acoustic information can be immediately converted into a spectrogram. The converted spectrogram can be immediately used as input to a sleep analysis model to analyze sleep stages, thereby enabling the generation of a hypnogram graph showing sleep stage information in real time simultaneously with the sleep stage analysis.

[0727] Figure 25b is a graph showing the time percentage for each sleep stage.

[0728] Figure 25b is a graph showing the time percentage for each sleep stage as measured by one embodiment of the present invention.

[0729] A graphic user interface including a graph displaying the time percentage for each sleep stage according to an embodiment of the present invention may also display the phrase "My Sleep at a Glance," as shown in reference no. 25201.

[0730] The proportion of time for each sleep stage according to the present invention may be calculated as the ratio of total sleep time to the time corresponding to each sleep stage. Specifically, total sleep time may be calculated as the time from falling asleep to waking up. The time corresponding to each sleep stage may be calculated based on the sleep stage inferred by an artificial intelligence model according to an embodiment of the present invention.

[0731] For example, as shown in reference no. 25203 in Figure 25b, if the total sleep time from falling asleep to waking up is 7 hours and 22 minutes, and the period inferred to be the general sleep (light sleep) stage is 4 hours and 34 minutes, the period inferred to be the REM sleep stage is 1 hour and 37 minutes, the period inferred to be the deep sleep stage is 58 minutes, and the period inferred to be the wakefulness stage is 13 minutes, then the proportion of each sleep stage may be displayed as a percentage of the time corresponding to each period divided by the total sleep time. However, the numerical values ​​for the specific times mentioned above are merely examples, and the present invention is not limited thereto.

[0732] Furthermore, in the graphic user interface according to the embodiment of the present invention, time information corresponding to each sleep stage and the proportion of each sleep stage may be displayed side by side (reference no. 202). Also, as shown in reference no. 25204 to 25207 in Figure 25b, the time proportion corresponding to each sleep stage can be visually represented using predetermined shapes.

[0733] Here, the colors of the figures corresponding to each sleep stage may be the same as the colors of the figures corresponding to the sleep stage information shown in the hypnogram in Figure 25a.

[0734] Furthermore, according to one embodiment of the present invention, as shown in reference numerals 25202 and 25204 to 25207 of Figure 25b, the figures corresponding to each sleep stage can be displayed in descending order of the time percentage corresponding to each sleep stage. However, according to yet another embodiment, the order in which the sleep stages are displayed can be changed.

[0735] For example, the system may be configured so that the deep sleep stage is displayed first and the wakefulness stage is displayed last. Or, the system may be configured so that the wakefulness stage is displayed first and the deep sleep stage is displayed last. Or, the system may be configured so that the REM sleep stage is displayed first and the general sleep stage is displayed first. Such an order can be directly selected by the user or may be automatically configured by an algorithm. The orders described above are merely examples, and the present invention is not limited thereto.

[0736] Figure 25c Respiratory Stability Graph

[0737] Figure 25c is a diagram showing a respiratory stability graph according to an embodiment of the present invention.

[0738] A graphical user interface including a respiratory stability graph according to an embodiment of the present invention may also display the phrase "respiratory stability," as shown in reference no. 25301.

[0739] Through sleep analysis using one embodiment of the present invention, it is possible to predict sleep disorders (e.g., sleep apnea) and their underlying causes (e.g., snoring).

[0740] According to embodiments of the present invention, the stability of respiration may be determined by criteria such as the respiratory cycle, respiratory frequency variation, and respiratory pattern.

[0741] Irregular or sudden changes in breathing patterns during sleep may indicate unstable breathing. Alternatively, significant fluctuations or irregular changes in breathing frequency during sleep may also be considered signs of unstable breathing.

[0742] According to one embodiment of the present invention, if respiratory instability is determined in this manner, the respiratory stability during that period can be indicated as "respiratory unstable" (reference number 25303). Furthermore, the respiratory stability during periods that do not fall under "respiratory unstable" can be indicated as "respiratory stable" (reference number 25302).

[0743] As shown in reference numbers 25302 and 25303 in Figure 25c, respiratory stability can be shown as a graph over time. In this case, if respiratory stability during sleep is judged to be unstable, the phrase "unstable" can also be displayed (reference number 25306).

[0744] The apnea-hypopnea index (AHI) is calculated by dividing the total frequency of apnea and hypopnea during sleep, as recorded through sleep analysis such as multi-factor sleep studies, by total sleep time. An AHI value of less than 5 is classified as normal, 5 to less than 15 as mild, 15 to less than 30 as moderate, and 30 or more as severe.

[0745] According to one embodiment of the present invention, the AHI value can be determined through sleep analysis using an artificial intelligence model, and the percentage of respiratory stability can be calculated and displayed on the interface based on the determined AHI value (reference numbers 25304 and 25305).

[0746] Furthermore, according to one embodiment of the present invention, the color of the figure corresponding to the point in time when respiratory instability is determined can be displayed more brightly. This display has the advantage of allowing for a better understanding of the occurrence of events that are problematic from the perspective of respiratory stability.

[0747] Furthermore, according to one embodiment of the present invention, a hypnogram graph showing sleep stage information and a graph showing respiratory stability may be displayed side by side, one above the other, on the same time axis. When displayed in this manner, users can grasp at a glance what their respiratory stability was like at each sleep stage, which can be effective for analyzing sleep.

[0748] For example, if a respiratory instability event occurs during REM sleep, it may be interpreted as more serious. However, if a hypnogram graph showing sleep stage information and a graph showing respiratory stability are displayed side-by-side on the same time axis, as shown above, the occurrence of such an event can be easily grasped at a glance, making it easier to interpret and / or judge the sleep information.

[0749] Furthermore, in the embodiments of the present invention, when a hypnogram graph showing sleep stage information and a graph showing respiratory stability are displayed side by side on the same time axis, there is an advantage that such information can be utilized for providing various services, or such graphic image information can be utilized for training deep learning models.

[0750] Furthermore, the embodiments of the present invention have the advantage that the correlation between sleep information obtained from the graphs displayed side-by-side as shown above can also be used to generate sleep evaluation information.

[0751] Figure 25e is a diagram showing a graphic user interface including an explanatory display for respiratory instability according to one embodiment of the present invention.

[0752] A graphic user interface including an explanatory display for respiratory instability according to an embodiment of the present invention may also display the phrase "What is respiratory instability?" as shown in reference no. 25401.

[0753] As shown in Figure 25e, the graphic user interface may display an explanation of respiratory instability (reference number 25402). Furthermore, when the user clicks on a designated area shown in reference number 25403, they may be linked to an external website explaining respiratory instability or a screen displaying more detailed information.

[0754] As shown in Figure 25e, in one embodiment of the present invention, when an explanation of respiratory instability is displayed, the graphic user interface located in the background may be rendered in a relatively darker color (reference number 25404).

[0755] Figure 26 shows a screen displaying sleep statistics.

[0756] Figures 26a and 26b are diagrams showing a graphic user interface including statistical information on sleep state according to an embodiment of the present invention.

[0757] A graphic user interface containing statistical information on sleep state according to an embodiment of the present invention may also display the phrase "sleep statistics," as shown in reference number 26501.

[0758] According to an embodiment of the present invention, daily sleep duration can be represented by a bar graph. Specifically, the x-axis may display the days on which sleep state information was acquired (reference number 26503), and the y-axis may display the sleep duration values ​​(reference number 26504).

[0759] According to an embodiment of the present invention, the sleep duration calculated based on sleep state information acquired on the day corresponding to the date on the x-axis can be displayed in the form of a bar graph (reference no. 26502).

[0760] Furthermore, according to one embodiment of the present invention, factor information for the day on which the sleep state information was acquired may be displayed in the upper part of the bar graph (reference number 26505).

[0761] Furthermore, according to one embodiment of the present invention, information showing the average actual sleep time may be displayed below the bar graph (reference numbers 26506 to 26508). Specifically, at least one of the following can be displayed: the average sleep time obtained over a predetermined period, the average sleep time measured during weekdays, or the average sleep time measured during weekends.

[0762] Furthermore, according to one embodiment of the present invention, it is also possible to show the average sleep duration obtained over a predetermined period, the average sleep duration measured during weekdays, or the average sleep duration measured during weekends.

[0763] Furthermore, according to one embodiment of the present invention, a graphical user interface can be provided that, in order to compare the average sleep duration measured during weekdays and the average sleep duration measured during weekends with the average sleep duration measured over a predetermined period, divides the average values ​​with lines, as shown in reference number 26509. In this case, the dividing lines may be dotted or solid, and may be represented by curves or straight lines.

[0764] According to one embodiment of the present invention, if sleep state information is acquired only on weekdays and sleep analysis is not performed on weekends, and therefore sleep state information cannot be acquired, the graphic user interface may be displayed as shown in reference number 26508 in Figure 26a. In this case, the bar graphs corresponding to the 11th (Saturday) and 12th (Sunday) in Figure 26a may not be formed. The specific descriptions of days of the week and dates mentioned above are merely examples and are not limiting; for example, if sleep analysis is not performed on the 9th (Thursday), the bar graph corresponding to the 9th (Thursday) may not be formed.

[0765] Furthermore, as shown in Figure 26b, according to one embodiment of the present invention, when a specific bar graph is clicked, sleep time information acquired for the date corresponding to that bar graph may be displayed (reference number 26513). In this case, that bar graph may be displayed brightly (reference number 26512), while the other bar graphs may be displayed relatively darker.

[0766] Furthermore, as shown in Figures 26a and 26b, the average sleep time information obtained over a predetermined period may be displayed along with the bar graph. For example, if the average sleep time obtained over a predetermined period is 6 hours and 26 minutes, the average sleep time may be displayed as a line (e.g., a dotted line) or other shape on the y-axis of the bar graph at the position corresponding to 6 hours and 26 minutes (reference number 26510). Alternatively, referring to Figure 26b, if the average sleep time obtained over a predetermined period is 6 hours and 4 minutes, the average sleep time may be displayed as a line or other shape on the y-axis of the bar graph at the position corresponding to 6 hours and 4 minutes.

[0767] A graphical user interface containing statistical information on sleep state information according to one embodiment of the present invention may display both the phrase "sleep statistics" and the period information for which the sleep state information was acquired (for example, the period information "2022.6.6~6.12").

[0768] Furthermore, a graphic user interface containing statistical information on sleep status according to one embodiment of the present invention may also display a brief explanation for the graph (for example, "You slept this much this week," or "The more consistent the height of the sleep bar, the better"), as shown in reference no. 26511. The specific numerical values ​​for average sleep time and the brief explanatory phrases for the graph described above are merely examples, and the present invention is not limited thereto.

[0769] Figure 27: Sleep status information obtained over one week

[0770] Figures 27a and 27b are diagrams showing a graphic user interface that includes sleep state information acquired over one week, according to an embodiment of the present invention.

[0771] As shown in Figures 27a and 27b, sleep state information acquired over a week according to an embodiment of the present invention may be represented by a bar graph where the x-axis represents the day of the week (reference no. 27601) and the y-axis represents time information. Here, the time information on the y-axis may be expressed in 2-hour units (reference no. 27602). Lines (solid or dotted) corresponding to each time on the y-axis may also be displayed (reference no. 27603).

[0772] As shown in Figure 27a, according to one embodiment of the present invention, the bar graph may be represented entirely in a single color (reference number 27604).

[0773] Furthermore, as shown in Figure 27b, according to one embodiment of the present invention, the bar graph may be divided and displayed in portions corresponding to the sleep stages of REM sleep, deep sleep, light sleep, and wakefulness (reference numbers 27605 to 27608). Specifically, if the colors of the figures corresponding to each sleep stage information are different in the hypnogram graph, the bar graph for that date may display only the proportion of each sleep stage in the color of the figure corresponding to that sleep stage information.

[0774] Furthermore, in one embodiment of the present invention, when the portion corresponding to each sleep stage is displayed separately, the portion corresponding to the wakefulness stage may be placed at the top of the bar graph (reference number 27608), and the portion corresponding to deep sleep may be placed at the bottom of the bar graph (reference number 27605). Alternatively, portions corresponding to sleep stages with a higher proportion of time may be placed lower on the bar graph, and portions corresponding to sleep stages with a lower proportion of time may be placed higher on the bar graph. On the other hand, such arrangement orders are merely examples, and the present invention is not limited thereto. For example, the portions corresponding to the deep sleep stage, the light sleep stage, the REM sleep stage, and the wakefulness stage may be arranged in that order from the bottom.

[0775] Figure 28 Sleep state information acquired during a predetermined period.

[0776] Figures 28a and 28b are diagrams showing a graphic user interface that includes sleep state information acquired over a predetermined period of time, according to an embodiment of the present invention.

[0777] As shown in Figures 28a and 28b, the sleep state information obtained over a week according to the embodiment of the present invention may be represented by a bar graph in which the x-axis represents date (reference numbers 28701a and 28701b) and the y-axis represents visual information.

[0778] Referring to reference numbers 28701a and 28701b in Figure 28a, when sleep state information is obtained through sleep analysis performed from the date shown at the top to the date shown at the bottom, a bar graph can be displayed based on the information about when the person fell asleep and when they woke up, which is included in the acquired sleep state information. For example, if it is determined that the person fell asleep after the evening of the 12th and woke up after the early morning of the 13th, this can be represented in the form of the leftmost bar graph shown in Figure 28a. In this case, as shown in Figures 28a and 28b, the ends of the bar graph of sleep state information acquired during a predetermined period may correspond to the time of falling asleep and the time of waking up, respectively.

[0779] Here, the visual information on the y-axis may be expressed in units of time, but it may also be expressed using words such as noon, dawn, midnight, evening, and noon, as shown in Figures 28a and 28b (reference number 28702). Lines (solid or dotted) corresponding to each time on the y-axis may also be displayed (reference number 28703).

[0780] As shown in Figure 28a, according to one embodiment of the present invention, when lines corresponding to each time on the y axis are displayed together, the line corresponding to midnight may be displayed as a relatively thicker or brighter line than the lines corresponding to the other times to distinguish them.

[0781] As shown in Figure 28a, according to one embodiment of the present invention, the bar graph may be represented entirely in a single color (reference number 28704).

[0782] Furthermore, as shown in Figure 28b, according to one embodiment of the present invention, the bar graph may be divided and displayed in portions corresponding to the sleep stages of REM sleep, deep sleep, light sleep, and wakefulness. Specifically, if the colors of the figures corresponding to each sleep stage information are different in the hypnogram graph, the bar graph for that date may display only the proportion of each sleep stage using the color of the figure corresponding to that sleep stage information.

[0783] Furthermore, as shown in Figure 28b, in one embodiment of the present invention, when a bar graph is divided and displayed with portions corresponding to sleep stages—REM sleep, deep sleep, light sleep, and wakefulness—the portions corresponding to each sleep stage may be arranged in the order of the time when that sleep stage was detected.

[0784] Explanation of the flowchart in Figure 29

[0785] Figure 29 is a flowchart of a method for generating and providing one or more graphic user interfaces that show information about a user's sleep, according to one embodiment of the present invention.

[0786] As shown in Figure 29, according to one embodiment of the present invention, a method for generating one or more graphic user interfaces that show information about sleep may include the steps of acquiring sleep information (S29120), converting sleep information acquired in the time domain into information in the frequency domain (S29140), generating a graphic user interface (S29160), and providing a graphic user interface (S29180).

[0787] Here, the sleep information acquired during the sleep information acquisition stage may include environmental sensing information or sleep acoustic information.

[0788] Furthermore, according ...

Claims

1. A method for generating a graphic user interface that shows an evaluation of a user's sleep, The sleep information acquisition step involves acquiring sleep information from one or more sleep information sensor devices, the sleep information including the user's sleep acoustic information, A sleep phrase generation step, which generates a sleep phrase containing at least two words that indicate an evaluation of the user's sleep based on the acquired sleep information, The steps include: displaying a graphic user interface containing the generated phrase; A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep.

2. The aforementioned sleep phrase generation step is A first sleep phrase generation stage generates text that is of high importance among the user's evaluation of sleep, A second sleep phrase generation stage includes generating text that constitutes an evaluation of the user's sleep, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 1.

3. The aforementioned sleep phrase generation step is A third sleep phrase generation stage is included, which generates advice text based on the user's sleep evaluation. A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 2.

4. The aforementioned sleep phrase generation step is Lookup table sleep phrase extraction step: Extracting the sleep phrases based on the lookup table. Further including, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in any one of claims 1 to 3.

5. The aforementioned lookup table sleep phrase extraction step is: Based on the aforementioned sleep information, a lookup table classifies the user's sleep characteristics into stages, A step of extracting sleep phrases based on the lookup table corresponding to the sleep characteristics of the user. including, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 4.

6. The aforementioned sleep phrase generation step is Large-scale language model sleep phrase generation stage, which generates the sleep phrases based on a large-scale language model. Further including, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in any one of claims 1 to 3.

7. The aforementioned large-scale language model sleep phrase generation stage is, Based on the aforementioned sleep information, a large-scale language model classifies the user's sleep characteristics, and The steps include generating sleep phrases using the user's sleep characteristics as input to the large-scale language model, and including, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 6.

8. The aforementioned sleep information acquisition stage is, Sleep information conversion step: Converting the user's sleep acoustic information in the time domain into information in the frequency domain. Further including, A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 1.

9. The aforementioned sleep information acquisition stage is, This method further includes a sleep information inference stage, in which sleep acoustic information is used as input to a sleep information inference deep learning model to infer information about sleep. A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 1 or 8.

10. The aforementioned sleep information includes sleep environment information or user lifestyle information. A method for generating one or more graphic user interfaces that show an evaluation of a user's sleep as described in claim 1.

11. A method for providing a graphic user interface that displays information about a user's sleep, The process involves acquiring sleep information of the user from one or more sleep information sensor devices, including the user's sleep acoustic information, and Based on the user's sleep information obtained above, the user's sleep state information is obtained in real time - the user's sleep state information includes multiple sleep stage information - sleep state information acquisition stage, A sleep state information graph generation step, which generates a graph showing the user's sleep state information according to time based on the acquired sleep state information, The steps include displaying a graphic user interface that includes the generated graph, and including, A method for providing a graphic user interface that displays information about a user's sleep.

12. A method for providing a graphic user interface that displays information about a user's sleep, The process involves acquiring sleep information of the user from one or more sleep information sensor devices, including the user's sleep acoustic information, and Based on the user's sleep information obtained above, the user's sleep state information is obtained – the user's sleep state information includes multiple sleep stage information – sleep state information acquisition step, Based on the acquired sleep state information, a graph is generated showing the user's sleep state information according to time – the graph includes multiple rectangles – a sleep state information graph generation step, The steps include displaying a graphic user interface that includes the generated graph, and including, A method for providing a graphic user interface that displays information about a user's sleep.

13. The aforementioned sleep state information graph includes a plurality of rectangles, each corresponding to one of the plurality of sleep stages. A method for providing a graphic user interface that displays information about a user's sleep, as described in claim 12.

14. A method for providing a graphic user interface that displays information about a user's sleep, The process involves acquiring sleep information of the user from one or more sleep information sensor devices, including the user's sleep acoustic information, and Based on the user's sleep information obtained above, the user's sleep state information is obtained – the user's sleep state information includes multiple sleep stage information – sleep state information acquisition step, Based on the acquired sleep state information, a graph showing the user's sleep state information according to time is generated - the graph showing the user's sleep state information represents the sleep state information discretely - sleep state information graph generation stage, The steps include displaying a graphic user interface that includes the generated graph, and including, A method for providing a graphic user interface that displays information about a user's sleep.

15. The sleep state information graph includes a plurality of discrete figures, each corresponding to one of the plurality of sleep stages. A method for providing a graphic user interface that displays information about a user's sleep, as described in claim 14.

16. A method for providing a graphic user interface that displays information about a user's sleep, The process involves acquiring sleep information of the user from one or more sleep information sensor devices, including the user's sleep acoustic information, and Based on the user's sleep information obtained above, the user's sleep state information is obtained – the user's sleep state information includes multiple sleep stage information – sleep state information acquisition step, Based on the acquired sleep state information, a graph is generated showing the user's sleep state information according to time – the graph includes multiple rectangles – a sleep state information graph generation step, The steps include displaying a graphic user interface that includes the generated graph, and Includes, The sleep state information graph includes a plurality of rectangles, each corresponding to one of the plurality of sleep stages. The graphic user interface includes a plurality of regions, each assigned to one of the plurality of sleep stages. Each of the multiple rectangles included in the sleep state information graph is displayed only in the region assigned to the corresponding sleep stage among the multiple regions. A method for providing a graphic user interface that displays information about a user's sleep.

17. The boundaries between multiple regions included in the aforementioned graphic user interface are represented in such a way that at least one of them is distinguished by a line. A method for providing a graphic user interface that displays information about a user's sleep as described in claim 16.

18. A method for providing a graphic user interface that displays information about a user's sleep, The process involves acquiring sleep information of the user from one or more sleep information sensor devices, including the user's sleep acoustic information, and Based on the user's sleep information obtained above, the user's sleep state information is obtained – the user's sleep state information includes multiple sleep stage information – sleep state information acquisition step, Based on the acquired sleep state information, a graph is generated showing the user's sleep state information over time—each graph contains multiple figures corresponding to multiple sleep stages, but at least one of the figures is isolated from the rest—a sleep state information graph generation stage, The steps include displaying a graphic user interface that includes the generated graph, and including, A method for providing a graphic user interface that displays information about a user's sleep.

19. The aforementioned sleep state information acquisition step is: The acoustic information in the time domain included in the acquired sleep information is converted into information in the frequency domain. The method is characterized by acquiring the user's sleep state information based on the converted information in the frequency domain. A method for providing a graphic user interface that displays information about a user's sleep as described in any one of claims 11 to 18.

20. In the sleep state information acquisition stage, the converted information in the frequency domain is characterized in that it is a spectrogram. A method for providing a graphic user interface that displays information about a user's sleep, as described in claim 19.

Images