Head mounted display device and control method thereof

Abstract

This method performed by an HMD device may comprise the operations of: determining a sleep onset preparation start time; and displaying, on a display, a sleep onset preparation screen to which a visual effect is applied in stages during a sleep onset preparation time from the sleep onset preparation start time. The visual effect to be applied in stages may include a visual effect of switching a virtual screen outputted by means of the entire display area of the display to a VST screen. Here, the VST screen may display the virtual screen on the entire display area of the display with a non-virtual screen as a background, the non-virtual screen may be based on an image captured by means of a front camera, and the virtual screen may be based on content executed by the HMD device.

Description

Head-mounted display device and control method thereof

[0001] The present disclosure relates to a head-mounted display device for inducing sleep in a wearer and a method for controlling the same.

[0002] A wearable device is an electronic device developed to be worn by a user, such as clothing, shoes, glasses, watches, or rings. The wearable device may include a head-mounted display (HMD). The HMD may be used as a display device. The HMD may also be referred to as a face-mounted display (FMD).

[0003] The aforementioned HMD devices were initially developed for military use, but have since been commercialized for the general public, based on augmented reality (AR), virtual reality (VR), or video see-through (VST). For example, the aforementioned HMD devices are implemented as head-mounted goggle-style displays, allowing users to perform tasks or enjoy games while wearing them.

[0004] When a user falls asleep while wearing an HMD device, visual stimulation (e.g., a display screen) or auditory stimulation (e.g., audio output through speakers) from content services may interfere with falling asleep and / or deep sleep.

[0005] According to an example, a method performed by an HMD device may include an operation of determining a time for starting preparation for going to bed, and an operation of displaying a preparation screen for going to bed, to which visual effects are gradually applied during the preparation time from the preparation start time, on a display. The gradually applied visual effects may include a visual effect for converting a virtual screen output through the entire display area of ​​the display into a VST screen. The VST screen may display the virtual screen with a non-virtual screen as a background in the entire display area of ​​the display. The non-virtual screen may be based on an image captured through a front camera, and the virtual screen may be based on content executed by the HMD device.

[0006] According to an example, an HMD device may include at least one sensor, at least one camera, a display, and a processing circuit, and may include at least one processor operably connected to the at least one sensor, the at least one camera, or the display. Here, at least one processor (861) may be configured to determine a sleep preparation start time and display a sleep preparation screen on the display (851) to which visual effects are gradually applied from the sleep preparation start time during the sleep preparation time. The visual effects to be gradually applied may include a visual effect for switching a virtual screen output through the entire display area of ​​the display to a VST screen. The VST screen may display the virtual screen with a non-virtual screen as a background on the entire display area, and the non-virtual screen may be a screen based on an image captured through a front camera, and the virtual screen may be based on content executed by the HMD device.

[0007] In one embodiment, a computer-readable non-transitory medium stores instructions that, when executed by a processor of an HMD device, cause the processor to: determine a sleep preparation start time, and, from the sleep preparation start time, display a sleep preparation screen on a display to which visual effects are gradually applied during a sleep preparation time. The gradually applied visual effects may include a visual effect that switches a virtual screen output through the entire display area of ​​the display to a VST screen. The VST screen may display the virtual screen on the entire display area of ​​the display with a non-virtual screen as a background, and the non-virtual screen may be based on an image captured through a front camera, and the virtual screen may be based on content executed by the HMD device.

[0008] FIG. 1 is a diagram illustrating an example of use of an HMD device according to various embodiments of the present disclosure.

[0009] FIG. 2 is an exemplary time flow diagram for inducing a face in an electronic device according to various embodiments of the present disclosure.

[0010] FIG. 3 is a diagram illustrating exemplary state transitions of an electronic device according to various embodiments of the present disclosure.

[0011] FIG. 4 is an exemplary operation diagram for providing a VST screen in an electronic device according to various embodiments of the present disclosure.

[0012] FIGS. 5A to 5D are diagrams illustrating examples of arrangement of hardware components in an electronic device according to various embodiments of the present disclosure.

[0013] FIG. 6 is an exemplary diagram for predicting fatigue of a wearer in an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 7 is an example of wearing a wearable device for obtaining status information of a wearer in an electronic device according to one embodiment of the present disclosure.

[0015] FIG. 8 is an exemplary block diagram of an electronic device according to various embodiments of the present disclosure.

[0016] FIG. 9 is a control flowchart illustrating an example for inducing a face in an electronic device according to one embodiment of the present disclosure.

[0017] FIG. 10 is a control flow diagram illustrating an example for determining a start time for preparing to enter a sleeping position in an electronic device according to one embodiment of the present disclosure.

[0018] FIG. 11 is a control flowchart for determining a start time for preparing to enter a sleeping position in an electronic device, according to one embodiment of the present disclosure.

[0019] FIGS. 12A and 12B are control flowcharts illustrating exemplary procedures for preparing a surface in an electronic device according to one embodiment of the present disclosure.

[0020] FIGS. 13A to 13C are diagrams illustrating examples of screens to which visual effects are applied for each stage of preparation for entering a surface in an electronic device according to one embodiment of the present disclosure.

[0021] Figure 14a is a drawing showing an example of implementing a visual effect (a) that changes the depth of the screen in Figure 13b.

[0022] Figure 14b is a drawing showing an example of implementing a visual effect (b) that changes the screen size in Figure 13b.

[0023] FIG. 15 is a diagram illustrating an example of applying visual effects to each object in an electronic device according to one embodiment.

[0024] FIGS. 16A to 16F are diagrams illustrating examples of application of visual effects in an electronic device according to one embodiment.

[0025] FIG. 17 is a control flow diagram illustrating an exemplary procedure for inducing a face in an electronic device according to one embodiment.

[0026] FIG. 18a is a drawing showing an example of a screen with visual effects applied in the first stage of preparation for entering a face in an electronic device according to one embodiment.

[0027] FIG. 18b is a diagram illustrating an example of applying visual effects to each object in an electronic device according to one embodiment.

[0028] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0030] In one embodiment, an HMD device and a control method thereof may be provided that induce a user's sleeping position by gradually changing an external stimulus based on the user's expected (e.g., determined) sleeping position.

[0031] According to one embodiment, a sleeping environment that allows the user to have a good quality sleep can be provided by gradually reducing external stimuli, such as visual or auditory stimuli, based on the user's expected sleeping time in the HMD device.

[0032] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned above can be derived from the exemplary embodiments of the present disclosure by a person having ordinary knowledge in the relevant technical field.

[0033] The effects that can be achieved by the exemplary embodiments of the present disclosure can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure pertain, from the following description. In other words, unintended effects resulting from the implementation of the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0034] The above disclosure provides examples and descriptions, but is not intended to be exhaustive or to limit implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure, or may be made while implementing the embodiments. Furthermore, one or more features or components of one embodiment may be incorporated or combined with another embodiment (or one or more features of another embodiment). Furthermore, it should be understood that in the flowcharts and operational descriptions described, one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) concurrently, and the order of one or more operations may be reversed.

[0035] It will be apparent that the systems and / or methods described herein may be implemented in other forms, including hardware, firmware, or a combination of hardware and software. Implementation is not limited to the specific control hardware or software code used to implement these systems and / or methods. Accordingly, the operation and behavior of the systems and / or methods have been described without reference to specific software code, and it should be understood that software and hardware can be designed to implement the systems and / or methods based on the described content.

[0036] Although specific combinations of features are described in the claims or disclosed in the specification, this is not intended to limit the possible implementations to such combinations. In fact, many of these features can be combined in ways not specifically described in the claims or disclosed in the specification. While each listed dependent claim may directly depend on only one claim, the disclosure of possible implementations includes combinations of each dependent claim with all other claims in the set of claims.

[0037] No element, action, or instruction used herein should be construed as significant or essential unless explicitly stated to be so. Furthermore, as used herein, "a" and "an" are intended to include one or more items, and can be used interchangeably with "one or more." Where only one item is intended, "one" or a similar term is used. Furthermore, as used herein, the terms "has or have," "having," "include," and "including" are considered open-ended terms. Furthermore, the phrase "based on" is deemed to mean "at least in part, based on," unless explicitly stated otherwise. Furthermore, expressions such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include A alone, B alone, or both A and B.

[0038] Furthermore, the features, advantages, and characteristics described in this disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. Alternatively, additional features and advantages that are not present in all embodiments of the present disclosure may be recognized in a particular embodiment.

[0039] FIG. 1 is a diagram illustrating an example of use of an HMD device according to one embodiment.

[0040] Referring to FIG. 1, the HMD device (10) may be a device that a user (20) can wear on his or her head. In the present disclosure, for the sake of clarity of explanation, the terms “user” and “wearer” may be used interchangeably to refer to a person wearing the HMD device (10). In one example, the HMD device (10) may include a strap that wraps around the user’s head to secure the HMD to the user’s head. In one example, the HMD device (10) may be worn in a manner similar to a pair of glasses. After wearing the HMD device (10), the wearer (20) may use (e.g., execute) a desired content service. The content service may be, for example, a service that provides content such as a game or a movie. The content service may be a streaming content service. The various embodiments can be applied equally or with slight modifications to all devices that can supply visual information through a display while worn by the user (20), in addition to the HMD device (10) that the user (20) can wear on his / her head. For example, the various embodiments can be applied to electronic devices that the user (20) can wear, such as earphones, a headset, glasses, or a fish-eye lens, in addition to the HMD device (10). In the following description, the term "electronic device (10)" will be used to refer to a target to which the various embodiments are applied. The electronic device (10) can have a technical meaning referring not only to the HMD device (10), but also to a device to which the various embodiments can be applied.

[0041] External stimuli, such as visual or auditory, may affect the quality and / or quantity of sleep of the wearer (20). Therefore, to provide the wearer (20) with good quality and / or quantity of sleep, it may be desirable to minimize the extent to which the wearer (20) is exposed to external stimuli by suspending services provided by the electronic device (10) (e.g., powering off or pausing) and dimming the lighting.

[0042] The electronic device (10) may determine and / or acquire the onset of sleep preparation of the wearer (20). In one example, sleep may be the transition from a waking state to a sleeping state. Sleep may include several sub-stages. Sleep may be characterized by changes at the subjective, behavioral, cognitive, or physiological level. Sleep may lead to non-rapid eye movement (NREM) sleep, but in some circumstances may transition directly to rapid eye movement (RAM) sleep. The onset of sleep preparation indicates the time to initiate sleep preparation behavior that gradually or gradually reduces external stimuli to induce sleep of the wearer (20). The external stimuli may be stimuli that can affect the wearer (20) as the electronic device (10) provides content services. The external stimulus may be an input of information through one or more of the five senses (e.g., sight, hearing, smell, taste, touch) that the wearer (20) can feel (e.g., experience). Typical stimuli that the wearer (20) may receive due to the operation of the electronic device (10) may be visual stimuli and / or auditory stimuli. The visual stimulus may be a stimulus (e.g., glare) that a screen output through a display exerts on the eyes of the wearer (20). The visual stimulus may be gradually or stepwise reduced by adjusting at least one requirement for screen display, such as brightness (e.g., illuminance), size of the display area, or color. In the following disclosure, for convenience, the terms “gradually,” “stepwise,” or “step” will be used to collectively express the degree of reduction of the external stimulus (e.g., visual effect or auditory effect).For example, incrementally changing a parameter means changing a parameter (e.g., volume or brightness) by a predetermined amount based on a predetermined decision. For example, incrementally decreasing the volume might mean reducing the volume of a content service by 3 dB every 10 seconds for one minute.

[0043] According to one example, the electronic device (10) can obtain a sleep preparation start time based on the sleep target time or wake-up target time of the wearer (20). The sleep target time may be a target time set by the wearer (20) on his / her other electronic device (e.g., a smartphone (30) or a smart watch (40)) for sleep. The sleep target time may be set for a sleep alarm. The wake-up target time may be a target time set by the wearer (20) on his / her other electronic device (30, 40) for sleep. The wake-up target time may be set for a wake-up alarm. When the wearer (20) has set a target sleep time on his / her other electronic device (30, 40), the electronic device (10) can obtain a sleep preparation start time based on one of the sleep target time or wake-up target time according to the target sleep time. The above target sleep time may be a time interval (e.g., time interval) from the target sleep time to the target wake-up time.

[0044] The electronic device (10) can obtain the sleep preparation start time based on at least one of the predicted sleep preparation time based on the set sleep target time or wake-up target time and the sleep preparation time (time interval). The sleep preparation time may be the total time for the electronic device (10) to apply the visual effect in stages. For example, assuming that the sleep target time is 8 PM and the sleep preparation time is 20 minutes, the electronic device (10) can determine or obtain the sleep preparation start time at 7:40 PM, which takes into account the 20-minute sleep preparation time based on 8 PM. For example, assuming that the target sleep time is 8 hours, the target wake-up time is 6 AM, and the sleep preparation time is 20 minutes, the electronic device (10) can determine or obtain the sleep preparation start time at 9:40 PM, which takes into account the 20-minute sleep preparation time based on 10 PM, which is 8 hours before 6 AM. The electronic device (10) can obtain the sleep preparation start time by giving priority to the sleep target time when both the sleep target time and the wake target time are provided.

[0045] According to one example, the electronic device (10) can obtain the sleep preparation start time based on the sleep data recorded for the wearer (20). The electronic device (10) can collect the sleep data of the wearer (20) from a wearable device (40), such as a smart watch, worn by the wearer (20). According to one example, the sleep data can estimate the user's sleep based on the user's movement or heart rate. For example, if the user's movement is below a movement threshold for a predetermined period of time or the user's heart rate is below a heart rate threshold for a predetermined period of time, the user can be determined to be sleeping. For example, if the sleep data is analyzed to indicate that the wearer's (20) predicted sleep time is 10 PM, the electronic device (10) can determine or obtain 9:40 PM, which takes into account 20 minutes of sleep preparation time based on 10 PM, as the sleep preparation start time. For example, if the sleep data is analyzed to determine that the wearer's (20) predicted sleep time is 8 hours and the predicted wake-up time is 6:00 AM, the electronic device (10) can determine or obtain 9:40 PM, which is 20 minutes before 6:00 AM, as the sleep preparation start time, considering 10:00 PM, which is 8 hours before 6:00 AM.

[0046] For example, the electronic device (10) may determine that the time to prepare for sleeping has arrived based on the measurement of fatigue that satisfies a predetermined threshold level. The electronic device (10) may determine that the time to prepare for sleeping has arrived (e.g., started) when, for example, the fatigue level obtained by the fatigue measurement reaches the predetermined threshold level. To this end, the electronic device (10) may measure the fatigue of the wearer (20). The electronic device (10) may measure the fatigue of the wearer (20) periodically or aperiodically. In order to measure the fatigue periodically, the electronic device (10) may preset a cycle for measuring the fatigue. In order to measure the fatigue aperiodically, the electronic device (10) may monitor the occurrence of an aperiodic event. The aperiodic event may be, for example, a movement in which the wearer (20) suddenly lowers and then raises his or her head. Such movement may be detected by a sensing (e.g., detection) signal from at least one sensor.

[0047] The electronic device (10) can adjust the sleep preparation start time based on the measured fatigue. For example, the electronic device (10) can change the sleep preparation start time in proportion to the amount by which the fatigue increases or decreases. For example, if the adjustment time unit is 10 minutes, the electronic device (10) can reduce the sleep preparation time by 10 minutes each time the fatigue increases by a threshold value. For example, if the sleep preparation start time is 9:40 PM and the fatigue increases by a threshold value, the electronic device (10) can change the sleep preparation start time to 9:30 PM. If the fatigue further increases by a threshold value, the electronic device can change the sleep preparation start time to 9:20 PM. For example, if the sleep preparation time is 9:40 PM and the fatigue decreases by a threshold value, the electronic device (10) can change the sleep preparation start time to 9:50 PM.

[0048] The electronic device (10) may measure the wearer's (20) eye closure duration, the number of eye blinks, the wearer's (20) pupil diameter, the wearer's (20) ocular surface temperature, or the wearer's (20) heart rate variability in order to measure fatigue. The electronic device (10) may measure at least one of the wearer's (20) eye closure duration, eye blinks, the pupil diameter, or the ocular surface temperature based on a signal sensed by at least one sensor. The electronic device (10) may measure the wearer's (20) heart rate variability based on information provided from a wearable device (40) worn by the wearer (20). The electronic device (10) can obtain a fatigue level value indicating a level of fatigue based on at least one of the eye closure maintenance time, the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability.

[0049] The electronic device (10) may display a message on the display indicating whether to initiate sleep preparation at the time of initiating sleep preparation. For example, the guidance message may be "It's time to get ready for bed. Should I activate sleep preparation mode?" For example, the guidance message may be "Your fatigue level is high. Should I activate fatigue minimization mode?" The electronic device (10) may display a button on the display indicating the execution of the sleep preparation mode (e.g., a "yes" button) and / or a button not indicating the execution of the sleep preparation mode (e.g., a "no" button). The electronic device (10) may determine whether to activate the sleep preparation mode based on interaction with the wearer (20).

[0050] The electronic device (10) can gradually reduce external stimuli that may affect the quality or depth of sleep of the wearer (20) for a predetermined period of time (e.g., sleep preparation time) from the time of starting to prepare for sleep. The method for gradually reducing the external stimuli can be determined based on, for example, the type of content service that the wearer (20) was using. The method for gradually reducing the external stimuli can be determined based on, for example, the display mode applied to the display. Accordingly, the electronic device (10) can provide a sleep environment (e.g., a virtual environment) that facilitates the transition to sleep of the wearer (20). The electronic device (10) can, for example, output a sleep preparation screen that applies a visual effect that can gradually reduce visual stimuli over time through the display. The electronic device (10) may apply a visual effect to the wearer's preparation screen that switches a virtual screen (e.g., a virtual screen according to an immersive mode) output through the entire display area of ​​the display to a VST (video see-through) screen (e.g., a virtual screen according to a partial immersive mode). The immersive mode that outputs the virtual screen through the entire display area of ​​the display may be, for example, a VR mode. In the VR mode, the electronic device (10) may output the screen in a 360-degree shielded manner through the display. The VST screen is a screen that may be output through the display according to the switch from the immersive mode to the partial immersive mode. The VST screen according to the partial immersive mode may reduce the wearer's (20) visual immersion, thereby reducing the activation of the sympathetic nerve (e.g., the nerve that copes with a dangerous or stressful environment). The visual immersion may be proportional to the size of the screen output through the display. For example, the immersion may increase as the screen size increases, and may decrease as the screen size decreases.

[0051] The VST screen may be a screen including a virtual scene with a real scene (or real screen) (e.g., a non-virtual scene or screen) as a background. The VST screen may be a full screen displayed on the entire display area of ​​the display. The real screen may be a screen created based on an image captured of a real space (e.g., a space or place including an object) through a front camera. The virtual screen may be a screen created based on a content service being used by a user wearing the electronic device (10). For example, the electronic device (10) may output the VST screen through the display so that the visual realism that the content service should provide is maintained by considering the properties of an object included in the virtual screen or an object included in the real screen. In this regard, the VST screen may maintain one or more objects of the content service so that the wearer is not surprised by a sudden interruption in viewing the content service.

[0052] When applying visual effects to the VST screen in stages, the electronic device (10) can gradually reduce the proportion of the virtual screen on the VST screen over time. The electronic device (10) can gradually reduce the size of the virtual screen on the VST screen. The electronic device (10) can gradually deepen the depth of the virtual screen on the VST screen. In order to deepen the depth of the virtual screen, the electronic device (10) can move the virtual screen away from the wearer (20) on the entire screen expressing a three-dimensional space. In this regard, even if the wearer (20) is in a fixed position, the depth of the virtual screen can be adjusted so that the screen appears to be moving away from the user.

[0053] When applying visual effects in stages to the VST screen, the electronic device (10) can output a VST screen with visual effects that display preset background images at different points in time instead of the actual screen. For example, if the ratio of the virtual screen to the VST screen is reduced, the visual stimulation to the wearer (20) can be reduced.

[0054] The electronic device (10) may provide a visual effect that differentially controls blue light for each object included in a VST screen based on the properties of the objects. The blue light stimulates the brain and affects the circadian rhythm, which may disturb or disrupt sleep. The electronic device (10) may apply a visual effect that gradually increases the blue light that is differentially controlled for each object over time to the VST screen. The properties of the objects may include a first indicator that indicates one of a real screen or a virtual screen in which the object is included. The properties of the objects may include a second indicator that indicates whether the object is emitting light. The properties of the objects may include a third indicator that indicates the type of light source of the object. The properties of the objects may include at least one of a first indicator that indicates one of a real screen or a virtual screen in which the object is included, a second indicator that indicates whether the object is emitting light, or a third indicator that indicates the type of light source of the object. The properties of the above objects may include a first indicator indicating one of a real screen or a virtual screen containing the object, a second indicator indicating whether the object is luminous, and a third indicator indicating the type of light source of the object.

[0055] The electronic device (10) may provide a visual effect that differentially adjusts the illumination for each object included in the VST screen by considering the properties of the objects. The illumination may stimulate the brain and affect the circadian rhythm, thereby disturbing or disrupting sleep. The electronic device (10) may apply a visual effect that gradually increases the illumination that is differentially adjusted for each object over time to the VST screen. The properties of the objects may include a first indicator indicating one of the real screen or the virtual screen in which the object is included. The properties of the objects may include a second indicator indicating whether the object is emitting light. The properties of the objects may include a third indicator indicating the type of light source of the object. The properties of the objects may include at least one of the first indicator indicating one of the real screen or the virtual screen in which the object is included, the second indicator indicating whether the object is emitting light, or the third indicator indicating the type of light source of the object. The properties of the above objects may include a first indicator indicating one of a real screen or a virtual screen containing the object, a second indicator indicating whether the object is luminous, and a third indicator indicating the type of light source of the object.

[0056] As described above, when the electronic device (10) gradually adjusts the blue light and / or illuminance for each object on the VCT screen over time (e.g., when the blue light and / or illuminance is gradually adjusted over a predetermined time period), the degree of visual disturbance to the wearer (20) is sequentially reduced, but visual realism can be maintained.

[0057] The electronic device (10) may, when performing a sleep preparation function that gradually reduces external stimuli based on the passage of time during the sleep preparation time, gradually reduce the volume output by the content service. As a method for gradually reducing external stimuli based on the passage of time, the electronic device (10) may vary the sound source output for each stage. For example, the electronic device (10) may operate to output a sound source that is helpful for sleep (e.g., a sleep-inducing sound source) in proportion to the passage of time.

[0058] FIG. 2 is an example of a time flow diagram for inducing a user's facial expression in an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment, and FIG. 3 is a diagram illustrating an example of a state transition of an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment.

[0059] Referring to FIG. 2 or FIG. 3, a wearer (20) may enter sleep mode while wearing an electronic device (10). The electronic device (10) may monitor the sleep state (e.g., state or process) of the wearer (20). The electronic device (10) may perform an operation for leading or inducing sleep of the wearer (20) based on the sleep state (e.g., state or process) of the wearer (20). The operation for inducing sleep of the wearer (20) may be classified into a normal operation phase (e.g., an active state (310) of FIG. 3), a sleep preparation phase (e.g., a sleep preparation state (320) of FIG. 3), or a sleep phase (e.g., a sleep state (330) of FIG. 3). In addition, a rest phase (e.g., a rest state (340) of FIG. 3) may be further included.

[0060] The normal operation step (310) may be a step in which the electronic device (10) provides visual information and / or auditory information according to a content service to the wearer (20). The visual information may be provided to the wearer (20), for example, by outputting it to a screen in the entire area or a portion of the display. The auditory information may be provided to the wearer (20), for example, through an internal or external speaker. The normal operation step (310) may be an operation state that can be performed by the electronic device (10) before the arrival of a time for preparing to go to sleep. The normal operation step (310) may be an operation state that can be performed by the electronic device (10) when the wearer (20) is awake.

[0061] The above normal operation step (310) may be a step in which normal operation is performed by the electronic device (10) during a normal operation time (T1). The above normal operation time (T1) may be a time period corresponding to a normal operation step in which the wearer (20) is awake, in which the sympathetic nerve plays a dominant role compared to the parasympathetic nerve (e.g., a nerve that relaxes the user). The main functions of the sympathetic nerve may include dilating the pupil, inhibiting saliva secretion, accelerating the heartbeat, widening the bronchi, inhibiting the distribution of digestive fluids, secreting adrenaline or noradrenaline, or inhibiting bladder contraction. The main functions of the parasympathetic nerve may include constricting the pupil, stimulating saliva secretion, slowing the heartbeat, narrowing the bronchi, stimulating the distribution of digestive fluids, stimulating the secretion of bile, or contracting the bladder. For example, the main function of the sympathetic nerve may be activated when a person is awake. For example, the main function of the parasympathetic nerve may be activated when a person is asleep.

[0062] The above normal operation time (T1) may be defined by the time period from the time the user (10) wears the electronic device (10) or the time period from the time a predetermined content service is started after wearing the device to the time period from which the user (10) begins to prepare for sleep (220). The above normal operation time (T1) may also be defined by the time period from the time the wearer (10) wakes up from sleep (hereinafter referred to as the "time period from which the wearer wakes up") to the time period from which the user (10) begins to prepare for sleep (220).

[0063] In the above normal operation step (310), the electronic device (10) can determine and / or confirm the time of starting to prepare for going to bed (operation 212 or operation 311). As described above with reference to FIG. 1, the electronic device (10) can obtain the time of starting to prepare for going to bed based on preset time information (e.g., target time of going to bed or target time of waking up) or the fatigue level of the wearer (20).

[0064] In the normal operation step (310), the electronic device (10) can monitor and determine whether the sleep preparation start time has arrived. When the sleep preparation start time (220) has arrived, the electronic device (10) can transition from the normal operation step (310) to the sleep preparation step (320) (e.g., operation 313). When a sleep state transition request occurs in the normal operation step (310), the electronic device (10) can transition from the normal operation step (310) to the sleep stage (330) (operation 315). The sleep state transition request can occur when the wearer (20) instructs a transition to the sleep stage (310) for the purpose of sleeping.

[0065] The above-described sleep preparation step (320) may be a step in which, when the sleep preparation start time (220) arrives (operation 313), the electronic device (10) reduces (e.g., decreases) the sympathetic nerve activation of the wearer (20) and gradually reduces external stimuli based on the passage of time (e.g., in units of N seconds, where N is a positive integer) (operations 230-1 to 230-n or operation 323). The electronic device (10) may transition from the normal operation step (310) to the sleep preparation step (320) in response to the arrival of the sleep preparation start time (operation 313). The electronic device (10) may transition from the sleep preparation step (320) to the normal operation step (310) in response to a sleep preparation deactivation request (operation 321). The above deactivation request may occur, for example, because the electronic device (10) does not select execution of the wearer-ready mode based on interaction with the wearer (20).

[0066] The above sleep preparation step (320) is a step in which an operation for leading or inducting the wearer (20) into sleep is performed by the electronic device (10) during the sleep preparation time (T2). The sleep preparation time (T2) may be a time period in which a sleep preparation step is performed to prepare for a transition from a normal operation time (T1) in which the sympathetic nerves play a leading role to a sleep time (T3) in which the parasympathetic nerves play a leading role. The sleep preparation time (T2) may be defined by a time period from the sleep preparation start time (220) to the time (240) of entering a sleep state (hereinafter referred to as “sleep time”).

[0067] The electronic device (10) can gradually reduce external stimuli to be exerted on the wearer (20) during the sleep preparation time (T2) from the start time of sleep preparation (operations 230-1 to 230-n or operation 323). The electronic device (10) can gradually output a sleep preparation screen in which the visual stimuli to be exerted on the wearer (20) are gradually reduced based on the passage of time. The sleep preparation screen can be a VST screen according to a partial immersion mode. The VST screen can reduce the visual immersion of the wearer (20) and thus reduce the activation of the sympathetic nerves. The visual immersion can be proportional to the size of the screen output through the display. For example, the immersion can increase as the screen size increases and can decrease as the screen size decreases.

[0068] The above VST screen can display a virtual screen with a real screen (e.g., a real location or space) as the background. The real screen can display an image of a real space captured by the electronic device (10). The virtual screen can provide a content service. The electronic device (10) can output the VST screen so that the visual realism that the content service should provide is maintained by considering the properties of objects included in the virtual screen or objects included in the real screen.

[0069] The electronic device (10) can gradually reduce the ratio of the virtual screen to the VST screen over time (operation 230-1 to operation 230-n). The end time of each of the n (wherein, n is a positive integer) entrance preparation steps (e.g., entrance preparation step 1 to entrance preparation step n) illustrated in FIG. 2 may correspond to the start time of the next entrance preparation step. For example, the operation according to entrance preparation step 1 may be performed during a first time period (T) from the time point (220) when the entrance preparation start time arrives to the end time (230-1) of entrance preparation step 1. 2,1) can be performed during the period of time T. For example, the operation according to the n-step preparation for entrance is performed during the n-1 time interval (T) from the start time (230-n-1) of the n-1 step of entrance preparation to the end time (230-n) of the n-step preparation for entrance 2,n-1 ) can be performed during. In one example, the entrance preparation time (T2) can be defined as an interval including n evenly divided stages. For example, if the entrance preparation time (T2) is 10 minutes and includes 10 evenly divided stages, where stage 1 includes the entrance preparation start time, the visual effects can be applied in stages at 1-minute intervals. In one example, the intervals of the stages during the interval corresponding to the entrance preparation time can be non-uniform.

[0070] The electronic device (10) can switch from an immersive mode to a partial immersive mode at the time of starting preparation for sleep (220). The immersive mode may be a display mode that outputs a virtual screen through the entire display area of ​​the display. The partial immersive mode may be a display mode that outputs a VST screen through the entire display area of ​​the display. The electronic device (10) may be configured to perform an operation according to the first stage of preparation for sleep during a first time period (T 2,1 ) can output the initial VST screen through the display.

[0071] The above electronic device (10) is a second time period (T) in which operations according to the second stage of preparation for entrance to the n-1 stage of preparation for entrance are performed. 2,2 ) to the n-2 time interval (T 2,n-2) can output a VST screen with the size of the virtual screen gradually reduced through the display. The electronic device (10) can gradually deepen the depth of the virtual screen on the VST screen. This operation can provide the effect of moving the virtual screen away from the wearer (20) in three-dimensional space. The electronic device (10) can gradually reduce the proportion of the virtual screen on the VST screen. This can provide the effect of reducing the size of the virtual screen in two-dimensional space.

[0072] The above electronic device (10) is configured to perform an operation according to the n-th stage of preparation for the entrance (T) during the n-th time interval 2,n-1 ) can apply differentiated screen effects (e.g., blue light or illumination) to each object included in the VST screen. For example, the electronic device (10) can differentially adjust the blue light for each object included in the VST screen in consideration of the properties of the objects. The blue light can stimulate the brain and disrupt the circadian rhythm, thereby disturbing or disrupting sleep. The electronic device (10) can also gradually increase the blue light differentially adjusted for each object over time. The electronic device (10) can differentially adjust the illumination for each object included in the VST screen in consideration of the properties of the objects. The illumination can stimulate the brain and disrupt the circadian rhythm, thereby disturbing or disrupting sleep. The electronic device (10) can also gradually increase the illumination differentially adjusted for each object over time.

[0073] As described above, when the electronic device (10) gradually adjusts the blue light and / or illuminance for each object on the VST screen over time, the degree of visual disturbance to the wearer (20) is sequentially reduced, but visual realism can be maintained.

[0074] In one example, when the electronic device (10) performs (e.g., executes) a sleep preparation function that gradually reduces external stimuli based on the passage of time during the sleep preparation time, the electronic device (10) may gradually reduce the volume output by the content service. As a method for gradually reducing external stimuli based on the passage of time, the electronic device (10) may vary the sound source output for each stage. For example, the electronic device (10) may operate to output a sound source that is helpful for sleep (e.g., a sleep-inducing sound source) in proportion to the passage of time.

[0075] In the sleep preparation step (320), the electronic device (10) can monitor and determine whether the sleep preparation time has elapsed. If the sleep preparation time has elapsed, the electronic device (10) can transition from the sleep preparation step (320) to the sleep step (330) (operation 325). The sleep step may correspond to a step after the brain transitions to a sleep state.

[0076] The sleep time (T3) during which the electronic device (10) operates in the sleep stage (330) may be a period of time (e.g., a sleep period) during which the wearer (20) is asleep, in which the parasympathetic nerve plays a dominant role compared to the sympathetic nerve. The sleep time (T3) may be defined by the period of time from the time of entering sleep (240) until the wearer (20) wakes up from sleep. For example, when a request for transitioning from an out-of-sleep state occurs in the sleep stage (330), the electronic device (10) may transition from the sleep stage (330) to the normal operation stage (310) (operation 331). The request for transitioning from an out-of-sleep state may occur when the wearer (20) wakes up from sleep on his / her own (e.g., without external support or assistance) and instructs the resumption of the content service.

[0077] The electronic device (10) can monitor whether a deactivation request occurs in each of the normal operation phase (310), the sleep preparation phase (320), or the sleep phase (330). The deactivation request can occur when the wearer (20) instructs the termination of the content service. When the deactivation request occurs, the electronic device (10) can transition to the idle state (340) (operations 317, 327, and 333).

[0078] The electronic device (10) may transition from the idle state (340) to the activated state (310) when an activation request occurs in the idle state (340) (operation 341). The activation request may occur when the wearer (20) instructs the resumption of a content service that was terminated by the wearer.

[0079] FIG. 4 is an exemplary operation diagram for providing a VST screen in an electronic device (e.g., HMD device (10) of FIG. 1) according to one embodiment.

[0080] Referring to FIG. 4, the electronic device (10) may include a camera (410), a head tracker (420), a screen generator (430), a screen compositor (440), or a monitor (450).

[0081] The above camera (410) may be positioned to face forward while the user (20) is wearing the electronic device (10). The camera (410) may output an image (or a real screen) (411) captured of a real space or place (460). The real space may be a space where the user (20) actually (e.g., physically) exists.

[0082] The head tracker (420) can track the position of the moving head of the user (20) while wearing the electronic device (10). The head tracker (420) can output head position information (e.g., head locators) (421) according to the tracking.

[0083] The above screen generator (430) can generate a virtual screen (graphic images) (431) to be provided to the user (20) by considering the head position information provided from the head tracker (420) in a virtual screen according to the content service.

[0084] The above screen synthesizer (440) can synthesize (e.g., combine) the real screen (411) provided from the camera (410) and the virtual screen (431) provided from the screen generator (430) to generate a combined screen (e.g., VST screen) (441). The combined screen (441) generated by the screen synthesizer (440) can be transmitted to the monitor (450). For example, the combined screen (441) can display one or more objects corresponding to the real screen (411) as a background, and a virtual screen (431) including one or more objects can be displayed by overlapping the real screen (411).

[0085] The above monitor (450) can output the combined screen (441) transmitted from the screen synthesizer (440) as visual information. The user (20) can view the combined screen (441) output from the monitor (450).

[0086] FIGS. 5A to 5D are diagrams illustrating examples of arrangement of hardware components in an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment.

[0087] Referring to FIGS. 5A to 5D , the electronic device (10) may include a front side exposed to the outside or an inner side (e.g., an inner surface) facing toward a wearer (e.g., a wearer (20) of FIG. 1 ). The front side of the electronic device (10) may be configured with an external sensor unit (e.g., an external sensor unit (810) of FIG. 8) including left / right depth cameras (811a, 811b) (e.g., a depth camera (811) of FIG. 8 ), left / right side cameras (813a, 813b) (e.g., a side camera (813) of FIG. 8 ), left / right downward cameras (815a, 815b) (e.g., a downward camera (815)), or left / right infrared (IR) sensors (817a, 817b) (e.g., an IR sensor (817) of FIG. 8 )) (see FIG. 5A ). The positions where the left / right depth cameras (811a, 811b), left / right side cameras (813a, 813b), left / right downward cameras (815a, 815b), or left / right IR sensors (817a, 817b) are arranged on the front surface are not necessarily fixed, and the positions may be determined during the design stage (e.g., during design) of the electronic device (10). It will be understood by those skilled in the art that the positions of the components of the electronic device (10) may be arranged to optimize information collection and performance of the electronic device (10).

[0088] An eye recognition unit (e.g., eye recognition unit (820) of FIG. 8) including an IR camera (821) (e.g., IR camera (821) of FIG. 8) or an LED ring (823) (e.g., LED ring (823) of FIG. 8) may be configured on the inner surface of the electronic device (10). A sound unit (e.g., sound unit (840) of FIG. 8) including a microphone (841) (e.g., microphone (841) of FIG. 8) or a speaker (843) (e.g., speaker (843) of FIG. 8) may be configured on the inner side (e.g., inner surface) of the electronic device (10) (see FIG. 5b).

[0089] The electronic device (10) may include at least one processor (861) (e.g., processor (861) of FIG. 8) therein. The electronic device (10) may be equipped with a location recognition unit (e.g., location recognition unit (830) of FIG. 8) including an IMU sensor (831) (e.g., IMU (Inertial Measurement Unit) sensor (831) of FIG. 8) or an UWB (Ultra-Wideband) (833) (e.g., UWB (833) of FIG. 8) therein (see FIG. 5c).

[0090] A display unit (e.g., display unit (850) of FIG. 8) including a display (851) (e.g., display (851) of FIG. 8) or a lens (853) (e.g., lens (853) of FIG. 8) may be configured on the inside (e.g., inner surface) of the electronic device (10) (see FIG. 5d). As an example, the display (851) may be a single display having a width or length that can cover both eyes of the user. As an example, the electronic device (10) may include individual displays (851) for each eye of the user.

[0091] FIG. 6 is an exemplary diagram for predicting fatigue of a wearer (e.g., the wearer (20) of FIG. 1) in an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment.

[0092] Referring to FIG. 6, the electronic device (10) may include one or more sensor modules (e.g., IR camera modules) (e.g., first to fourth IR cameras (821a, 821b, 821c, 821d) of FIG. 5b) arranged on the inside to face the face of the wearer (20) (e.g., facing the eyes). For example, at least one first IR camera (e.g., the third or fourth IR camera (821c, 821d) of FIG. 5b) may be arranged on the inside to face the right eye. The at least one first IR camera (821c, 821d) may be a right-eye IR camera. For example, at least one second IR camera (e.g., the first or second IR camera (821a, 821b) of FIG. 5b) may be arranged on the inside to face the left eye. The above at least one first IR camera (821a, 821b) may be a left-eye IR camera.

[0093] The electronic device (10) can obtain an eye image (e.g., a right eye image (610a) and / or a left eye image (610b)) of the wearer (20) based on an image captured by the at least one first IR camera (821c, 821d) and / or the at least one second IR camera (821a, 821b). As an example, the electronic device (10) can obtain a right eye image (610a) of the wearer (20) based on an image captured by the at least one first IR camera (821c, 821d). As an example, the electronic device (10) can obtain a left eye image (610b) of the wearer (20) based on an image captured by the at least one second IR camera (821a, 821b).

[0094] The electronic device (10) can obtain a first reference point (hereinafter referred to as a “right-eye reference point”) (630a) from a right-eye image (610a). The electronic device (10) can obtain information or data regarding whether the right-eye reference point (630a) and the first alignment point (hereinafter referred to as a “right-eye alignment point”) (620a) match. The electronic device (10) can obtain a second reference point (hereinafter referred to as a “left-eye reference point”) (630b) from a left-eye image (610b). The electronic device (10) can obtain information or data regarding whether the left-eye reference point (630b) and the second alignment point (hereinafter referred to as a “left-eye alignment point”) (620b) match. The electronic device (10) can determine whether the wearer (20) is sleeping based on whether the right eye reference point (630a) and the right eye alignment point (620a) match. The electronic device (10) can measure the fatigue of the wearer (20) based on whether the left eye reference point (630b) and the left eye alignment point (620b) match. The electronic device (10) can obtain the eye closure duration, eye blink count, pupil diameter, or ocular surface temperature of the wearer (20) based on sensing signals detected by one or more sensor modules (e.g., IR camera modules) (e.g., the first to fourth IR cameras (821a, 821b, 821c, 821d) of FIG. 5b).

[0095] The electronic device (10) can determine a fatigue level value indicating the level of fatigue of the wearer (20) based on whether the right eye reference point (630a) and the right eye alignment point (620a) match and information acquired thereafter. Information items for determining the level of fatigue of the wearer (20) may include the eye closure maintenance time (e.g., the period during which the eyes are closed), the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability. The electronic device (10) can measure the eye closure maintenance time based on an image captured by a camera (e.g., the IR camera (821) of FIG. 8). The electronic device (10) can measure the number of eye blinks through image processing that recognizes the pupil in the image captured by the camera (e.g., the IR camera (821) of FIG. 8). For example, an ultrahigh speed camera may be used as a camera for measuring the number of eye blinks. The electronic device (10) may not collect information for measuring the number of eye blinks while performing an interaction with a user. The electronic device (10) may remove reflected light due to IR illumination from an image captured by an IR camera (e.g., the IR camera (821) of FIG. 8) and measure a pupil diameter based on the distribution of black pixels among the candidate images from which the reflected light has been removed. The electronic device (10) may extract an eye position from an image captured by a thermal imaging camera and predict (e.g., determine or estimate) a temperature measured at the extracted eye position as an ocular surface temperature. The electronic device (10) may obtain heart rate variability in real time based on information provided from a wearable device such as a smart watch.

[0096] below shows an example of predicting the fatigue of a wearer (20) based on the wearer's (20) eye closure time, number of blinks, pupil diameter, ocular surface temperature, or heart rate variability.

[0097] Data Measurement Value Evaluation Results Eye closure duration N1 (min) 1 (warning) Number of blinks N2 (times) 0 (normal) Pupil diameter N3 (mm) 1 (warning) Ocular surface temperature N4 (℃) 1 (warning) Heart rate variability N5 (times / min) 0 (normal)

[0098] For example, in the above , the evaluation result was defined as an example of the result of comparing the measured values ​​(N1, N2, N3, N4, N5) with each of the reference threshold values ​​(N_th1, N_th2, N_th3, N_th4, N_th5). As the evaluation result, for example, when the measured values ​​(N1, N2, N3, N4, N5) are each higher than the reference threshold values ​​(N_th1, N_th2, N_th3, N_th4, N_th5), a value set to "1" indicating that the fatigue has increased was given. As the evaluation result, for example, when the measured values ​​(N1, N2, N3, N4, N5) are each not higher than the reference threshold values ​​(N_th1, N_th2, N_th3, N_th4, N_th5), a value set to "0" indicating that the fatigue has not increased was given. The electronic device (10) may determine that the wearer's (20) level of fatigue is high when it is determined that a predetermined number or more of data is abnormal. For example, the electronic device (10) may advance the onset time of sleep by a predetermined time (e.g., N minutes, where N is a positive integer) from the average sleep time per day of the wearer (20) whenever the measured value of the corresponding item increases by a threshold value compared to the time of wearing. For example, in a situation where the onset time of sleep is 9:40 PM, when the measured value of the corresponding item increases by a threshold value, the onset time of sleep may be adjusted by 5 minutes to 9:30 PM.

[0099] FIG. 7 is an example of wearing a wearable device (e.g., a wearable device (40) of FIG. 1) for obtaining status information of a wearer (e.g., a wearer (20) of FIG. 1) from an electronic device (e.g., an HMD device (10) of FIG. 1) according to one embodiment.

[0100] Referring to FIG. 7, the wearable device (40) may be provided in various forms, such as glasses, a ring, or a watch. The wearable device (40) may include, for example, a smartwatch (700). The smartwatch (700) may be worn on a user's wrist. The smartwatch (700) may include various sensors. For example, the smartwatch (700) may include an electrical biosensor (710), an optical heart rate sensor (720), an acceleration sensor, and / or a gyro sensor.

[0101] The smartwatch (700) can acquire the user's movement and / or biometric information (e.g., heart rate, respiration, pulse, electrocardiogram) based on a sensing signal detected by at least one sensor. The smartwatch (700) can determine whether the user (10) is asleep (e.g., in a sleeping state) based on the acquired information (e.g., heart rate variability). The smartwatch (700) can also predict (e.g., determine or estimate) the fatigue level of the sleeping user (20) based on the acquired information. For example, the smartwatch (700) can communicate the information collected by the smartwatch (700) to the electronic device (10).

[0102] FIG. 8 is a block diagram of an exemplary electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment. Each device (unit) illustrated in FIG. 8 may be implemented as a separate circuit.

[0103] Referring to FIG. 8, the electronic device (10) may include an external sensor unit (810), an eye recognition unit (820), a location recognition unit (830), a sound unit (840), a display unit (850), a data transmission / reception unit (e.g., a communication module including a communication circuit) (860), or a memory (870). The external sensor unit (810) may include a depth camera (811), a side camera (813), a downward camera (815), or an infrared ray (IR) sensor (817). The eye recognition unit (820) may include an IR camera (821) or an LED ring (823). The location recognition unit (830) may include an inertial measurement unit (IMU) sensor (831) or an ultra-wideband (UWB) (833). The sound unit (840) may include a microphone (841) or a speaker (843). The display unit (850) may include a display (851) or a lens (853). The data transmission / reception unit (communication module) (860) may include at least one processor (861) (e.g., including a processing circuit). The memory (870) may include a user recognition module (871), a function control module (873), or a control transfer module (875).

[0104] For example, the external sensor unit (810) may include at least one sensor. The external sensor unit (810) may output a sensing signal for recognizing (e.g., identifying) an external object. The external sensor unit (810) may detect an operating state (e.g., power or temperature) of the electronic device (10), an external environmental state (e.g., user state), or hand or eye movement for interaction with a user, and generate an electrical signal or data value corresponding to the detected state. In addition to a depth camera (811), a side camera (813), a downward camera (815), or an IR sensor (817), the external sensor unit (810) may further include a gesture sensor, a gyro sensor, a pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor. The depth sensor (811) may output a sensing signal corresponding to a distance from an object. The side camera (813) can output an image photographed in the external side direction of the electronic device (10). The downward camera (815) can output an image photographed in the external downward direction of the electronic device (10). The IR sensor (817) can output a sensing signal capable of recognizing (e.g., identifying) the presence of an object.

[0105] For example, the eye recognition unit (820) may include an IR camera (821) or an LED ring (823) positioned toward the wearer (20). The IR camera (821) may output an image capturing the eyes and the area around the eyes of the wearer (20). The LED ring (823) may output a sensing signal for checking (e.g., monitoring) the alignment status of the eyes of the wearer (20).

[0106] For example, the location recognition unit (830) may include an IMU sensor (831) or UWB (833) for obtaining location information of the electronic device (10). The IMU sensor (831) may obtain location information of the electronic device (10) on a three-dimensional coordinate axis. The IMU sensor (831) may output the obtained location information as a sensing signal. The UWB (833) may obtain location information of the electronic device (10) based on ultra-wideband technology. The UWB (833) may output a sensing signal corresponding to the obtained location information.

[0107] For example, the external sensor unit (810), the eye recognition unit (820), or the position recognition unit (830) may constitute a sensor module. The sensor module may detect an operating state of the electronic device (10) (e.g., power or temperature) or an external environmental state (e.g., a user state such as head or eye movement) and generate an electrical signal or data value corresponding to the detected state. The sensor module may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0108] For example, the sound unit (840) may include a microphone (351) that converts external audio input into an electrical signal or a speaker (843) that converts the electrical signal into an audible audio signal. The speaker (843) may generate a detachment induction sound under the control of the processor (861).

[0109] For example, the display unit (850) can visually provide visual information (e.g., real image information or virtual image information) to the outside of the electronic device (10) (e.g., a wearer (e.g., the wearer (20) of FIG. 1)). The display unit (850) may include, for example, a display (851), a lens (853), a hologram device, or a projector and a control circuit for controlling the device. The display unit (850) may include a touch sensor set to detect a touch, or a pressure sensor set to measure the intensity of a force generated by the touch. The display unit (850) may be controlled in its operation by an electrical connection with the processor (861).

[0110] For example, the data transceiver (860) may include at least one processor (861). The processor (861) may execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) included in the electronic device (10) and perform various data processing or operations. As at least a part of the various data processing or operations, the processor (861) may store commands or data received from other components (e.g., an external sensor unit (810), an eye recognition unit (820), a position recognition unit (830), or a sound unit (840)) in a volatile memory (871), process the commands or data stored in the volatile memory (871), and store result data in a non-volatile memory (873). As at least part of the above various data processing or operations, the processor (861) may process commands or data received from other components (e.g., an external sensor unit (810), an eye recognition unit (820), a location recognition unit (830), or a sound unit (840)) and output result data through the display unit (850).

[0111] For example, the processor (120) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor (e.g., a communication module)) that can operate independently or together with the main processor. For example, when the electronic device (10) includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as a part thereof.

[0112] For example, the data transceiver (860) may include a communication module (not shown). The communication module may support establishment of a direct (e.g., wired) communication channel or a wireless communication channel with another device (e.g., a smart phone (30) or a wearable device (40) of FIG. 1), and performance of communication through the established communication channel. The communication module may operate independently from the processor (861) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. The communication module may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules can communicate with an external device via a network (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association), or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0113] For example, the memory (870) may store various data used by at least one component of the electronic device (10) (e.g., an external sensor unit (810), an eye recognition unit (820), a location recognition unit (830), or a processor (861)). The data may include, for example, input data or output data for software (e.g., a program) and commands related thereto. The memory (870) may include a volatile memory (871) or a non-volatile memory (873).

[0114] For example, the processor (861) may determine and / or obtain a time point at which the wearer (20) begins to prepare for sleeping. The time point at which the wearer (20) begins to prepare for sleeping indicates a time point at which to initiate a sleeping preparation operation that gradually reduces external stimuli in order to induce the wearer (20) to sleep. The external stimulus may be a stimulus that may affect the wearer (20) as the electronic device (10) provides a content service. The external stimulus may be an input of information through one or more of the five senses (e.g., sight, hearing, smell, taste, touch) that the wearer (20) may feel. As a stimulus that the wearer (20) may receive due to the operation of the electronic device (10), a visual stimulus and / or an auditory stimulus may be provided. The visual stimulus may be a stimulus (e.g., glare) that a screen output through the display (851) exerts on the eyes of the wearer (20). The above visual stimulus may be gradually reduced by adjusting at least one requirement for screen display, such as brightness (e.g., illuminance), size of the display area, or color.

[0115] For example, the processor (861) can obtain the sleep preparation start time based on the sleep target time or wake-up target time of the wearer (20). The above-described target time for going to sleep may be a target time set by the wearer (20) on his / her other electronic device (e.g., a smartphone (30) or a smart watch (40)) for going to sleep. The above-described target time for going to sleep may be set for a going to sleep alarm. The above-described target time for waking up may be a target time set by the wearer (20) on his / her other electronic device (e.g., a smartphone (30) or a smart watch (40)) for going to sleep. The above-described target time for going to sleep may be set for a going to sleep alarm. When the wearer (20) sets a target sleep time on his / her other electronic device (e.g., a smartphone (30) or a smart watch (40)), the processor (861) may obtain a time for preparing to go to sleep based on one of the target sleep time or the target wake-up time according to the target sleep time. The target sleep time may be a time period from the target sleep time to the target wake-up time. For example, an electronic device (e.g., a smartphone (30) or a smart watch (40)) may be configured to obtain an electronic The device (10) can cooperate with each other to determine the target wake-up time.

[0116] The processor (861) may obtain the sleep preparation start time based on at least one of the predicted sleep target times based on the set sleep target time or wake-up target time and the sleep preparation time. The sleep preparation time may be the total time for the electronic device (10) to apply the visual effects in stages. For example, if the sleep target time is 8 PM and the sleep preparation time is 20 minutes, the processor (861) may determine or obtain 7:40 PM as the sleep preparation start time based on the 20-minute sleep preparation time with respect to 8 PM as the sleep target time. For example, if the target sleep time is 8 hours, the target wake-up time is 6 AM, and the sleep preparation time is 20 minutes, the processor (861) may determine or obtain 9:40 PM as the sleep preparation start time with respect to 10 PM, which is 8 hours before 6 AM, and the 20-minute sleep preparation time. The above processor (861) can obtain (or determine) the entrance preparation start time by giving priority to the entrance target time when both the entrance target time and the wake-up target time are provided.

[0117] For example, the processor (861) may obtain (or determine) a sleep preparation start time based on sleep data recorded for the wearer (20). The processor (861) may, for example, collect sleep data of the wearer (20) from a wearable device (40) worn by the wearer (20). For example, if the sleep data is analyzed (or determined) to indicate that the wearer's (20) predicted sleep time is 10 PM, the processor (861) may determine or obtain 9:40 PM as the sleep preparation start time based on a sleep preparation time of 20 minutes based on 10 PM, which is the predicted sleep time. For example, if the sleep data is analyzed to determine that the wearer's (20) predicted sleep time is 8 hours and the predicted wake-up time is 6:00 AM, the processor (861) can determine or obtain 9:40 PM as the sleep preparation start time based on the sleep preparation time of 20 minutes, based on 10:00 PM, which is 8 hours before 6:00 AM.

[0118] For example, the processor (861) may determine that the sleep preparation start time has arrived (e.g., started) in response to the measurement of fatigue that substantially satisfies a predetermined threshold level. The processor (861) may determine that the sleep preparation start time has arrived (e.g., started) when, for example, the fatigue level obtained by the fatigue measurement reaches a predetermined threshold level. For example, the processor (861) may measure fatigue of the wearer (20). The processor (861) may measure the fatigue of the wearer (20) periodically or aperiodically. In order to measure the fatigue periodically, the processor (861) may preset a cycle for measuring the fatigue. In order to measure the fatigue aperiodically, the processor (861) may monitor the occurrence of an aperiodic event. The aperiodic event may be, for example, the occurrence of a movement such as suddenly lowering the head of the wearer (20). This can be detected by a sensing signal from at least one sensor.

[0119] For example, the processor (861) may measure the wearer's (20) eye closure duration (e.g., the time the eyes are closed), the number of eye blinks of the wearer (20), the pupil diameter of the wearer (20), the ocular surface temperature of the wearer (20), or the heart rate variability of the wearer (20) in order to measure fatigue. The processor (861) may measure at least one of the wearer's (20) eye closure duration, eye blinks, pupil diameter, or ocular surface temperature based on a signal detected by at least one sensor. The processor (861) may measure the heart rate variability of the wearer (20) based on information provided from a wearable device (40) worn by the wearer (20). The processor (861) can obtain a fatigue level value indicating a level of fatigue based on at least one of the eye closure duration, the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability.

[0120] For example, the processor (861) may display a message through the display (851) guiding whether to start sleep preparation when the sleep preparation start time has arrived. For example, the processor (861) may display a guidance message such as "It is time to get ready for bed. Should I start sleep preparation mode?" For example, the processor (861) may display a guidance message such as "Your fatigue level is high. Should I start fatigue minimization mode?" The processor (861) may display a button through the display (851) that instructs execution of the sleep preparation mode (e.g., a "yes" button) and / or a button that does not instruct execution of the sleep preparation mode (e.g., a "no" button). The processor (861) may determine whether to start the sleep preparation mode based on interaction with the wearer (20).

[0121] For example, the processor (861) may gradually reduce external stimuli that may affect the quality or depth of sleep of the wearer (20) for a predetermined period of time (e.g., sleep preparation time) from the time of starting to prepare for sleep. The method of gradually reducing the external stimuli may be determined based on, for example, the type of content service that the wearer (20) was using. The method of gradually reducing the external stimuli may be determined based on, for example, the display mode applied to the display (851). Accordingly, the processor (861) may provide a sleep environment (e.g., a virtual environment) that facilitates the transition to sleep of the wearer (20). The processor (861) may output a sleep preparation screen to which a visual effect that can gradually reduce visual stimuli over time is applied through, for example, the display (851). The processor (861) may apply a visual effect that converts a virtual screen that was output through the entire display area of ​​the display (851) into a VST screen to the sleep preparation screen. The immersive mode that outputs the virtual screen through the entire display area of ​​the display (851) may be, for example, a VR mode. In the VR mode, the electronic device (10) may output the screen in a 360-degree shielded manner through the display (851). The VST screen may be output through the display (851) based on a transition from the immersive mode to the partial immersive mode. The VST screen according to the partial immersive mode may reduce the visual immersion of the wearer (20) and thus reduce the activation of the sympathetic nerves. The visual immersion may be proportional to the size of the screen output through the display (851). The immersion may, for example, increase as the screen size increases and may decrease as the screen size decreases.

[0122] For example, the processor (861) can gradually reduce the proportion of the virtual screen on the VST screen over time. The processor (861) can gradually reduce the size of the virtual screen on the VST screen. The processor (861) can gradually deepen the depth of the virtual screen on the VST screen. In order to deepen the depth of the virtual screen, the processor (861) can move the virtual screen away from the wearer (20) on the entire screen expressing the three-dimensional space.

[0123] For example, the processor (861) may output a VST screen that reflects visual effects that display preset background images at different points in time instead of a real screen. For example, if the proportion of the virtual screen occupied by the VST screen is reduced, the visual stimulation given to the wearer (20) may be reduced.

[0124] For example, the processor (861) may provide a visual effect that differentially controls blue light for each object included in a VST screen, taking into account the properties of the objects. The blue light stimulates the brain and disrupts circadian rhythms, thereby disturbing or disrupting sleep. The processor (861) may apply a visual effect that gradually increases the differentially controlled blue light for each object over time to the VST screen.

[0125] For example, the processor (861) may provide a visual effect that differentially adjusts the brightness of each object included in the VST screen, taking into account the properties of the objects. The brightness may stimulate the brain, disrupting circadian rhythms and thus disrupting sleep. The processor (861) may apply a visual effect to the VST screen that gradually increases the brightness that is differentially adjusted for each object over time.

[0126] As described above, when the processor (861) gradually adjusts the blue light and / or illuminance for each object on the VST screen over time, the degree of visual disturbance to the wearer (20) is sequentially reduced, but visual realism can be maintained.

[0127] For example, when the processor (861) performs a sleep preparation function that gradually reduces external stimuli based on the passage of time during the sleep preparation time, the processor (861) may gradually reduce the volume output by the content service. As a method for gradually reducing external stimuli based on the passage of time, the processor (861) may vary the sound source output for each stage. For example, the processor (861) may operate to output a sound source that is helpful for sleep (e.g., sleep-inducing sound source) in proportion to the passage of time.

[0128] FIG. 9 is a control flow diagram illustrating an example of face induction in an electronic device (e.g., HMD device (10) of FIG. 1) according to one embodiment.

[0129] Referring to FIG. 9, the electronic device (10) may, in operation 910, determine and / or obtain a time when the wearer (e.g., the wearer (20) of FIG. 1) begins to prepare for wearing.

[0130] The electronic device (10) can obtain the sleep preparation start time based on at least one of the sleep target time or the wake-up target time of the wearer (20) and the sleep preparation time. For example, if the sleep target time is 8 PM and the sleep preparation time is 20 minutes, the electronic device (10) can determine the sleep preparation start time as 7:40 PM based on the sleep preparation time of 20 minutes with 8 PM as the sleep target time as the reference. For example, if the target sleep time is 8 hours, the wake-up target time is 6 AM, and the sleep preparation time is 20 minutes, the electronic device (10) can determine the sleep preparation start time as 9:40 PM based on the sleep preparation time of 20 minutes with 10 PM, which is 8 hours before 6 AM.

[0131] The electronic device (10) can determine the sleep preparation start time based on the recorded sleep data for the wearer (20). For example, if the sleep preparation time of the wearer (20) predicted (e.g., determined) based on the sleep data is 10 PM, the electronic device (10) can determine the sleep preparation start time as 9:40 PM based on 20 minutes of sleep preparation time based on 10 PM, which is the predicted sleep time. For example, if the predicted sleep time of the wearer (20) based on the sleep data is 8 hours, and the predicted wake-up time is 6 AM, the electronic device (10) can determine the sleep preparation start time as 9:40 PM based on 20 minutes of sleep preparation time based on 10 PM, which is 8 hours before 6 AM.

[0132] The electronic device (10) may change the sleep preparation start time in proportion to (e.g., in relation to) an increase or decrease in fatigue. For example, the electronic device (10) may periodically or aperiodically measure the fatigue of the wearer (10) and obtain a fatigue change amount based on the measured value. The electronic device (10) may measure the fatigue of the wearer (20), for example, by considering the eye closure maintenance time of the wearer (20), the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability. For example, in a situation where the adjustment time unit is 10 minutes and the sleep preparation start time is 9:40 PM, if the fatigue increases by a threshold value, the electronic device (10) may change the sleep preparation start time to 9:30 PM. For example, in a situation where the adjustment time unit is 10 minutes and the bed preparation start time is 9:40 PM, if the fatigue decreases by a threshold value, the electronic device (10) can change the bed preparation start time to 9:50 PM. The adjustment time unit (e.g., 20 minutes) for delaying the bed preparation start time due to the decrease in the fatigue may be different from the adjustment time unit (e.g., 10 minutes) for pulling forward the bed preparation start time due to the increase in the fatigue.

[0133] The electronic device (10) can, in operation 920, generate a sleeping preparation screen with visual effects applied in stages based on the passage of time until the sleeping preparation time elapses after the sleeping preparation start time arrives.

[0134] More specifically, when the time to prepare for bed arrives, the electronic device (10) may display a sleep preparation guidance message through the display. For example, the electronic device (10) may output "It's time to get ready for bed. Should I start the sleep preparation mode?" or "My fatigue level is high. Should I start the fatigue minimization mode?" as the sleep preparation guidance message. The electronic device (10) may display a button for approving the execution of the sleep preparation mode (e.g., a "yes" button) and / or a button for rejecting the execution of the sleep preparation mode (e.g., a "no" button) through the display. The electronic device (10) may determine whether to execute the sleep preparation mode based on interaction with the wearer (20).

[0135] The electronic device (10), when the sleep preparation mode is executed, can gradually reduce external stimuli such as visual stimulation and / or auditory stimulation for a predetermined period of time (e.g., sleep preparation time). The method for gradually reducing the external stimulation can be determined based on the type of content service. The method for gradually reducing the external stimulation can be determined based on the display mode applied to the display. For example, the electronic device (10) can generate a sleep preparation screen that applies a visual effect that can gradually reduce the visual stimulation based on the passage of time. The electronic device (10) can, for example, generate the sleep preparation screen as a VST screen. The VST screen can be a screen that includes a virtual screen with a real screen as a background. In order to gradually apply the visual effect to the VST screen, the electronic device (10) can gradually reduce the proportion of the virtual screen on the VST screen over time. The visual stimulation exerted on the wearer (20) can be reduced in proportion to the decrease in the proportion of the virtual screen on the VST screen.

[0136] The electronic device (10) can apply a visual effect that gradually adjusts the blue light for each object included in the VST screen based on the passage of time, taking into account the properties of the objects. For example, the amount of blue light adjustment for each object can be the same or different. The electronic device (10) can apply a visual effect that gradually increases the blue light adjusted evenly or differentially for each object based on the passage of time, for example, to the VST screen. The properties of the objects can be either a real screen (e.g., a non-virtual screen) or a virtual screen including the object, whether the object is emitting light, or the type of light source of the object.

[0137] The electronic device (10) can apply a visual effect that gradually adjusts the illumination for each object included in the VST screen based on the passage of time, taking into account the properties of the objects. At this time, the amount of illumination adjustment for each object can be the same or different. The electronic device (10) can apply a visual effect that gradually increases the illumination, which is adjusted evenly or differentially for each object, based on the passage of time, to the VST screen, for example. The properties of the objects can be either one of the real screen or virtual screen including the object, whether the object is emitting light, or the type of light source of the object.

[0138] The electronic device (10) may display a screen for preparing to enter the entrance through a display at operation 930. The visual effects applied to the screen for preparing to enter the entrance may be changed in stages based on the passage of time.

[0139] The electronic device (10) may sequentially reduce the degree of disruption to sleep that the wearer (20) experiences by repeatedly performing operations 920 and 930 periodically or aperiodically until the sleep preparation time elapses. The electronic device (10) may also gradually reduce auditory stimulation due to audio output from a content service based on the passage of time during the sleep preparation time. The electronic device (10) may also output a sound source that is helpful for sleep (e.g., sleep-inducing sound source) instead of the audio from the content service during the sleep preparation time.

[0140] FIG. 10 is a control flowchart for determining a time for starting a face-to-face preparation in an electronic device (e.g., the HMD device (10) of FIG. 1), according to one embodiment.

[0141] Referring to FIG. 10, an electronic device (10) may be connected to an external device (e.g., a smart phone (30) of FIG. 1) through a communication link based on a predetermined communication method (e.g., a wireless short-range communication method) in operation 1011. The electronic device (10) may perform an authentication procedure (e.g., account recognition) for personal information protection with the external device (30).

[0142] The electronic device (10) can determine, in operation 1013, whether a target time for sleeping has been set when connection and / or authentication with the external device (30) is completed. The target time for sleeping can be set by the user (20) in the external device (30) for sleeping.

[0143] If a target time for going to bed is set, the electronic device (10) may determine a time for starting preparation for going to bed based on the target time for going to bed in operation 1015. For example, the time for starting preparation for going to bed may be determined by considering the target time for going to bed and the time for getting ready for going to bed. The time for getting ready for going to bed may be set in advance. For example, if the target time for going to bed is 9:00 PM and the time for getting ready to go to bed is 30 minutes, the electronic device (10) may determine the time for starting preparation for going to bed as 8:30 PM.

[0144] When connection and / or authentication with the external device (30) is completed, the electronic device (10) can determine, in operation 1017, whether a target wake-up time has been set (e.g., a wake-up alarm has been set). The target wake-up time can be set by the user (20) in the external device (30) for waking up.

[0145] If a target wake-up time is set, the electronic device (10) may determine a sleep preparation start time based on the target wake-up time in operation 1019. In this case, the electronic device (10) may additionally consider the user's (20) sleep time. The user's sleep time may be set by the user (20) in the external device (30). The user's sleep time may be determined by an average sleep time obtained by analyzing sleep data provided by the external device (30). For example, the sleep preparation start time may be determined by considering the target wake-up time, the expected sleep time, and the sleep preparation time. The sleep preparation time may be set in advance. For example, if the target wake-up time is 6:00 AM, the expected sleep time is 8 hours, and the sleep preparation time is 30 minutes, the electronic device (10) may determine the sleep preparation start time as 9:30 PM.

[0146] FIG. 11 is a control flow diagram illustrating an exemplary procedure for determining a time for starting a face-up preparation in an electronic device (e.g., the HMD device (10) of FIG. 1), according to one embodiment.

[0147] Referring to FIG. 11, an electronic device (10) may be connected to an external device (e.g., a smart phone (30) of FIG. 1) through a communication link based on a predetermined communication method (e.g., a wireless short-range communication method) in operation 1101. The electronic device (10) may perform an authentication procedure (e.g., account recognition) for personal information protection with the external device (30).

[0148] When connection and / or authentication with the external device (30) is completed, the electronic device (10) may receive sleep data for a predetermined period (e.g., 3 months) from the external device (30) in operation 1103. The electronic device (10) may analyze the sleep data. For example, the electronic device (10) may analyze the average sleep time of the user (20) by day of the week based on the sleep data. The electronic device (10) may analyze the bedtime of the user (20) by day of the week based on the sleep data.

[0149] The electronic device (10) may, in operation 1105, obtain a predicted target time for falling asleep based on the average sleep time per day of the analyzed user (20) or the predicted time for falling asleep per day of the analyzed user (20). The predicted target time for falling asleep may be a time for falling asleep that the user (20) is expected to desire.

[0150] The electronic device (10) may monitor changes in the user's (20) fatigue level in operation 1107. The electronic device (10) may measure changes in the user's (20) fatigue level by considering biometric information provided from an external device (e.g., a smart phone (30) or a wearable device (40) of FIG. 1). The electronic device (10) may periodically or aperiodically measure the user's (20) fatigue level and evaluate a weight according to the amount of change. In order to periodically measure the fatigue level, the electronic device (10) may preset a period (e.g., 30 minutes) for measuring the fatigue level. In order to aperiodically measure the fatigue level, the electronic device (10) may monitor the occurrence of an aperiodic event. The aperiodic event may be, for example, a movement in which the user (20) suddenly lowers and then raises his / her head. This may be detected by a sensing signal from at least one sensor. As an example, the electronic device (10) can evaluate real-time fatigue weights based on monitored fatigue changes.

[0151] The electronic device (10), in operation 1109, can determine whether the fatigue level has changed beyond a threshold. For example, the electronic device (10) can determine whether the number of eye blinks of the user (20) has increased by a minimum predetermined number of times. For example, the electronic device (10) can determine whether the time for which the user (20) keeps his / her eyes closed has increased by a predetermined amount of time. For example, the electronic device (10) can determine whether the pupil diameter of the user (20) has increased by a predetermined amount. For example, the electronic device (10) can determine whether the ocular surface temperature of the user (20) has increased by a minimum predetermined value. For example, the electronic device (10) can determine whether the heart rate variability (HRV) of the user (20) has increased by a minimum predetermined level. The electronic device (10) may determine that fatigue has increased when a change occurs in at least one or more of the items for measuring fatigue (e.g., number of blinks, eye closure time, or pupil diameter) that can be predicted to be at an abnormal level.

[0152] If the electronic device (10) determines that the fatigue level has increased beyond a threshold level, the electronic device (10) may adjust the predicted target time for sleeping based on a real-time weight (or level) corresponding to the level of fatigue increase in operation 1111. For example, the electronic device (10) may change the predicted target time for sleeping in proportion to the increase or decrease in fatigue. For example, if the adjustment time unit is 10 minutes, the electronic device (10) may decrease the target time for sleeping in units of 10 minutes each time the fatigue level increases by the threshold value. For example, if the target time for sleeping is 9:40 PM and the fatigue level increases beyond a threshold value, the electronic device (10) may change the target time for sleeping to 9:30 PM. For example, if the target time for sleeping is 9:40 PM and the fatigue level decreases beyond a threshold value, the electronic device (10) may change the target time for sleeping to 9:50 PM.

[0153] The electronic device (10) can, in operation 1113, determine a sleeping preparation start time based on the determined sleeping target time. The electronic device (10) can start sleeping preparation at the sleeping preparation start time.

[0154] FIGS. 12A and 12B are control flowcharts for preparing a face in an electronic device (e.g., the HMD device (10) of FIG. 1), according to one embodiment.

[0155] Referring to FIG. 12A or FIG. 12B, the electronic device (10) can determine whether the time for starting preparation for bedtime has arrived in operation 1211. When the time for starting preparation for bedtime has arrived, the electronic device (10) can activate a mode for performing the bedtime preparation operation according to the first stage of bedtime preparation in operation 1213. For example, the electronic device (10) can display a bedtime preparation guidance message through the display. The electronic device (10) can output, through the display, a bedtime preparation guidance message such as, “It is time to get ready for bed. Shall we execute bedtime preparation mode?” The electronic device (10) can display, together with the bedtime preparation guidance message, a button for approving execution (e.g., a “Yes” button) and / or a button for rejecting execution (e.g., a “No” button). The electronic device (10) may determine whether to execute the sleep preparation mode in response to one of the two buttons selected based on interaction with the user (20).

[0156] When the first stage of the face-to-face preparation is activated, the electronic device (10) can determine, at operation 1215, whether a full display mode (e.g., VR mode) is applied, in which a virtual screen according to a content service is displayed over the entire area of ​​the display. If the electronic device (10) determines that the full display mode is not applied, the electronic device (10) can determine that a partial display mode (e.g., VST mode) is applied and proceed to operation 1225 for a stage transition.

[0157] If the electronic device (10) determines that the full display mode is applied, in operation 1217, it can determine whether a partial display mode (e.g., VST mode) can be applied. For example, the electronic device (10) can determine whether a switchable partial display mode (e.g., VST mode) option exists in a content service that provides a virtual screen by the full display mode.

[0158] The electronic device (10), if the partial display mode can be applied, can output a screen (e.g., VST screen) according to the partial display mode through the display in operation 1223. The VST screen according to the partial display mode can be a combined screen (e.g., combined screen (441) of FIG. 4) that synthesizes a virtual screen (e.g., virtual screen (431) of FIG. 4) with a real screen (e.g., real screen (411) of FIG. 4) as a background.

[0159] If the partial display mode cannot be applied, the electronic device (10) may switch to a virtual display display mode in operation 1219. When switching to the virtual display display mode, the electronic device (10) may display a partial virtual image in a virtual display area in operation 1221. For example, the electronic device (10) may determine a field of view (FOV) of the user (20) based on head position information of the user (20) (e.g., head position information (421) of FIG. 4). The electronic device (10) may obtain a partial virtual screen according to the field of view from among the entire virtual screen. The electronic device (10) may display the partial virtual screen in the virtual display area.

[0160] The electronic device (10) can determine, at operation 1225, whether a stage transition point has arrived. The stage transition point can be reached after a predetermined time (e.g., N minutes, where N is a positive integer) has elapsed after the first stage of the face preparation is activated. The electronic device (10) can perform an operation according to the first stage of the face preparation until the stage transition point is reached.

[0161] When the step transition point arrives, the electronic device (10) may activate a mode for performing a face-to-face preparation operation according to the face-to-face preparation step 2 in operation 1227. For example, the electronic device (10) may adjust the layout of a virtual screen on a VST screen according to a partial display mode in the face-to-face preparation step 2. Adjusting the layout may include adjusting the distance between the virtual screen and the user (20) on the VST screen (see FIG. 14a). Adjusting the layout may include adjusting the size of the virtual screen on the VST screen (see FIG. 14b).

[0162] More specifically, when the second stage of the face preparation is activated, the electronic device (10) can determine whether a partial display mode (e.g., VST mode) is being applied in operation 1229. If the partial display mode is not being applied, the electronic device (10) can determine that there is no option to support the partial display mode (e.g., VST mode) in the corresponding content service and proceed to operation 1239 for a stage transition.

[0163] The electronic device (10), when the partial display mode is applied, can determine in operation 1231 whether it is possible to change the layout of the partial display screen (VST screen) according to the partial display mode. For example, the electronic device (10), when it determines that the distance between the user (20) and the virtual screen is within a threshold distance, can determine that the layout change is possible.

[0164] The electronic device (10), if it is possible to change the layout of the virtual screen, can determine whether the virtual screen includes text in operation 1233. For example, the electronic device (10) can analyze the properties of the content providing the virtual screen. If the electronic device (10) determines that the properties of the analyzed content cannot include text, the electronic device (10) can change the layout by reducing the size of the virtual screen or applying a display effect that increases the distance from the user (20) in operation 1235 (see FIG. 14a or FIG. 14b). If the electronic device (10) determines that the properties of the analyzed content can include text, the electronic device (10) can change the layout by increasing the size of the text included in the virtual screen, reducing the size of the virtual screen, or applying a display effect that increases the distance from the user (20) in operation 1237.

[0165] The electronic device (10) can determine, at operation 1239, whether a stage transition point has arrived. The stage transition point can be reached after a predetermined time (e.g., N minutes, where N is a positive integer) has elapsed after the second stage of the face-up preparation is activated. The electronic device (10) can perform an operation according to the second stage of the face-up preparation until the stage transition point is reached.

[0166] The electronic device (10) may perform a surface preparation operation according to the surface preparation step 3 when the stage transition point arrives. For example, the electronic device (10) may detect objects included in the screen in operation 1241 and analyze the properties of each of the detected objects. The electronic device (10) may apply different visual effects to virtual objects (e.g., objects detected in a virtual screen (rendered image)) or real objects (e.g., objects detected in a real screen (captured image)) according to the analysis result (e.g., see FIG. 15). For example, the electronic device (10) may adjust a visual effect (e.g., blue light and / or illuminance) for a virtual object in operation 1243 (e.g., see FIG. 16D). For example, the electronic device (10) may, in operation 1245, differentially adjust a visual effect (e.g., blue light and / or illuminance) for each real object (e.g., see FIG. 16b). For example, the electronic device (10) may classify a light-emitting object among real objects, and differentially adjust a visual effect (e.g., blue light and / or illuminance) of a masking area according to the type of the light-emitting object (e.g., natural light source, artificial light source, or real display). The masking area may be an area set to apply a visual effect (e.g., blue light or illuminance adjustment) to each light-emitting object. The electronic device (10) may apply a differentiated visual effect (e.g., blue light and / or illuminance) to the masking areas of the remaining real objects excluding the light-emitting object among the real objects.

[0167] FIGS. 13A to 13C are drawings showing examples of screens to which visual effects are applied for each stage of preparation for wearing in an electronic device (e.g., HMD device (10) of FIG. 1) according to one embodiment.

[0168] Referring to FIG. 13a, the electronic device (10) may display a full display screen (131a) including a guidance message (1313a) saying "It's time to get ready for bed. Would you like to start the sleep preparation mode?" and a button (1315a) instructing execution (e.g., a "Yes" button) and / or a button (1317a) not instructing execution, by overlapping the virtual screen (1311a) at the time when the first stage of sleep preparation is activated (e.g., the time when sleep preparation starts). The electronic device (10) may select either the "Yes" button (1315a) or the "No" button (1317a) through interaction with the user (20) (1319a). For example, a “Yes” or “No” button may be activated by a user manipulating an external device connected to the electronic device (10), by a user providing a voice command, or by a user performing a motion indicating selection of the “Yes” or “No” button.

[0169] The electronic device (10) may, when the "Yes" button (1315a) is selected by the user (20), switch the full display screen (131a) (e.g., VR screen) to a partial display screen (131b) (e.g., VST screen). The partial display screen (131b) may be a screen that synthesizes (e.g., integrates or combines) a virtual screen (1311b) according to a content service with a real screen (1312b) based on a captured image. The real screen (1312b) or the virtual screen (1311b) may include a guidance message (1314b) stating, "Sleep preparation mode is executed in stages." The guidance message (1314b) may be displayed across the real screen (1312b) and the virtual screen (1311b).

[0170] Referring to FIG. 13b, the electronic device (10) can switch the first partial display screen (132a) to the second partial display screen (132b) when the second stage of the sleeping preparation is activated (e.g., a predetermined time has passed from the sleeping preparation start time). The second partial display screen (132b) may be a screen that applies a predetermined visual effect to the first partial display screen (132a). For example, the first partial display screen (132a) may be a screen that synthesizes a first virtual screen (1321a) according to a content service into a first reality screen (1322a) based on a captured image. For example, the second partial display screen (132b) may be a screen that synthesizes a second virtual screen (1321b) according to a content service into a second reality screen (1322b) based on a captured image. For example, a visual effect (b) that reduces the size of a virtual screen (1321a) in the first partial display screen (132a) may be applied to the second partial display screen (132b). For example, a visual effect (a) that increases the distance from the user (1327a, 1327b) compared to the first virtual screen (1321a) in the first partial display screen (132a) may be applied to the second virtual screen (1321b) included in the second partial display screen (132b). However, the size of the text (e.g., NOWHERE 2050) included in the first virtual screen (1321a) may be maintained the same in the second virtual screen (1321b).

[0171] Referring to FIG. 13C, the electronic device (10) can switch the third partial display screen (133a) to the fourth partial display screen (133b) when the third stage of the entrance preparation is activated (e.g., a predetermined time has passed since the second stage of the entrance preparation is activated). The fourth partial display screen (133b) may be a screen that applies a predetermined visual effect to the third partial display screen (133a). For example, the third partial display screen (133a) may be a screen that synthesizes a third virtual screen (1331a) according to a content service into a third reality screen (1332a) based on a captured image. For example, the fourth partial display screen (133b) may be a screen that synthesizes a fourth virtual screen (1331b) according to a content service into a fourth reality screen (1332b) based on a captured image. In the fourth partial display screen (133b), for example, visual effects (e.g., blue light and / or illuminance) may be differentially applied to each masking area (T1, T2, T3, T4). The masking areas (T1, T2, T3, T4) may be determined for each of the light-emitting objects (e.g., a stand, a window, a door, a TV) among the real objects included in the third reality screen (1332a) in the third partial display screen (133a). In the fourth partial display screen (133b), for example, a fourth virtual screen (1331b) in which visual effects (e.g., blue light and / or illuminance) are applied to the third virtual screen (1331a) in the third partial display screen (133a) may be included.

[0172] FIG. 14a is a drawing showing an example of implementing a visual effect (a) that changes the depth of the screen in FIG. 13b, and FIG. 14b is a drawing showing an example of implementing a visual effect (b) that changes the screen size in FIG. 13b.

[0173] Referring to FIG. 14a, the virtual screen (1411b) synthesized onto the real space (1412b) in the partial display screen (141b) after the visual effect (a) is applied may have a relatively greater distance from the user (1417a, 1417b) compared to the virtual screen (1411a) synthesized onto the real space (1412a) in the partial display screen (141a) before the visual effect (a) is applied. In this case, the user (1417a, 1417b) may have a lower degree of immersion in the screen after the visual effect (a) is applied.

[0174] Referring to FIG. 14b, the virtual screen (1421b) synthesized onto the real space (1422b) in the partial display screen (142b) after the visual effect (b) is applied may have a smaller screen size (b1>b2) compared to the virtual screen (1421a) synthesized onto the real space (1422a) in the partial display screen (141a) before the visual effect (a) is applied. In this case, the user's immersion in the screen may be reduced after the visual effect (b) is applied.

[0175] FIG. 15 is a diagram illustrating an example of applying visual effects to each object in an electronic device (e.g., HMD device (10) of FIG. 1) according to one embodiment.

[0176] Referring to FIG. 15, in the third stage of the preparation of the entrance, visual effects (e.g., blue light and / or illuminance) can be differentially applied to masking areas (T1, T2, T3, T4, T5, T6) that select target objects according to their properties. For example, the first masking area (T1) selects a stand (artificial light) that is one of the light-emitting objects included in the reality screen (1502), the second masking area (T2) selects a window (e.g., natural light) that is one of the light-emitting objects included in the reality screen (1502), the third masking area (T3) selects a door (light from another indoor space) that is one of the light-emitting objects included in the reality screen (1502), and the fourth masking area (T4) selects a TV (real display light) that is one of the light-emitting objects included in the reality screen (1502). The fifth masking area (T5) selects the entire virtual area (1501), the sixth masking area (T6) selects the entire real screen (1502), and the seventh masking area (T7) selects an area other than the light source in the real screen (1502).

[0177] For example, in the first to fourth masking areas (T1, T2, T3, T4) for selecting light-emitting objects in the reality screen (1502), for example, illuminance and / or blue light may be differentially applied. In the fifth masking area (T5) for selecting the entire area of ​​the virtual screen (1501) and / or the sixth masking area (T6) for selecting the entire area of ​​the reality screen (1502), for example, blue light and / or illuminance may be selectively and / or stepwise applied. In the seventh masking area (T7) for selecting an area other than a light source in the reality screen (1502), for example, both blue light and illuminance may be stepwise applied. As a result of the masking, the illuminance and / or blue light may be gradually reduced, thereby preventing the user's sympathetic nerves from being activated.

[0178] FIGS. 16A to 16F are drawings illustrating examples of application of visual effects in an electronic device (e.g., HMD device (10) of FIG. 1) according to one embodiment.

[0179] The partial display screen (160a) in FIG. 16a may be a VST screen that does not apply a visual effect (e.g., blue light), and the partial display screen (160b) in FIG. 16b may be a VST screen that applies a visual effect (e.g., blue light) to a masking area (T8, T9, T10) selected for light-emitting objects included in the real screen (1602b) and does not apply a visual effect (e.g., blue light) to the virtual screen (1601b).

[0180] The partial display screen (160a) in FIG. 16c may be a VST screen that applies a visual effect (e.g., blue light) to a masking area (T11) that selects the entire real screen (1602c) and does not apply a visual effect (e.g., blue light) to a virtual screen (1601c).

[0181] The partial display screen (160d) in FIG. 16d may be a VST screen that applies a visual effect (e.g., blue light) to a masking area (T12) that selects the entire virtual screen (1601d) without applying a visual effect (e.g., blue light) to the entire real screen (1602d).

[0182] The partial display screen (160e) in FIG. 16e may be a VST screen that differentially applies a visual effect (e.g., blue light) to the masking area (T8, T9, T10) selected for the light-emitting objects included in the real screen (1602e) and the masking area (T13) selected for the entire virtual screen (1601e).

[0183] The partial display screen (160f) in FIG. 16f may be a VST screen that applies a visual effect (e.g., blue light) to a masking area (T11) that selects the entire real screen (1602f) and a masking area (T13) that selects the entire virtual screen (1601f).

[0184] FIG. 17 is a control flow diagram illustrating an exemplary procedure for inducing a facial expression in an electronic device (e.g., the HMD device (10) of FIG. 1), according to one embodiment.

[0185] Referring to FIG. 17, the electronic device (10) may output a content screen through a display in operation 1711. The content screen may be a virtual screen according to a content service. The content screen may be a full display screen that outputs a virtual screen across the entire area of ​​the display.

[0186] The electronic device (10) may, in operation 1713, determine whether the content service outputting the virtual screen corresponds to content requiring fatigue management of the user (20). The fatigue of the user (20) may be, for example, eye fatigue. The content requiring fatigue management may include, for example, game content or movie content that provides high visual stimulation to the user (20), such as flashing lights.

[0187] If the electronic device (10) requires fatigue management, it can determine, at operation 1715, whether a set time has elapsed. The set time can determine the cycle for measuring fatigue. The set time can be, for example, 30 minutes. In this case, the electronic device (10) can measure fatigue in 30-minute cycles to obtain the amount of change in fatigue.

[0188] The electronic device (10) can measure the fatigue of the user (20) when a set time has elapsed, in operation 1717. The electronic device (10) can obtain a change in fatigue according to the measured fatigue. The electronic device (10) can measure a change in the fatigue of the user (20) by considering biometric information provided from an external device (e.g., a smart phone (30) or a wearable device (40) of FIG. 1). The electronic device (10) can evaluate a fatigue weight in real time based on the amount of change in the fatigue. In addition to periodic measurement, the electronic device (10) can measure fatigue non-periodically. For example, the electronic device (10) can measure the fatigue of the user (20) when an abnormal behavior, such as a movement in which the user (20) suddenly lowers and then raises his / her head, is detected.

[0189] The electronic device (10), in operation 1719, may determine whether the amount of change in the fatigue level is greater than or equal to a threshold level. For example, the electronic device (10) may determine whether the number of eye blinks of the user (20) has increased by a predetermined number of times or more. For example, the electronic device (10) may determine whether the time for which the user (20) keeps his / her eyes closed has increased by a predetermined amount of time. For example, the electronic device (10) may determine whether the pupil diameter of the user (20) has increased by a predetermined amount. For example, the electronic device (10) may determine whether the ocular surface temperature of the user (20) has increased by a predetermined value or more. For example, the electronic device (10) may determine whether the heart rate variability (HRV) of the user (20) has increased by at least a predetermined level. The electronic device (10) may determine that fatigue has increased when a change occurs in at least one or more of the items for measuring fatigue (e.g., number of blinks, eye closure time, or pupil diameter) that can be predicted to be at an abnormal level.

[0190] If the electronic device (10) determines that the amount of change in the fatigue level is greater than or equal to a threshold level, in operation 1721, the electronic device (10) may provide a notification suggesting a transition to a fatigue minimization mode in which the user (20) performs a fatigue reduction operation. The electronic device (10) may provide the notification to the user (20) using auditory information and / or auditory information. For example, the electronic device (10) may output a guidance message (e.g., guidance message (1813a) of FIG. 18a) through the display, saying, "The fatigue level is high. Should we execute the fatigue minimization mode?"

[0191] If the user (20) requests a transition to the fatigue minimization mode in response to the guidance message (1813a), the electronic device (10) may perform a fatigue reduction operation by switching to the fatigue minimization mode in operation 1723. For example, the electronic device (10) may output a button (e.g., a “Yes” button (1815a) of FIG. 18a) indicating a transition to the fatigue minimization mode and / or a button (e.g., a “No” button (1817a) of FIG. 18a) not indicating a transition to the fatigue minimization mode through a display. If the electronic device (10) interacts with the user (20) and the “Yes” button (1815a) is selected, the electronic device (10) determines that the user (20) has requested a transition to the fatigue minimization mode and may switch to the fatigue minimization mode to perform an operation for reducing fatigue.

[0192] According to one example, the electronic device (10) may activate the first stage of sleep preparation when the transition to the fatigue minimization mode is made (e.g., the time of sleep preparation start). When the first stage of sleep preparation is activated, the electronic device (10) may switch from the full display mode (VR mode) to the partial display mode (VST mode) (e.g., see FIG. 18a). The full display mode (VR mode) may be a display mode that displays a full display screen (VR screen) that displays a virtual screen according to a content service on the entire area of ​​the display. The partial display mode (VST mode) may be a display mode that displays a combined screen (VST screen) that synthesizes a virtual screen according to a content service onto a real screen on the entire area of ​​the display.

[0193] According to one example, the electronic device (10) may activate the second stage of preparation for entering the screen when the stage transition time arrives. When the second stage of preparation for entering the screen is activated, the electronic device (10) may analyze the properties of objects included in the screen. The electronic device (10) may apply different visual effects to virtual objects (e.g., objects detected in a virtual screen (rendered image)) or real objects (e.g., objects detected in a real screen (captured image)) depending on the properties of the objects (e.g., see FIG. 18b). The electronic device (10) may adjust the visual effects (e.g., blue light and / or illuminance) for virtual objects, or may differentially adjust the visual effects (e.g., blue light and / or illuminance) for real objects. The electronic device (10) can, for example, classify light-emitting objects among real objects, and differentially adjust the visual effect (e.g., blue light and / or illuminance) of a masking area (e.g., masking area (T14, T15, T16 or T17) of FIG. 18b) according to the type of the light-emitting object (e.g., natural light source, artificial light source or real display). The masking area (T14, T15, T16 or T17) may be an area set to apply a visual effect (e.g., blue light or illuminance adjustment) to each light-emitting object. The electronic device (10) can apply the visual effect (e.g., blue light and / or illuminance) to the masking area of ​​the remaining real objects excluding the light-emitting objects among real objects, or to the entire area of ​​the real screen (e.g., the real screen (1822) of FIG. 18b) (e.g., masking area (T18) of FIG. 18b). The electronic device (10) can apply a visual effect (e.g., blue light and / or illuminance) to the entire area (e.g., masking area (T19) of FIG. 18b) of a virtual screen (e.g., virtual screen (1821) of FIG. 18b).

[0194] FIG. 18a is a drawing showing an example of a screen to which visual effects are applied in the first stage of preparation for wearing in an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment.

[0195] Referring to FIG. 18a, when the first stage of the sleep preparation is activated (e.g., the time of the sleep preparation start), the electronic device (10) can display a full display screen (181a) including a guidance message (1813a) saying "Your fatigue level is high. Should we run the fatigue minimization mode?" and a button (1815a) instructing execution (e.g., a "Yes" button) and / or a button (1817a) not instructing execution. The electronic device (10) can select either the "Yes" button (1815a) or the "No" button (1817a) through interaction with the user (20) (1819a).

[0196] The electronic device (10) may, when the "Yes" button (1815a) is selected by the user (20), switch the full display screen (181a) (e.g., VR screen) to a partial display screen (181b) (e.g., VST screen). The partial display screen (181b) may be a screen that synthesizes (e.g., integrates or combines) a virtual screen (1811b) according to a content service with a real screen (1812b) based on a captured image. The real screen (1812b) or the virtual screen (1811b) may include a guidance message (1814b) stating, "Fatigue minimization mode is being executed in stages." The guidance message (1814b) may be displayed across the real screen (1812b) and the virtual screen (1811b).

[0197] FIG. 18b is a diagram illustrating an example of applying visual effects to each object in an electronic device (e.g., the HMD device (10) of FIG. 1) according to one embodiment.

[0198] Referring to FIG. 18b, in the third stage of the elevation preparation, visual effects (e.g., blue light and / or illuminance) may be differentially applied to masking areas (T14, T15, T16, T17, T18, T19) that select target objects according to their properties. For example, the first masking area (T14) selects a TV (real display light) that is one of the light-emitting objects included in the reality screen (1822), the second masking area (T15) selects a door (light from another indoor space) that is one of the light-emitting objects included in the reality screen (1822), and the third masking area (T16) and the fourth masking area (T17) select two windows (natural light) that are arranged side by side that are one of the light-emitting objects included in the reality screen (1822). The fifth masking area (T18) selects the entire real screen (1822), and the sixth masking area (T19) selects the entire virtual area (1821).

[0199] For example, illuminance and / or blue light may be differentially applied to the first to fourth masking areas (T14, T15, T16, T17) for selecting light-emitting objects in the above-mentioned real screen (1822). For example, blue light and / or illuminance may be selectively and / or stepwise applied to the fifth masking area (T18) for selecting the entire area of ​​the above-mentioned real screen (1822) and / or the sixth masking area (T19) for selecting the entire area of ​​the above-mentioned virtual screen (1821).

[0200] The terminology used in this disclosure is merely used to describe specific embodiments and is not intended to limit the present disclosure. For example, a singular element should be understood to include plural elements unless the context clearly indicates only a singular element. As used herein, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” can each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. It should be understood that the term “and / or” as used herein encompasses any and all possible combinations of one or more of the listed items. The terms "include," "have," "comprise," and the like used in this disclosure are intended to specify only the presence of a feature, component, part, or combination thereof described in this disclosure, and the use of these terms does not exclude the possibility of the presence or addition of one or more other features, components, parts, or combinations thereof. The expressions "first," "second," and the like used in this disclosure can modify various components regardless of order and / or importance, and are only used to distinguish one component from another, without limiting the components.

[0201] The expression "configured to" used in the present disclosure can be appropriately used interchangeably with, for example, "suitable for," "capable of," "designed to," "modified to," "made to," or "capable of." The term "configured to" may not necessarily mean only something "specially designed" in terms of hardware. Instead, in some situations, the expression "a device configured to" may mean that the device is "capable of" doing something together with other devices or components. For example, the phrase "a device configured (or set) to perform A, B, and C" may mean a dedicated device for performing the corresponding operations, or a general-purpose device that can perform various operations including the corresponding operations.

[0202] Meanwhile, the terms 'upper side', 'lower side', and 'front-rear direction' used in the present disclosure are defined based on the drawing, and the shape and position of each component are not limited by these terms.

[0203] Although the foregoing description in this disclosure has focused on specific embodiments, it should be understood that this disclosure is not limited to such specific embodiments, but rather encompasses various modifications, equivalents, and / or alternatives of the various embodiments.

Claims

1. A method performed by a head mounted display (HMD) device (10), Actions that determine the start time of sleep preparation; and From the above-mentioned entrance preparation start time, during the entrance preparation time, an action is included to display an entrance preparation screen with visual effects applied in stages on the display, The above stepwise applied visual effects include a visual effect that converts a virtual screen output through the entire display area of ​​the display into a VST (video see through) screen, Here, the VST screen displays the virtual screen with a non-virtual screen as the background in the entire display area of ​​the display, The above non-virtual screen is based on an image captured through the front camera, The above virtual screen is a method based on content executed by the HMD device.

2. In paragraph 1, The action for determining the start time of the above-mentioned entrance preparation is: An action for measuring fatigue of the above user; and A method comprising an action of determining the start time of the sleep preparation based on the fatigue level satisfying a predetermined threshold level.

3. In paragraph 2, The above fatigue measuring action is, A first measurement operation for measuring the time that the user's eyes are closed; A second measurement operation measuring the number of times the user blinks his / her eyes; A third measurement operation for measuring the pupil diameter of the user; A fourth measurement operation for measuring the ocular surface temperature of the user; A fifth measurement operation for measuring the heart rate variability of the user; and A method comprising the action of obtaining a fatigue level value indicating the level of fatigue based on at least one of the time the eye is closed, the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability.

4. In any one of paragraphs 1 to 3, The action for determining the start time of the above-mentioned entrance preparation is: An operation of obtaining the sleep preparation start time based on at least one of the user's sleep target time or wake-up target time and the sleep preparation time, or An operation for obtaining the sleep preparation start time based on the sleep data of the user, In this case, the visual effects are applied step by step during the preparation time of the entrance.

5. In any one of paragraphs 1 to 4, A method according to claim 1, wherein the visual effects to be applied in stages include at least one of a visual effect of gradually reducing the proportion of the virtual screen on the VST screen during the entrance preparation time or a visual effect of displaying a preset background screen over time instead of the non-virtual screen during the entrance preparation time.

6. In any one of paragraphs 1 to 4, The visual effects that will be applied in stages above are: A visual effect that displays the VST screen with the blue light differentially adjusted for each object based on the properties of one or more objects included in the VST screen, or A visual effect that displays the VST screen with differential brightness for each object based on the properties of one or more objects included in the VST screen, A method wherein the properties of one or more of the objects included in the VST screen include at least one of a first indicator indicating one of the non-virtual screen or the virtual screen including the object, a second indicator indicating whether the object is emitting light, or a third indicator indicating the type of light source of the object.

7. In any one of paragraphs 1 to 4, The action of applying the above visual effects in stages during the preparation time for entering sleep is: An action of displaying a message through the display guiding whether to start preparing for bed at the above-mentioned bed preparation start time; and A method comprising an action for determining whether to apply the visual effect in stages based on an interaction with the user.

8. In a head mounted display (HMD) device (10), At least one sensor (310, 320, 330); At least one camera; display (851); and A processing circuit comprising at least one processor (861) operably connected to at least one sensor (310, 320, 330), at least one camera or the display (851), Here, at least one processor (861) Decide when to start preparing for the entrance exam, It is configured to display a sleeping preparation screen with visual effects applied in stages from the above sleeping preparation start time to the sleeping preparation time on the display (851). The visual effects to be applied in stages above include a visual effect that switches the virtual screen output through the entire display area of ​​the display to a VST (video see through) screen. Here, the VST screen displays the virtual screen with a non-virtual screen as the background in the entire display area of ​​the display, The above non-virtual screen is based on an image captured through the front camera, The above virtual screen is an HMD device based on content executed by the HMD device.

9. In paragraph 8, At least one processor (861), Measure the user's fatigue level based on information collected from at least one of the above at least one sensor (310, 320, 330) or at least one camera, An HMD device configured to determine the sleep preparation start time based on the fatigue level satisfying a predetermined threshold level.

10. In paragraph 9, At least one processor (861), measuring the time that the user's eyes are closed, or measuring the number of times the user blinks, or measuring the user's pupil diameter, or measuring the user's ocular surface temperature, or measuring the user's heart rate variability, An HMD device configured to obtain a fatigue level value indicating the level of fatigue based on at least one of the time the eyes are closed, the number of eye blinks, the pupil diameter, the ocular surface temperature, or the heart rate variability.

11. In any one of paragraphs 8 to 10, At least one processor (861), It is configured to acquire the sleep preparation start time based on the user's sleep target time or wake-up target time, or determine the sleep preparation start time based on at least one of the sleep target time or the wake-up target time and the sleep preparation time. Here, the HMD device in which the visual effects are applied step by step during the entrance preparation time.

12. In any one of paragraphs 8 to 11, At least one processor (861), An HMD device configured to obtain the sleep preparation start time based on the sleep data of the user.

13. In any one of paragraphs 8 to 12, At least one processor (861), An HMD device that controls the display (851) so that the ratio of the virtual screen to the VST screen is gradually reduced during the above-described sleep preparation time, or so that a preset background screen is displayed over time instead of the non-virtual screen during the above-described sleep preparation time.

14. In any one of paragraphs 8 to 12, At least one processor (861), Control the display (851) to display the VST screen in which blue light is differentially adjusted for each object based on the properties of one or more objects included in the VST screen, or to display the VST screen in which brightness is differentially adjusted for each object based on the properties of one or more objects included in the VST screen. An HMD device wherein one or more properties of the objects included in the VST screen include at least one of a first indicator indicating one of the non-virtual screen or the virtual screen including the object, a second indicator indicating whether the object is emitting light, or a third indicator indicating the type of light source of the object.

15. In any one of paragraphs 8 to 14, At least one processor (861), When the above-mentioned entrance preparation start time has arrived, a message is displayed by controlling the above-mentioned display to guide whether to start entrance preparation, An HMD device configured to determine whether to apply the visual effect in stages based on an interaction with the user using at least one sensor (310, 320, 330) or at least one camera.

Images