Head mounted display device and method of controlling the same

Abstract

The method performed by the HMD device may include the following operations: determining a sleep preparation start time; and displaying a sleep preparation screen to which the visual effect is applied in stages on the display during the sleep preparation time from the sleep preparation start time. The visual effect of the staged application can comprise the visual effect of switching a virtual picture output by the whole display area of the display into a VST picture. Here, the VST screen may display a virtual screen with a non-virtual screen as a background on an entire display area of the display, the non-virtual screen may be based on an image captured by the front-facing camera, and the virtual screen may be based on content executed by the HMD device.

Description

Technical Field

[0001] This disclosure relates to a head-mounted display device and a method for controlling the head-mounted display device to induce sleep. Background Technology

[0002] Wearable devices are electronic devices developed to be worn by users (e.g., clothing, shoes, glasses, watches, or rings). Wearable devices can include head-mounted display (HMD) devices. HMD devices can be used as display devices. HMD devices can also be called face-mounted display (FMD) devices.

[0003] HMD devices were originally developed for military use, but based on augmented reality (AR), virtual reality (VR), or video see-through (VST) technologies, they have also been commercialized for the public. For example, HMD devices are used in goggle-style displays that can be worn on the head as interactive devices, allowing users to perform tasks or play games while wearing the device.

[0004] If a user falls asleep while wearing an HMD device, visual stimuli (such as displayed images) or auditory stimuli (such as audio output provided by a content service through a speaker) may interfere with falling asleep and / or achieving deep sleep. Summary of the Invention

[0005] [Technical Solution]

[0006] According to an example, a method performed by a head-mounted display (HMD) device includes: determining a sleep preparation start time and displaying a sleep preparation screen on the display, wherein, starting from the sleep preparation start time, visual effects are applied to the sleep preparation screen in a progressive manner during sleep preparation intervals. The progressively applied visual effects may include switching a virtual image output across the entire display area of ​​the display to a video perspective (VST) screen. Within the entire display area, the VST screen may display the virtual image against a non-virtual image as a background. The non-virtual image may be based on an image captured by a front-facing camera, and the virtual image may be based on content performed by the HMD device.

[0007] According to an example, a head-mounted display (HMD) device includes at least one sensor, at least one camera, a display, and at least one processor including processing circuitry and operatively coupled to the at least one sensor, the at least one camera, and the display. The at least one processor can be configured to: determine a sleep preparation start time and display a sleep preparation screen on the display, wherein, from the sleep preparation start time, during a sleep preparation interval, visual effects are applied to the sleep preparation screen in a progressive manner. The progressively applied visual effects may include a visual effect that switches a virtual image output across the entire display area of ​​the display to a video perspective (VST) screen. Within the entire display area, the VST screen may display the virtual image against a non-virtual image as a background. The non-virtual image may be based on an image captured by a front-facing camera, and the virtual image may be based on content performed by the HMD device.

[0008] According to one or more embodiments, a non-transitory computer-readable medium stores instructions that, when executed by a processor in a head-mounted display (HMD) device, cause the processor to perform a method comprising: determining a sleep preparation start time and displaying a sleep preparation screen on the display, wherein, from the sleep preparation start time, during a sleep preparation interval, visual effects are applied to the sleep preparation screen in a progressive manner. The progressively applied visual effects may include a visual effect of switching a virtual image output across the entire display area of ​​the display to a video perspective (VST) screen. The VST screen may display the virtual image against a non-virtual image as a background across the entire display area. The non-virtual image may be based on an image captured by a front-facing camera, and the virtual image may be based on content performed by the HMD device. Attached Figure Description

[0009] Figure 1 This is a diagram illustrating usage examples of an HMD device according to various embodiments;

[0010] Figure 2 This is a timeline diagram illustrating example timelines for inducing a user to fall asleep in an electronic device according to various embodiments;

[0011] Figure 3 This is a diagram illustrating example transition states of an electronic device according to various embodiments;

[0012] Figure 4 This is a diagram illustrating examples of operations for providing a VST screen in an electronic device according to various embodiments;

[0013] Figures 5a to 5d This is a diagram illustrating an example arrangement of hardware components in an electronic device according to various embodiments;

[0014] Figure 6 This is a diagram illustrating an example of predicting the wearer's fatigue level in an electronic device according to an embodiment;

[0015] Figure 7 This is a diagram illustrating an example of wearing a wearable device in an electronic device according to an embodiment to obtain the wearer's state information;

[0016] Figure 8 This is a block diagram illustrating an example configuration of an electronic device according to an embodiment;

[0017] Figure 9 This is a control flow diagram illustrating an example of inducing sleep in an electronic device according to an embodiment;

[0018] Figure 10 This is a control flow diagram illustrating an example of determining the start time of sleep preparation in an electronic device according to an embodiment;

[0019] Figure 11 This is a control flow diagram illustrating an example of determining the start time of sleep preparation in an electronic device according to an embodiment;

[0020] Figure 12a and Figure 12b This is a control flow diagram illustrating an example process of preparing to fall asleep in an electronic device according to an embodiment;

[0021] Figures 13a to 13c This is a diagram illustrating example images of visual effects that can be applied to each sleep preparation step in an electronic device according to an embodiment;

[0022] Figure 14a It shows the implementation for changing Figure 13b An example of the visual effect of image depth (a) in the image;

[0023] Figure 14b It shows the implementation for changing Figure 13b An example diagram illustrating the visual effect of screen size in (b);

[0024] Figure 15 This is a diagram illustrating an example of visual effects applied to each object in an electronic device according to an embodiment;

[0025] Figures 16a to 16f This is a diagram illustrating an example of applying visual effects in an electronic device according to an embodiment;

[0026] Figure 17 This is a control flow diagram illustrating an example process for inducing sleep in an electronic device according to an embodiment;

[0027] Figure 18aThis is a diagram illustrating an example screen showing the application of visual effects in step 1 of sleep preparation in an electronic device according to an embodiment; and

[0028] Figure 18b This is a diagram illustrating an example of visual effects applied to each object in an electronic device according to an embodiment.

[0029] Throughout the specification and drawings, the same or similar reference numerals may be used to refer to the same or similar elements. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in detail with reference to the accompanying drawings in a manner readily practiced by those skilled in the art. However, this disclosure may be implemented in various other forms and is not limited to the embodiments set forth herein. Throughout the specification and drawings, the same or similar reference numerals may be used to refer to the same or similar elements.

[0031] According to an embodiment, a head-mounted display (HMD) device and its control method are provided, which induces a user to fall asleep by gradually changing external stimuli based on a predicted (e.g., determined) user sleep time.

[0032] According to an embodiment, the HMD device can gradually reduce external stimuli (such as visual or auditory stimuli) based on the user's predicted sleep onset time, thereby providing the user with a sleep environment that enables them to obtain high-quality sleep.

[0033] The technical objectives of this disclosure are not limited to the foregoing, and those skilled in the art can derive other technical objectives from the example embodiments of this disclosure.

[0034] The effects of this disclosure are not limited to the foregoing, and other unmentioned effects will become apparent to those skilled in the art from the following description. In other words, those skilled in the art can also obtain unexpected effects when practicing the embodiments of this disclosure from the exemplary embodiments of this disclosure.

[0035] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementation to the precise forms disclosed. Modifications and variations are possible based on the foregoing disclosure, or may be obtained from practice of the implementation. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Additionally, in the flowcharts and descriptions of operations provided below, it should be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and the order of one or more operations may be interchanged.

[0036] Clearly, the systems and / or methods described herein can be implemented in various forms, including hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, this document describes the operation and behavior of the systems and / or methods without mentioning specific software code—however, it is understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0037] Although specific combinations of features are listed in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible embodiments. In fact, many of these features can be combined in ways not specifically listed in the claims and / or not disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of possible embodiments includes a combination of each dependent claim with all other claims in the claim set.

[0038] The elements, actions, or instructions used herein should not be construed as critical or essential unless explicitly described otherwise. Furthermore, as used herein, the article “a” is intended to include one or more items and may be used interchangeably with “one or more.” The term “a” or similar language is used when intended to indicate only one item. Furthermore, as used herein, the terms “having,” “comprising,” “including,” etc., are intended to indicate open-ended terms. Additionally, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.” Furthermore, expressions such as “at least one of [A] and [B]” or “at least one of [A] or [B]” should be understood to include only A, only B, or both A and B.

[0039] Furthermore, the features, advantages, or characteristics described herein can be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize from the description herein that this disclosure can be practiced without employing one or more specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be present in certain embodiments, and these features and advantages may not be present in all embodiments of this disclosure.

[0040] Figure 1 This is a diagram illustrating an example of the use of an HMD device according to an embodiment.

[0041] refer to Figure 1HMD device 10 can be a device worn by user 20 on his / her head. In this disclosure, "user" or "wearer" may be used interchangeably as a term referring to a person wearing HMD device 10. In an example, HMD device 10 may include a strap around the user's head for securing the HMD to the user's head. In an example, the way HMD device 10 is worn may be similar to wearing glasses. After wearing HMD device 10, wearer 20 can use (e.g., perform) desired content services. This content service may be, for example, a service providing content such as games or movies. This content service may be a streaming content service. Besides HMD device 10 worn on the head by user 20, this embodiment can also be applied in the same or slightly modified way to other devices capable of receiving visual information via a display while user 20 is wearing HMD device 10. For example, in addition to HMD device 10, various embodiments can be applied to electronic devices such as headphones, headsets, glasses, or fisheye lenses that user 20 can wear. In the following description, the term "electronic device 10" will be used to apply various embodiments. The technical meaning of electronic device 10 not only refers to HMD device 10, but can also refer to devices applicable to various embodiments.

[0042] For wearer 20, external stimuli (such as visual or auditory stimuli) may affect sleep quality and / or sleep duration. Therefore, in order for wearer 20 to obtain the desired sleep quality and / or sleep duration, it is preferable to stop the service of electronic device 10 (e.g., turn it off or pause it) and dim (e.g., reduce) the lighting to reduce the degree to which wearer 20 is exposed to external stimuli.

[0043] Electronic device 10 can determine and / or obtain the time when wearer 20 begins to prepare for sleep. For example, falling asleep can be a transition from wakefulness to sleep. Falling asleep can involve multiple sub-stages. Falling asleep can be characterized by changes in subjective, behavioral, cognitive, and physiological levels. Falling asleep can induce non-rapid eye movement (NREM) sleep, but in some cases, it can also transition directly to rapid eye movement (REM) sleep. The sleep preparation start time indicates the time when the sleep preparation operation begins to gradually or progressively reduce external stimuli to guide wearer 20 to sleep. External stimuli can be stimuli that may affect wearer 20 while electronic device 10 is providing content services. External stimuli can be information input through one or more of the five (5) senses (e.g., visual, auditory, olfactory, gustatory, and tactile) that wearer 20 may feel (experience). Visual and / or auditory stimuli can be typical stimuli that wearer 20 can receive due to the operation of electronic device 10. Visual stimuli can be stimuli (e.g., glare) imposed on wearer 20's eyes by images output from a display. Visual stimuli can be gradually or progressively reduced by adjusting at least one requirement of the screen display in the display area (e.g., brightness (illuminance), size, or color). In the following text, for ease of description, the terms “gradual,” “step,” or “progressive” will be used collectively to describe the degree of reduction of external stimuli (e.g., visual or auditory effects). For example, changing a parameter progressively can refer to changing a parameter (e.g., volume or illuminance) by a predetermined amount based on predetermined determined values. For example, reducing volume progressively can refer to reducing the volume of a content service by 3 decibels (dB) every 10 seconds over one (1) minute.

[0044] According to an embodiment, electronic device 10 can obtain a sleep preparation start time based on a wearer 20's target sleep time or target wake-up time. The target sleep time can be a sleep time set by wearer 20 in another electronic device (e.g., smartphone 30 or smartwatch 40). The target sleep time can be set as a sleep alarm. The target wake-up time can be a wake-up time set by wearer 20 in another electronic device 30 or 40. The target wake-up time can be set as a wake-up alarm. If wearer 20 sets a target sleep time for another electronic device (30 or 40), electronic device 10 can obtain a sleep preparation start time based on either the target sleep time or the target wake-up time. The target sleep time can be a time period (e.g., a time interval) between the target sleep time and the target wake-up time.

[0045] Electronic device 10 can obtain a sleep preparation start time by at least one of the following: a set sleep target time, or a sleep target time predicted based on a wake-up target time and a sleep preparation time (e.g., a time interval). The sleep preparation time interval can be the entire time during which electronic device 10 applies visual effects in a stepwise manner. For example, assuming a sleep target time of 8 p.m. and a sleep preparation time interval of 20 minutes, electronic device 10 can determine (or obtain) 7:40 p.m. as the sleep preparation start time (taking into account the 20-minute sleep preparation time interval relative to 8 p.m.). For example, assuming a target sleep time of 8 hours, a wake-up target time of 6 a.m., and a sleep preparation time interval of 20 minutes, electronic device 10 can determine or obtain 9:40 p.m. as the sleep preparation start time based on a 20-minute sleep preparation time interval relative to 10 p.m. (i.e., eight (8) hours) before 6 a.m. If both a sleep target time and a wake-up target time are provided, electronic device 10 can obtain the sleep preparation start time by prioritizing the sleep target time.

[0046] According to an embodiment, electronic device 10 can determine the sleep preparation start time based on sleep data recorded for wearer 20. For example, electronic device 10 can collect sleep data of wearer 20 from wearable device 40 (e.g., a smartwatch) worn by wearer 20. In the example, the sleep data can provide an estimate of the user's sleep status based on the user's movement or heart rate. For example, if the user's movement is below a movement threshold or the user's heart rate is below a heart rate threshold within a predetermined time period, it can be determined that the user is asleep. For example, if the predicted sleep onset time for wearer 20 is 10 p.m. based on sleep data analysis, electronic device 10 can determine or obtain 9:40 p.m. as the sleep preparation start time based on a 20-minute sleep preparation time interval relative to 10 p.m. For example, if the wearer 20 is predicted to sleep for eight (8) hours by analyzing sleep data and is predicted to wake up at 6 a.m., the electronic device 10 can determine or obtain 9:40 p.m. as the start time for sleep preparation based on a 20-minute sleep preparation interval relative to 10 p.m. (i.e., eight (8) hours before 6 a.m.).

[0047] According to an embodiment, electronic device 10 can determine that the sleep preparation start time has been reached based on a measurement of fatigue level meeting a predetermined threshold. For example, if the fatigue level obtained by fatigue measurement reaches the predetermined threshold, electronic device 10 can determine that the sleep preparation start time has been reached (e.g., started). In the example, electronic device 10 can measure the fatigue level of wearer 20. Electronic device 10 can measure the fatigue level of wearer 20 periodically or non-periodically. To measure fatigue level periodically, electronic device 10 can be configured with a predetermined period for measuring fatigue level. To measure fatigue level non-periodically, electronic device 10 can monitor the occurrence of non-periodic events. A non-periodic event could be, for example, a sudden up-and-down movement of the wearer 20's head. At least one sensor can detect this movement by sensing (e.g., detecting) a signal.

[0048] Electronic device 10 can adjust the sleep preparation start time based on measured fatigue levels. As an example, electronic device 10 can change the sleep preparation start time by an amount proportional to the increase or decrease in fatigue levels. For example, if the adjustment unit is 10 minutes, then when the fatigue level increases by a threshold, electronic device 10 can decrease the sleep preparation time by 10 minutes. For example, if the sleep preparation start time is 9:40 PM and the fatigue level increases by a threshold, electronic device 10 can change the sleep preparation start time to 9:30 PM. If the fatigue level further increases by a threshold, the electronic device can change the sleep preparation start time to 9:20 PM. For example, if the sleep preparation start time is 9:40 PM and the fatigue level decreases by a threshold, electronic device 10 can change the sleep preparation start time to 9:50 PM.

[0049] To measure fatigue levels, electronic device 10 can measure the duration of eye closure of wearer 20, the number of blinks of wearer 20, the pupil diameter of wearer 20, the surface temperature of the eyeball of wearer 20, or the variability of heart rate of wearer 20. Electronic device 10 can measure the duration of eye closure, blink count, pupil diameter, or surface temperature of at least one eye of wearer 20 based on signals sensed by at least one sensor. Electronic device 10 can measure the variability of heart rate of wearer 20 based on information provided by wearable device 40 worn by wearer 20. Electronic device 10 can obtain a fatigue level value indicating the degree of fatigue based on the duration of eye closure, blink count, pupil diameter, surface temperature of the eyeball, or heart rate variability of at least one eye.

[0050] Electronic device 10 can display a message on its screen indicating whether to begin sleep preparation when sleep preparation begins. For example, the message could be "Ready to sleep. Activate sleep preparation mode?" Or, a guidance message could be "High fatigue level. Activate fatigue minimization mode?" Electronic device 10 can display a button (e.g., a "Yes" button) indicating whether to activate sleep preparation mode, and / or a button (e.g., a "No" button) indicating whether not to activate sleep preparation mode. Electronic device 10 can determine whether to activate sleep preparation mode based on interaction with wearer 20.

[0051] At the start of sleep preparation time, electronic device 10 can gradually reduce external stimuli over a predetermined period (e.g., sleep preparation time) to improve the sleep quality or depth of wearer 20. The method of gradually reducing external stimuli can be determined, for example, based on the type of content service being used by wearer 20. The method of gradually reducing external stimuli can be determined, for example, based on the display mode applied to the monitor. Accordingly, electronic device 10 can provide a sleep environment (e.g., a virtual environment) that wearer 20 can easily switch to in sleep mode. For example, electronic device 10 can output a sleep preparation screen through the monitor, which applies visual effects that can gradually reduce visual stimuli over time. Electronic device 10 can switch from a virtual image output across the entire display area of ​​the monitor (e.g., a virtual image according to immersion mode) to a VST image (e.g., a virtual image according to partial immersion mode) of the sleep preparation screen, thereby applying visual effects. The immersion mode used to output the virtual image across the entire display area of ​​the monitor can be, for example, a VR mode. In VR mode, electronic device 10 can output images through a 360° shielded monitor. In the example, a VST image refers to an image output through the monitor according to the switch from immersion mode to partial immersion mode. According to a partial immersion mode, VST images can reduce the wearer's visual immersion by 20%, thereby reducing the activation of the sympathetic nervous system (e.g., nerves responsible for responding to dangerous or stressful situations). The degree of visual immersion can be proportional to the size of the image output through the display. For example, the degree of immersion can increase with increasing image size and decrease with decreasing image size.

[0052] A VST screen can be a screen that includes a virtual scene (or image) against a real scene (or image) (e.g., a non-virtual scene or image). The VST screen can be a full-screen display covering the entire display area of ​​the monitor. A real-world image can be created based on an image of a real space (e.g., a space or place including objects) captured by a front-facing camera. A virtual image can be created based on a content service being used by a user wearing electronic device 10. In the example, electronic device 10 outputs a VST screen to maintain the visual realism provided by the content service, taking into account the properties of objects included in the virtual or real-world images. To this end, the VST screen can maintain one or more objects of the content service to prevent sudden interruptions in viewing the content service and thus prevent the wearer from being startled.

[0053] If the visual effects are applied to the VST screen in a progressive manner, the electronic device 10 can progressively reduce the proportion of the virtual image in the VST screen over time. The electronic device 10 can progressively reduce the size of the virtual image in the VST screen. The electronic device 10 can progressively increase the depth of the virtual image in the VST screen. To increase the depth of the virtual image, the electronic device 10 can move the virtual image away from the wearer 20 across the entire screen representing 3D space. In this way, even if the wearer 20 remains stationary, the depth of the virtual image can be adjusted to make it appear as if the scene is moving away from the user.

[0054] If the visual effects are applied to the VST screen in a progressive manner, the electronic device 10 can output a VST screen with the following visual effects, rather than a real screen: the visual effect of displaying a preset background screen over time. For example, if the proportion of the virtual screen in the VST screen decreases, the visual stimulation received by the wearer 20 may be reduced.

[0055] Electronic device 10 can provide a visual effect that adjusts blue light differently for each object included in a VST screen based on the object's attributes. Blue light may stimulate the brain, affect circadian rhythms, and interfere with (e.g., disrupt) sleep. Electronic device 10 can apply a visual effect to the VST screen that gradually increases the blue light adjusted differently for each object over time. The object's attributes may include a first indicator for indicating one of a real or virtual screen including the corresponding object. The object's attributes may include a second indicator for indicating whether the corresponding object is emitting light. The object's attributes may include a third indicator for indicating the type of light source for the corresponding object. The object's attributes may include at least one of the following: a first indicator (indicating one of a real or virtual screen including the corresponding object), a second indicator (indicating whether the corresponding object is emitting light), or a third indicator (indicating the type of light source for the corresponding object). The object's attributes may include a first indicator (indicating whether a real or virtual screen including the corresponding object), a second indicator (indicating whether the corresponding object is emitting light), and a third indicator (indicating the type of light source for the corresponding object).

[0056] Electronic device 10 can take into account the attributes of objects and provide a visual effect that adjusts the illuminance differently for each object included in the VST screen. Illuminance may stimulate the brain, affect circadian rhythms, and interfere with (e.g., disrupt) sleep. Electronic device 10 can apply to the VST screen a visual effect that gradually increases the illuminance adjusted differently for each object over time. The attributes of the object may include a first indicator for indicating one of the real or virtual screens including the corresponding object. The attributes of the object may include a second indicator for indicating whether the corresponding object is emitting light. The attributes of the object may include a third indicator for indicating the type of light source for the corresponding object. The attributes of the object may include at least one of the first indicator (indicating one of the real or virtual screens including the corresponding object), the second indicator (indicating whether the corresponding object is emitting light), or the third indicator (indicating the type of light source for the corresponding object). The attributes of the object may include a first indicator (indicating whether the real or virtual screen includes the corresponding object), a second indicator (indicating whether the corresponding object is emitting light), and a third indicator (indicating the type of light source for the corresponding object).

[0057] As described above, if the electronic device 10 adjusts the blue light and / or illuminance for each object on the VST screen in a progressive manner over time (e.g., progressively adjusting the blue light and / or illuminance over a predetermined time period), the level of disturbance to the wearer 20 when falling asleep can be gradually reduced while maintaining the realism of the vision.

[0058] If the electronic device 10 performs a sleep preparation function by gradually reducing external stimuli over time during a sleep preparation interval, the electronic device 10 can also gradually reduce the volume of the sound output generated by the content service. The electronic device 10 can perform the operation of gradually reducing external stimuli over time and can also gradually change the sound source output. For example, the electronic device 10 can operate to output sleep-inducing sounds (e.g., sleep aid sounds) proportionally to the passage of time.

[0059] Figure 2 An electronic device according to an embodiment is shown (e.g., Figure 1 The timeline of an example timeline for HMD device 10 inducing a user to fall asleep. Figure 3 This is a diagram illustrating an example state transition of an electronic device according to an embodiment.

[0060] refer to Figure 2 or Figure 3 Wearer 20 may fall asleep while wearing electronic device 10. Electronic device 10 can monitor wearer 20's sleep status (e.g., state or process). Electronic device 10 can perform operations to guide (e.g., induce) wearer 20 to fall asleep based on wearer 20's sleep status (e.g., state or process). Operations to guide wearer 20 to fall asleep can be divided into normal operating steps (e.g., Figure 3 Activity status 310), sleep preparation steps (e.g., Figure 3 Sleep preparation state 320) or sleep steps (e.g., Figure 3 Sleep state 330 in the middle. In addition, idle steps may also be included (e.g., Figure 3 Idle state 340 in the middle.

[0061] Normal operation step 310 can be a step in which the electronic device 10 provides visual and / or auditory information to the wearer 20 based on content services. For example, visual information can be provided to the wearer 20 by outputting visual information about the screen over the entire or a portion of the display. Auditory information can be provided to the wearer 20 via, for example, a built-in speaker or an external speaker. Normal operation step 310 can be an operational state performed by the electronic device 10 before the sleep preparation start time arrives. Normal operation step 310 can also be an operational state performed by the electronic device 10 when the wearer 20 is awake.

[0062] Normal operating steps 310 can be the steps that electronic device 10 performs during normal operating time T1. Normal operating time T1 can be the time period corresponding to the normal operating steps when the wearer 20 is awake, during which the sympathetic nervous system is more active than the parasympathetic nervous system (e.g., the nerves responsible for relaxing the wearer). The main functions of the sympathetic nervous system can include pupil dilation, inhibition of salivation, increased heart rate, bronchial dilation, inhibition of digestive juice secretion, inhibition of adrenaline or noradrenaline secretion, or bladder contraction. The main functions of the parasympathetic nervous system can include pupil constriction, stimulation of salivation, slowing of heart rate, bronchial constriction, stimulation of digestive juice secretion, stimulation of bile secretion, or bladder contraction. In this example, the main activity of the sympathetic nervous system may be activated when the person is awake. In this example, the main activity of the parasympathetic nervous system may be activated when the person is asleep.

[0063] Normal operating time T1 can be defined as the period from when user 10 puts on electronic device 10 or starts subscribing to content services after putting on electronic device 10 to sleep preparation start time 220. Normal operating time T1 can also be defined as the period from when wearer 10 wakes up (e.g., referred to as "wake-up time") to when wearer 10 begins preparing to fall asleep at time 220.

[0064] In normal operating step 310, electronic device 10 can determine and / or identify the start time of sleep preparation (operation 212 or operation 311). (See above for reference.) Figure 1 The electronic device 10 can determine the start time for sleep preparation based on preset time information (e.g., target time for falling asleep or target time for waking up) or the wearer's level of fatigue.

[0065] In normal operation step 310, electronic device 10 can monitor and determine whether the sleep preparation start time has been reached. When the time reaches sleep preparation start time 220, electronic device 10 can switch from normal operation step 310 to sleep preparation step 320 (e.g., operation 313). If a request to switch to sleep mode occurs in normal operation step 310, electronic device 10 can switch from normal operation step 310 to sleep step 330 (operation 315). If wearer 20 instructs to switch to sleep step 310 to enter sleep mode automatically, a sleep mode switching request may be generated.

[0066] Sleep preparation step 320 can be a step where sleep preparation start time 220 has been reached (operation 313), causing electronic device 10 to reduce (or decrease) sympathetic activation in wearer 20 and gradually reduce external stimulation over time (e.g., N seconds, where N is a positive integer) (operations 230-1 to 230-n or operation 323). Electronic device 10 can switch from normal operation step 310 to sleep preparation step 320 in response to the reaching of sleep preparation start time (operation 313). Electronic device 10 can switch from sleep preparation step 320 to normal operation step 310 in response to a sleep preparation deactivation request (operation 321). For example, a deactivation request may occur because electronic device 10 was not selected to perform sleep preparation mode based on interaction with wearer 20.

[0067] Sleep preparation step 320 may be an operation performed by electronic device 10 during sleep preparation interval T2 to guide (or induce) the wearer 20 to fall asleep. Sleep preparation interval T2 may be the time period during which the sleep preparation step is performed to prepare for the transition from the normal operating time T1, where the sympathetic nervous system is more active, to the sleep time T3, where the parasympathetic nervous system is more active. Sleep preparation interval T2 may be defined as the time period from sleep preparation start time 220 to sleep entry time 240 (e.g., referred to as "sleep time").

[0068] Electronic device 10 may progressively reduce the external stimuli to be applied to wearer 20 during the sleep preparation start time (e.g., operation 230-1 to 230-n or operation 323) during the sleep preparation interval T2. Electronic device 10 may progressively output a sleep preparation screen based on time, during which the visual stimuli applied to wearer 20 on the sleep preparation screen are progressively reduced. The sleep preparation screen may be a VST screen based on a partial immersion mode. The VST screen can reduce the degree of visual immersion of wearer 20 to reduce sympathetic nerve activation. The degree of visual immersion may be proportional to the size of the screen output through the display. For example, the degree of immersion may increase with increasing screen size and decrease with decreasing screen size.

[0069] A VST screen can display a virtual image against a real-world background (e.g., a real location or space). The real-world image can display an image of the real space captured by the electronic device 10. The virtual image can provide content services. The electronic device 10 can output a VST screen such that the realistic visual experience provided by the content service is maintained, taking into account the attributes of objects included in the virtual image or the real image.

[0070] Electronic device 10 can gradually reduce the proportion of the virtual image in the VST image over time (operations 230-1 to 230-n). For example... Figure 2 As shown, the end time of each of the n sleep preparation steps (e.g., sleep preparation step 1 to sleep preparation step n) can correspond to the start time of the next sleep preparation step. For example, based on the operation of sleep preparation step 1, the start time can be determined in the first time period T. 2,1 This process is executed during a period from the start time of sleep preparation (time 220) to the end time of sleep preparation step 1 (time 230-1). For example, based on the operation of sleep preparation step n, it can be performed in the (n-1)th time period T. 2,n-1 The process is performed during a period from the start time 230n-1 of sleep preparation step n-1 to the end time 230-n of sleep preparation step n. In the example, the sleep preparation time interval T2 can be defined as an interval containing n steps, where these steps can be equally divided. For example, if the sleep preparation time interval is 10 minutes and contains 10 equally divided steps, the visual effects applied in a stepwise manner can be applied at intervals of one (1) minute, where step 1 includes the sleep preparation start time. In the example, the steps are not applied at equal intervals during the interval corresponding to the sleep preparation time interval.

[0071] Electronic device 10 can switch from immersive mode to partial immersive mode at the sleep preparation start time 220. Immersive mode can be a display mode that outputs virtual images across the entire display area of ​​the monitor. Partial immersive mode can be a display mode that outputs VST images across the entire display area of ​​the monitor. Electronic device 10 can switch from immersive mode to partial immersive mode at the first time period T. 2,1 During this period, the initial VST screen is output to the monitor, and this time period is used to perform the operations according to step 1 of the sleep preparation.

[0072] Electronic device 10 can be used in the second time period T 2,2 up to the (n-2)th time interval T 2,n-2 During this period (including the execution of sleep preparation steps 2 to n-1), a VST image is output to the display, in which the size of the virtual image gradually decreases. The electronic device 10 can gradually increase the depth of the virtual image in the VST image. This operation provides the effect of moving the virtual image away from the wearer 20 in three-dimensional (3D) space. The electronic device 10 can gradually decrease the proportion of the virtual image in the VST image. This provides the effect of reducing the size of the virtual image in two-dimensional (2D) space.

[0073] Electronic device 10 can apply visual effects (e.g., blue light or illuminance) during the (n-1)th time period T of performing sleep preparation step n. 2,n-1The electronic device 10 differentiates between each object included in the VST screen. As an example, the electronic device 10 can adjust the blue light differently for each object included in the VST screen, taking into account the object's attributes. Blue light may stimulate the brain, affect circadian rhythms, and interfere with (e.g., disrupt) sleep. The electronic device 10 can gradually increase the differently adjusted blue light for each object over time. The electronic device 10 can also adjust the illuminance differently for each object included in the VST screen, taking into account the object's attributes. Illuminance may stimulate the brain, affect circadian rhythms, and interfere with (e.g., disrupt) sleep. The electronic device 10 can gradually increase the differently adjusted illuminance for each object over time.

[0074] As described above, if the electronic device 10 adjusts the blue light and / or illuminance for each object on the VST screen in a progressive manner over time, the degree of interference with the wearer 20's sleep can be reduced sequentially while maintaining a realistic visual experience.

[0075] In the example, if electronic device 10 performs a sleep preparation function that progressively reduces external stimuli over time during a sleep preparation interval, electronic device 10 can progressively reduce the volume of sound output caused by the content service. Electronic device 10 can perform this method of progressively reducing external stimuli over time by progressively changing the sound source output. For example, electronic device 10 can operate to output sleep-inducing sounds (e.g., sleep aid sounds) proportionally to the passage of time.

[0076] In sleep preparation step 320, electronic device 10 can monitor and determine whether the sleep preparation time interval has passed. If the sleep preparation time interval has passed, electronic device 10 can switch from sleep preparation step 320 to sleep step 330 (operation 325). The sleep step can correspond to the steps after the brain switches to a sleep state.

[0077] The sleep time T3 during which the electronic device 10 operates in sleep step 330 can be a period of time in which parasympathetic activity is stronger than sympathetic activity and the wearer 20 is asleep (e.g., a sleep duration). Sleep time T3 can be defined as the period from sleep onset time 240 to wearer 20 waking up. For example, if a wake-up state switching request is generated in sleep step 330, the electronic device 10 can switch the state from sleep step 330 to normal operation step 310 (operation 331). A wake-up state switching request may be generated when wearer 20 wakes up on their own (e.g., without external assistance or help) and instructs to resume content services.

[0078] Electronic device 10 can monitor whether a deactivation request has been generated in each of the normal operation steps 310, sleep preparation steps 320, or sleep steps 330. A deactivation request can be generated by the wearer 20 instructing them to terminate the content service. If a deactivation request is generated, electronic device 10 can switch to an idle state 340 (operations 317, 327, and 333).

[0079] If an activation request is generated in idle state 340, the electronic device 10 can switch from idle state 340 to active state 310 (operation 341). The activation request can be instructed by the wearer 20 to resume content services that have terminated themselves.

[0080] Figure 4 This illustrates an electronic device according to an embodiment (e.g., Figure 1 A diagram showing an example of operation of the VST screen provided in the HMD device 10.

[0081] refer to Figure 4 The electronic device 10 may include a camera 410, a head tracker 420, an image generator 430, an image synthesizer 440, or a monitor 450.

[0082] Camera 410 can be configured to face forward while user 20 is wearing electronic device 10. Camera 410 can output image 411 obtained by capturing real space or location 460 (or "real view"). Real space or real view can be the space where user 20 is actually (e.g., physically).

[0083] The head tracker 420 can track the position of a moving head while the user 20 wears the electronic device 10. The head tracker 420 can output head position information (e.g., head locator) 421 based on the tracking results.

[0084] The image generator 430 can generate a virtual image 431 for the user 20 based on the content service and taking into account the head position information provided by the head tracker 420 on the virtual image.

[0085] The image compositor 440 can generate a composite image (e.g., a VST image) 441 by combining (e.g., compositing) the real image 411 provided by the camera 410 and the virtual image 431 provided by the image generator 430. The composite image 441 generated by the image compositor 440 can be transmitted to the monitor 450. In the example, the composite image 441 can display one or more objects corresponding to the real image 411 as a background, while the virtual image 431 containing one or more objects is superimposed on the real image 411.

[0086] The monitor 450 can output the composite image 441 sent from the image compositer 440 as visual information. The user 20 can view the composite image 441 output from the monitor 450.

[0087] Figure 5a , Figure 5b , Figure 5c and Figure 5d This illustrates an electronic device according to an embodiment (e.g., Figure 1 A diagram showing an example layout of the hardware components in the HMD device 10.

[0088] refer to Figures 5a to 5d Electronic device 10 may include a front portion exposed to the outside or facing the wearer (e.g., Figure 1 The inner side (e.g., surface) of the wearer 20. External sensor unit (e.g., Figure 8 The external sensor unit 810 can be configured at the front of the electronic device 10 (see [reference]). Figure 5a The external sensor unit includes left / right depth cameras 811a and 811b (e.g., Figure 8 The depth camera 811 in the middle), and the left / right cameras 813a and 813b (e.g., Figure 8 Side camera 813), left / right downward-looking cameras 815a and 815b (e.g., downward-looking camera 815) or left / right infrared (IR) sensors 817a and 817b (e.g., Figure 8 (IR sensor 817 in the middle). The frontal positions of the left / right depth cameras 811a and 811b, left / right side cameras 813a and 813b, left / right downward-facing cameras 815a and 815b, or left / right IR sensors 817a and 817b are not necessarily fixed, but can be determined during the design phase of the electronic device 10 (e.g., during the design process). Those skilled in the art will understand that the positions of the components of the electronic device 10 can be arranged to optimize the information acquisition and performance of the electronic device 10.

[0089] Eye recognition unit (e.g., Figure 8 The eye recognition unit 820 can be configured on the inner surface of the electronic device 10, and the eye recognition unit includes an IR camera 821 (e.g., Figure 8 The IR camera 821 or LED ring 823 (e.g., in the middle) Figure 8 LED ring 823 in the middle). Sound unit (e.g., Figure 8 The sound unit 840 in the electronic device 10 can be configured on the inside (e.g., surface) of the electronic device 10 (see [reference]). Figure 5b The sound unit includes a microphone 841 (e.g., Figure 8 The microphone 841 or speaker 843 (e.g.) in the microphone 841 or speaker 843 in the speaker 843 Figure 8(Speaker 843 in the middle).

[0090] Electronic device 10 may include at least one processor 861 (e.g., Figure 8 The processor 861 in the device 10 may include a location identification unit (e.g., a processor 861 in the device 10). Figure 8 The position identification unit 830 includes an inertial measurement unit (IMU) sensor 831 (e.g., Figure 8 The IMU sensor 831 or ultra-wideband (UWB) 833 (e.g., Figure 8 (UWB833 in) (see UWB833) Figure 5c ).

[0091] Display unit (e.g., Figure 8 The display unit 850 can be configured on the inside (e.g., surface) of the electronic device 10, and the display unit includes a display 851 (e.g., Figure 8 The display 851 or lens 853 (e.g., in the middle) Figure 8 (See shot 853) Figure 5d In the example, display 851 may be a single display, wide or long enough to cover the user's eyes. In the example, electronic device 10 may include a separate display 851 for each of the user's eyes.

[0092] Figure 6 This illustrates an electronic device according to an embodiment (e.g., Figure 1 In HMD devices 10, the wearer is predicted (e.g., Figure 1 The figure shows an example of the fatigue level of the wearer (20).

[0093] refer to Figure 6 The electronic device 10 may include one or more sensor modules (e.g., an IR camera module). Figure 5b The sensor module, comprising the first to fourth IR cameras 821a, 821b, 821c, and 821d, is configured to face the wearer 20's face (e.g., towards the eyes). For example, at least one first IR camera (e.g., Figure 5b The third IR camera 821c or the fourth IR camera 821d can be housed inside the electronic device 10, facing the right eye. At least one first IR camera 821c or 821d can be an IR camera for the right eye. For example, at least one second IR camera (e.g., Figure 5b The first IR camera 821a or the second IR camera 821b may be disposed inside the electronic device 10 and oriented toward the left eye. At least one of the second IR cameras 821a or 821b may be an IR camera for the left eye.

[0094] Electronic device 10 can obtain eye images of wearer 20 (e.g., right eye image 610a and / or left eye image 610b) based on images captured by at least one first IR camera 821c or 821d and / or at least one second IR camera 821a or 821b. For example, electronic device 10 can obtain right eye image 610a of wearer 20 based on images captured by at least one first IR camera 821c or 821d. For example, electronic device 10 can obtain left eye image 610b of wearer 20 based on images captured by at least one second IR camera 821a or 821b.

[0095] Electronic device 10 can obtain a first reference point 630a (hereinafter referred to as the "right eye reference point") from the right eye image 610a. Electronic device 10 can obtain information or data related to whether the right eye reference point 630a matches the first alignment point 620a (hereinafter referred to as the "right eye alignment point"). Electronic device 10 can obtain a second reference point 630b (hereinafter referred to as the "left eye reference point") from the left eye image 610b. Electronic device 10 can obtain information or data related to whether the left eye reference point 630b matches the second alignment point 620b (hereinafter referred to as the "left eye alignment point"). Electronic device 10 can determine whether the wearer 20 is asleep based on whether the right eye reference point 630a matches the right eye alignment point 620a. Electronic device 10 can determine the wearer 20's fatigue level based on whether the left eye reference point 630b matches the left eye alignment point 620b. Electronic device 10 can determine the wearer 20's fatigue level based on one or more sensor modules (e.g., an IR camera module). Figure 5b The first to fourth IR cameras (821a, 821b, 821c and 821d) detect the sensing signals to obtain the duration of the wearer's closed eyes, the number of blinks, the pupil diameter or the surface temperature of the eyeball.

[0096] Electronic device 10 can determine a fatigue level value indicating the wearer 20's fatigue level based on whether the right eye reference point 630a matches the right eye alignment point 620a and subsequent information. Information items used to determine the wearer 20's fatigue level may include eye closure duration (e.g., duration of eye closure), blink count, pupil diameter, ocular surface temperature, or heart rate variability. Electronic device 10 can also be based on a camera (e.g., ...). Figure 8 The duration of eye closure is measured by images captured by the IR camera 821 in the device. Electronic device 10 can use images captured by the camera (e.g., for identification purposes) to measure the duration of eye closure. Figure 8 The electronic device 10 measures the number of blinks by processing images of the pupil captured by an IR camera 821. In this example, a high-speed camera can be used as the camera for measuring the number of blinks. The electronic device 10 does not collect information for measuring the number of blinks while interacting with the user. The electronic device 10 can collect information from an IR camera (e.g., ...) Figure 8 The electronic device 10 removes reflected light caused by IR illumination from images captured by the IR camera 821 and measures the pupil diameter based on the distribution of black pixels in a candidate image group after removing reflected light. The electronic device 10 can extract the eye position from images captured by the thermal imaging camera and predict (e.g., determine or estimate) the temperature measured at the extracted eye position as the eye surface temperature. The electronic device 10 can obtain heart rate variability in real time based on information provided by a wearable device (e.g., a smartwatch).

[0097] Table 1 shows examples of predicting wearer 20's fatigue level based on wearer 20's eye closure duration, blink count, pupil diameter, ocular surface temperature, or heart rate variability.

[0098] [Table 1]

[0099]

[0100] In Table 1, in the example, evaluation results are defined, assuming that the measured values ​​(N1, N2, N3, N4, and N5) are compared with reference thresholds (N_th1, N_th2, N_th3, N_th4, and N_th5), respectively. As an evaluation result, for example, if the measured values ​​(N1, N2, N3, N4, and N5) are greater than or equal to the reference thresholds (N_th1, N_th2, N_th3, N_th4, and N_th5), the value assigned is set to "1," which is an indicator of increased fatigue. As an evaluation result, for example, if the measured values ​​(N1, N2, N3, N4, and N5) are less than or equal to the reference thresholds (N_th1, N_th2, N_th3, N_th4, and N_th5), the value assigned is set to "0," which is an indicator of no increased fatigue. If a predetermined number or more data anomalies are determined, the electronic device 10 can determine that the wearer 20 has a high level of fatigue. As an example, if the measurement value of the corresponding item increases by a threshold compared to the wearing time, the electronic device 10 can advance the sleep start time by a predetermined amount of time (e.g., N minutes, where N is a positive integer) from the wearer 20's average daily wearing time over a week. For example, if the sleep start time is 9:40 p.m., and the measurement value of the corresponding item increases by a threshold, the sleep start time can be adjusted to 9:30 p.m., advancing the sleep start time by five (5) minutes.

[0101] Figure 7 This illustrates an electronic device according to an embodiment (e.g., Figure 1 In the HMD device 10), wearable devices (e.g., Figure 1 Wearable devices 40 in the middle) obtain information about the wearer (e.g., Figure 1A diagram illustrating an example of the status information of the wearer (20) in the image.

[0102] refer to Figure 7 Wearable device 40 can be provided in various forms, such as glasses, a ring, or a watch. Wearable device 40 may include, for example, a smartwatch 700. Smartwatch 700 can be worn on a user's wrist. Smartwatch 700 may include various sensors. For example, smartwatch 700 may include an electrophysiological sensor 710, an optical heart rate sensor 720, an accelerometer, and a gyroscope sensor.

[0103] The smartwatch 700 can acquire a user's motion and / or biometric information (e.g., heart rate, respiration, pulse, and electrocardiogram) based on sensing signals detected by at least one sensor. The smartwatch 700 can determine whether the user 10 is asleep (e.g., in a sleep state) based on the acquired information (e.g., heart rate variability). The smartwatch 700 can predict (e.g., determine or estimate) the level of fatigue of the sleeping user 20 based on the acquired information. In this example, the smartwatch 700 can transmit the information it collects to the electronic device 10.

[0104] Figure 8 This illustrates an example electronic device according to an embodiment (e.g., Figure 1 Block diagram of HMD device 10 in the middle. Figure 8 Each unit shown can be implemented by an independent circuit.

[0105] refer to Figure 8 The electronic device 10 may include an external sensor unit 810, an eye recognition unit 820, a position recognition unit 830, a sound unit 840, a display 850, a data transmission / reception unit (e.g., a communication module including communication circuitry) 860, or a memory 870. The external sensor unit 810 may include a depth camera 811, a side-view camera 813, a downward-view camera 815, or an IR sensor 817. The eye recognition unit 820 may include an IR camera 821 or an LED ring 823. The position recognition unit 830 may include an IMU sensor 831 or a UWB 833. The sound unit 840 may include a microphone 841 or a speaker 843. The display unit 850 may include a display 851 or a lens 853. The data transmission / reception unit (communication module) 860 may include at least one processor 861 (e.g., including processing circuitry). The memory 870 may include a user identification module 871, a function control module 873, or a control transfer module 875.

[0106] In this example, the external sensor unit 810 may include at least one sensor. The external sensor unit 810 may output a sensing signal for identifying (e.g., marking) an external object. The external sensor unit 810 may detect the operating state of the electronic device 10 (e.g., power supply or temperature), the external environmental state (e.g., user state), or hand or gaze movements for user interaction, and may generate electrical signals or data values ​​corresponding to the detected state. In addition to the depth camera 811, side camera 813, downward-facing camera 815, or IR sensor 817, the external sensor unit 810 may also include a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor. The depth sensor 811 may output a sensing signal corresponding to the distance to the object. The side camera 813 may output an image obtained by photographing the outer side surface of the electronic device 10. The downward-facing camera 815 may output an image obtained by photographing the area below the electronic device 10. The IR sensor 817 can output a sensing signal that can identify (e.g., identify) the presence of an object.

[0107] In the example, the eye recognition unit 820 may include an IR camera 821 or an LED ring 823 positioned toward the wearer 20. The IR camera 821 may output images of the wearer 20's eyes and the surrounding environment. The LED ring 823 may output sensing signals for checking (e.g., monitoring) the wearer 20's eye alignment.

[0108] In the example, the position identification unit 830 may include an IMU sensor 831 or a UWB 833 for obtaining position information of the electronic device 10. The IMU sensor 831 can obtain position information about the electronic device 10 on a three-dimensional coordinate axis. The IMU sensor 831 can output the obtained position information as a sensing signal. The UWB 833 can obtain position information about the electronic device 10 based on UWB technology. The UWB 833 can output a sensing signal corresponding to the obtained position information.

[0109] In the example, external sensor unit 810, eye recognition unit 820, or position recognition unit 830 can constitute a sensor module. The sensor module can detect the operating state of electronic device 10 (e.g., power supply or temperature) or the environmental state outside electronic device 101 (e.g., user state such as head or eye movements), and then generate an electrical signal or data value corresponding to the detected state. The sensor module may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, IR sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0110] In the example, the sound unit 840 may include a microphone 851 for converting external audio into electrical signals, or a speaker 843 for converting electrical signals into audible audio signals. The speaker 843 may generate separate guide tones under the control of the processor 861.

[0111] In the example, the display unit 850 can project onto the outside of the electronic device 10 (e.g., to the wearer, e.g., Figure 1 The wearer 20 in the display unit 850 provides visual information (e.g., real or virtual image information) in a visual manner. The display unit 850 may include, for example, a display 851, a lens 853, a holographic projection device or projector, and control circuitry for controlling the corresponding device. The display unit 850 may include a touch sensor configured to detect touch, or a pressure sensor configured to measure the intensity of the force generated by a touch. The display unit 850 can be controlled via an electrical connection to the processor 861.

[0112] In the example, the data sending / receiving unit 860 may include at least one processor 861. The processor 861 may execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) included in the electronic device 10, and may perform various data processing or calculations. As at least part of various data processing or calculations, the processor 861 may store commands or data received from other components (e.g., external sensor unit 810, eye recognition unit 820, position recognition unit 830, or sound unit 840) in volatile memory 871, process the commands or data stored in volatile memory 871, and store the processed data in non-volatile memory 873. As at least part of various data processing or calculations, the processor 861 may process commands or data received from other components (e.g., external sensor unit 810, eye recognition unit 820, position recognition unit 830, or sound unit 840) and output the resulting data via display unit 850.

[0113] In the example, processor 120 may include a main processor (e.g., a central processing unit or application processor) or an auxiliary processor (e.g., a graphics processing unit, neural processing unit, image signal processor, sensor central processor, or communication processor (e.g., a communication module)). The auxiliary processor may operate independently of the main processor or in conjunction with the main processor. For example, if electronic device 10 includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power consumption than the main processor or specified for a particular function. The auxiliary processor may be implemented independently of the main processor or as part of the main processor.

[0114] In the example, the data sending / receiving unit 860 may include a communication module. The communication module may support communication with another device (e.g., Figure 1 The smartphone 30 or wearable device 40 establishes a direct (e.g., wired) or wireless communication channel and performs communication through the established communication channel. The communication module may include one or more communication processors that operate independently of processor 861 and support direct (e.g., wired) or wireless communication. The communication module may include, for example, a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system communication module) or a wired communication module (e.g., a local area network (LAN) communication module or a power line communication module). Corresponding communication modules in these communication modules can communicate via a network (e.g., such as Bluetooth). TM Communication modules can communicate with external devices via short-range communication networks such as Wi-Fi Direct or IR Data Association (IrDA), or long-range communication networks such as traditional cellular networks, 5G networks, next-generation communication networks, the Internet, or computer networks (e.g., LANs or WANs). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components that are separate from each other (e.g., multiple chips).

[0115] In the example, memory 870 may store various data used by at least one component of electronic device 10 (e.g., external sensor unit 810, eye recognition unit 820, position recognition unit 830, or processor 861). For example, the various data may include input or output data of software (e.g., programs) and commands associated therewith. Memory 870 may include volatile memory 871 or non-volatile memory 873.

[0116] In the example, processor 861 may determine and / or obtain the sleep preparation start time of wearer 20. The sleep preparation start time indicates the time at which a sleep preparation operation begins, progressively reducing external stimuli to guide (e.g., induce) wearer 20 to sleep. External stimuli may be stimuli that may affect wearer 20 when electronic device 10 provides content services. External stimuli may be information input through one or more of the five (5) senses (e.g., visual, auditory, olfactory, gustatory, and tactile) that wearer 20 may perceive. For example, visual and / or auditory stimuli may be provided as stimuli that wearer 20 may receive from the operation of electronic device 10. Visual stimuli may be stimuli (e.g., glare) imposed on wearer 20's eyes by a screen output by display 851. Visual stimuli may be progressively reduced by adjusting at least one requirement of the screen display (e.g., brightness (illuminance), size of the display area, or color).

[0117] In the example, processor 861 can obtain the sleep preparation start time based on the wearer 20's target sleep time or target wake-up time. The target sleep time can be a sleep time set by wearer 20 on other electronic devices (e.g., smartphone 30 or smartwatch 40). The target sleep time can be set as a sleep alarm. The target wake-up time can be a wake-up time set by wearer 20 on other electronic devices (e.g., smartphone 30 or smartwatch 40). A wake-up alarm can be set. If wearer 20 sets a target sleep time for other electronic devices (e.g., smartphone 30 or smartwatch 40), processor 861 can obtain the sleep preparation start time based on either the target sleep time or the target wake-up time. The target sleep time can be a time period between the target sleep time and the target wake-up time. In the example, the electronic devices (e.g., smartphone 30 or smartwatch 40) and electronic device 10 can coordinate with each other to determine the target wake-up time.

[0118] Processor 861 can obtain the sleep preparation start time by at least one of the following: a set sleep target time, or a sleep target time predicted based on a wake-up target time and a sleep preparation interval. The sleep preparation interval can be the entire time during which the electronic device 10 applies visual effects in a step-by-step manner. For example, if the sleep target time is 8 p.m. and the sleep preparation interval is 20 minutes, processor 861 can determine or obtain 7:40 p.m. as the sleep preparation start time based on a sleep preparation interval of 20 minutes relative to 8 p.m., the sleep target time. For example, if the target sleep time is 8 hours, the wake-up target time is 6 a.m., and the sleep preparation interval is 20 minutes, processor 861 can determine or obtain 9:40 p.m. as the sleep preparation start time based on a sleep preparation interval of 20 minutes relative to 10 p.m. (i.e., 8 hours before 6 a.m.). If both a sleep target time and a wake-up target time are provided, processor 861 can obtain (or determine) the sleep preparation start time by prioritizing the sleep target time.

[0119] In the example, processor 861 can obtain (or determine) the sleep preparation start time based on sleep data recorded for wearer 20. For example, processor 861 can collect sleep data of wearer 20 from wearable device 40 worn by wearer 20. For example, if the predicted sleep time of wearer 20 is 10 p.m. based on sleep data analysis (or determination), processor 861 can determine or obtain 9:40 p.m. as the sleep preparation start time based on a sleep preparation interval of 20 minutes relative to 10 p.m. as the sleep time. For example, if the sleep time of wearer 20 is predicted to be 8 hours and the predicted wake-up time is 6 a.m. based on sleep data analysis, processor 861 can determine or obtain 9:40 p.m. as the sleep preparation start time based on a sleep preparation interval of 20 minutes relative to 10 p.m. (i.e., 8 hours before 6 a.m.).

[0120] In the example, processor 861 can determine that a sleep preparation start time has been reached (e.g., started) in response to a fatigue level measurement substantially meeting a predetermined threshold. For example, if the fatigue level obtained by the fatigue measurement reaches the predetermined threshold, processor 861 can determine that a sleep preparation start time has been reached (e.g., started). In the example, processor 861 can measure the fatigue level of wearer 20. Processor 861 can measure the fatigue level of wearer 20 periodically or non-periodically. To measure fatigue level periodically, processor 861 can preset a period for measuring fatigue level. To measure fatigue level non-periodically, processor 861 can monitor the occurrence of non-periodic events. A non-periodic event could be, for example, a sudden up-and-down movement of the wearer 20's head. This can be detected by at least one sensor through sensing signals.

[0121] In the example, to measure fatigue level, processor 861 can measure the duration of eye closure of wearer 20 (e.g., duration of eye closure), the number of blinks of wearer 20, the pupil diameter of wearer 20, the surface temperature of the eyeball of wearer 20, or the variability of heart rate of wearer 20. Processor 861 can measure at least one of the following: duration of eye closure, number of blinks, pupil diameter, or surface temperature of the eyeball of wearer 20, based on signals detected by at least one sensor. Processor 861 can measure the variability of heart rate of wearer 20 based on information provided by wearable device 40 worn by wearer 20. Processor 861 can obtain a fatigue level value indicating the degree of fatigue based on at least one of the following: duration of eye closure, number of blinks, pupil diameter, surface temperature of the eyeball, or variability of heart rate.

[0122] In the example, processor 861 can display a message on display 851 indicating whether to begin sleep preparation when the time for sleep preparation to begin is reached. For example, processor 861 can display a message such as "Time to prepare for sleep. Run sleep preparation mode?" or "High fatigue level. Run fatigue minimization mode?". Processor 861 can display a button on display 851 indicating whether to execute sleep preparation mode (e.g., a "Yes" button) and / or a button indicating whether to execute sleep preparation mode (e.g., a "No" button). Processor 861 can determine whether to execute sleep preparation mode based on interaction with wearer 20.

[0123] In the example, at the start time of sleep preparation, processor 861 can gradually reduce external stimuli that may affect the sleep quality or depth of wearer 20 within a predetermined time (e.g., sleep preparation time). The method of gradually reducing external stimuli can be determined, for example, based on the type of content service being used by wearer 20. The method of gradually reducing external stimuli can also be determined, for example, based on the display mode applied to display 851. Accordingly, processor 861 can provide wearer 20 with a sleep environment (e.g., a virtual environment) that allows easy switching to sleep. For example, processor 861 can output a sleep preparation screen through display 851, which applies visual effects that can gradually reduce visual stimuli over time. Processor 861 can apply a visual effect that switches the virtual image output across the entire display area of ​​display 851 from a VST screen to a sleep preparation screen. The immersive mode for outputting the virtual image across the entire display area of ​​display 851 can be, for example, a VR mode. In VR mode, electronic device 10 can output images through a 360° shielded display 851. The VST display can be output via the display 851 based on a switch from immersive mode to partial immersive mode. Depending on the partial immersive mode, the VST display can reduce the wearer's visual immersion 20, thereby reducing sympathetic nerve activation. The degree of visual immersion can be proportional to the size of the display output via the display 851. For example, the degree of immersion can increase with increasing screen size and decrease with decreasing screen size.

[0124] In the example, processor 861 can progressively reduce the proportion of the virtual image in the VST image over time. Processor 861 can progressively reduce the size of the virtual image in the VST image. Processor 861 can progressively increase the depth of the virtual image in the VST image. To increase the depth of the virtual image, processor 861 can move the virtual image away from the wearer 20 across the entire image representing 3D space.

[0125] In the example, processor 861 can output a VST screen with the following visual effect, instead of a real screen: a visual effect that displays a preset background screen at each point in time over time. For example, if the proportion of the virtual screen in the VST screen is reduced, the visual stimulation received by the wearer 20 may be reduced.

[0126] In the example, processor 861 can take into account the properties of objects and provide a visual effect that adjusts the blue light differently for each object included in the VST screen. Blue light can stimulate the brain, affect circadian rhythms, and interfere with (e.g., disrupt) sleep. Processor 861 can apply a visual effect to the VST screen that gradually increases the blue light adjusted differently for each object over time.

[0127] In the example, processor 861 can take into account the properties of objects, providing a visual effect that adjusts the illumination differently for each object included in the VST screen. Illumination can stimulate the brain, affect circadian rhythms, and interfere with falling asleep. Processor 861 can apply a visual effect to the VST screen that gradually increases the illumination, adjusted differently for each object, over time.

[0128] As described above, if the processor 861 adjusts the blue light and / or illuminance for each object on the VST screen in a progressive manner over time, the level of interference with the wearer 20's ability to fall asleep can be gradually reduced while maintaining visual realism.

[0129] In the example, if processor 861 performs a sleep preparation function that progressively reduces external stimuli over time during a sleep preparation interval, electronic device 861 can progressively reduce the volume of sound output caused by the content service. Processor 861 is a method for progressively reducing external stimuli over time and can progressively change the sound source output. For example, processor 861 can be operated to output sleep-inducing sounds (e.g., sleep aid sounds) proportionally to the passage of time.

[0130] Figure 9 This illustrates an electronic device according to an embodiment (e.g., Figure 1 The control flow diagram of an example of inducing sleep in the HMD device 10 is shown.

[0131] refer to Figure 9 In operation 910, electronic device 10 can determine and / or obtain the wearer (e.g., Figure 1 The wearer (20) begins their sleep preparation time.

[0132] Electronic device 10 can determine the sleep preparation start time based on at least one of the wearer 20's target sleep time or target wake-up time and sleep preparation interval. For example, if the target sleep time is 8 p.m. and the sleep preparation interval is 20 minutes, electronic device 10 can determine 7:40 p.m. as the sleep preparation start time based on a sleep preparation interval 20 minutes relative to the target sleep time of 8 p.m. For example, if the target sleep time is 8 hours, the target wake-up time is 6 a.m., and the sleep preparation interval is 20 minutes, electronic device 10 can determine 9:40 p.m. as the sleep preparation start time based on a sleep preparation interval 20 minutes relative to 10 p.m. (i.e., 8 hours before 6 a.m.).

[0133] Electronic device 10 can determine the sleep preparation start time based on sleep data recorded for wearer 20. For example, if the sleep data predicts (e.g., determines) that wearer 20's sleep onset time is 10 p.m., electronic device 10 can determine 9:40 p.m. as the sleep preparation start time based on a 20-minute sleep preparation interval relative to 10 p.m. (the predicted sleep onset time). For example, if the sleep data predicts that wearer 20's sleep duration is 8 hours and the predicted wake-up time is 6 a.m., electronic device 10 can determine 9:40 p.m. as the sleep preparation start time based on a 20-minute sleep preparation interval relative to 10 p.m. (i.e., 8 hours before 6 a.m.).

[0134] Electronic device 10 can proportionally (e.g., relatively) change the sleep preparation start time in response to an increase or decrease in fatigue level. In an example, electronic device 10 can periodically or non-periodically measure the wearer's fatigue level and obtain the amount of change in fatigue level based on the measurement. For example, electronic device 10 can measure the wearer's fatigue level by taking into account the wearer's duration of eye closure, blink rate, pupil diameter, ocular surface temperature, or heart rate variability. For example, if the fatigue level increases by a threshold when the adjustment unit is 10 minutes and the sleep preparation start time is 9:40 PM, electronic device 10 can change the sleep preparation start time to 9:30 PM. Similarly, if the fatigue level decreases by a threshold when the adjustment unit is 10 minutes and the sleep preparation start time is 9:40 PM, electronic device 10 can change the sleep preparation start time to 9:50 PM. The time unit for adjusting the start time of sleep preparation due to decreased fatigue (e.g., 20 minutes) can be different from the time unit for adjusting the start time of sleep preparation due to increased fatigue (e.g., 10 minutes).

[0135] In operation 920, the electronic device 10 can generate a visually appealing sleep preparation screen in a progressive manner over time after the sleep preparation start time is reached, until the sleep preparation time interval has elapsed.

[0136] More specifically, if the sleep preparation start time is reached, the electronic device 10 can display a sleep preparation guidance message on the display. For example, the electronic device 10 can output "Ready to sleep. Do you want to activate sleep preparation mode?" or "High fatigue level. Do you want to run fatigue minimization mode?" as a sleep preparation guidance message. The electronic device 10 can display a button (e.g., a "Yes" button) for confirming the execution of the sleep preparation mode, and / or a button (e.g., a "No" button) for declining the execution of the sleep preparation mode. The electronic device 10 can determine whether to execute the sleep preparation mode based on the interaction with the wearer 20.

[0137] If a sleep preparation mode is activated, the electronic device 10 can gradually reduce external stimuli, such as visual and / or auditory stimuli, over a predetermined period of time (e.g., a sleep preparation interval). The method of gradually reducing external stimuli can be determined based on the type of content service. The method of gradually reducing external stimuli can be determined based on the display mode applied to the display. For example, the electronic device 10 can gradually generate a sleep preparation screen with visual effects to reduce visual stimulation over time. The electronic device 10 can generate, for example, a sleep preparation screen as a VST screen. The VST screen can be a virtual screen comprising a real screen as a background. To gradually apply visual effects to the VST screen, the electronic device 10 can gradually reduce the proportion of the virtual screen in the VST screen over time. The visual stimuli affecting the wearer 20 can be reduced accordingly as the proportion of the virtual screen in the VST screen decreases.

[0138] Electronic device 10 can take into account the attributes of objects and apply a visual effect that adjusts the blue light of each object included in the VST screen in a progressive manner over time. For example, the amount of blue light adjustment for each object can be the same or different. Electronic device 10 can apply a visual effect to the VST screen that increases the blue light adjusted uniformly or differently for each object in a progressive manner over time. The attributes of the object can be one of the following: including whether the corresponding object is a real image (e.g., a non-virtual image) or a virtual image, whether the corresponding object emits light, or the type of light source of the corresponding object.

[0139] Electronic device 10 can take into account the attributes of objects and apply a visual effect that adjusts the illuminance of each object included in the VST screen in a progressive manner over time. In this case, the amount of illuminance adjustment for each object can be the same or different. Electronic device 10 can apply a visual effect to the VST screen that increases the illuminance of each object uniformly or differently over time in a progressive manner. The attributes of the object can be one of the following: whether the corresponding object is a real or virtual image, whether the corresponding object is emitting light, or the type of light source of the corresponding object.

[0140] In operation 930, electronic device 10 can display a sleep preparation screen on a monitor. The visual effects applied to the sleep preparation screen in a gradual manner can change over time.

[0141] Electronic device 10 can repeat its operation periodically or non-periodically (920 and 930) until the sleep preparation time has passed, thereby sequentially reducing the degree of sleep obstruction for wearer 20. During the sleep preparation interval, electronic device 10 can gradually reduce auditory stimulation caused by audio output from the content service over time. During the sleep preparation interval, electronic device 10 can output sounds that aid sleep (e.g., sleep-inducing sounds) instead of audio, based on the content service.

[0142] Figure 10 This illustrates an electronic device according to an embodiment (e.g., Figure 1 The control flow diagram of an example process for determining the start time of sleep preparation in the HMD device 10.

[0143] refer to Figure 10 In operation 1011, electronic device 10 can connect to external device (e.g., wireless short-range communication) via a communication link based on a predetermined communication scheme (e.g., wireless short-range communication scheme). Figure 1 (Smartphone 30 in the device). Electronic device 10 can perform authentication processes (e.g., account identification) to protect personal information associated with external device 30.

[0144] If connection and / or authentication with external device 30 is completed, electronic device 10 can determine in operation 1013 whether a sleep target time has been set. The sleep target time can be set by user 20 in external device 30 for falling asleep.

[0145] If a target sleep time is set, the electronic device 10 can determine the sleep preparation start time in step 1015 based on the target sleep time. For example, the sleep preparation start time can be determined taking into account the sleep preparation interval for the target sleep time. The sleep preparation interval can be preset. For example, if the target sleep time is 9 p.m. and the sleep preparation interval is 30 minutes, the electronic device 10 can set the sleep preparation start time to 8:30 p.m.

[0146] If connection and / or authentication with external device 30 is completed, in operation 1017, electronic device 10 can determine whether a wake-up target time has been set (e.g., whether a wake-up alarm has been set). The wake-up target time can be set by user 20 in external device 30 for waking up.

[0147] If a target wake-up time is set, the electronic device 10 can determine the sleep preparation start time based on the target wake-up time in operation 1019. In this case, the electronic device 10 can additionally consider the user 20's sleep time. The user's sleep time can be set by the user 20 in the external device 30. The user's sleep time can be determined by analyzing the average sleep time obtained from the sleep data provided by the external device 30. As an example, the sleep preparation start time can be determined by considering the target wake-up time, the predicted sleep time, and the sleep preparation interval. The sleep preparation interval can be preset. For example, if the target wake-up time is 6:00 AM, the predicted sleep time is 8 hours, and the sleep preparation interval is 30 minutes, the electronic device 10 can determine the sleep preparation start time as 9:30 PM.

[0148] Figure 11 This illustrates an electronic device according to an embodiment (e.g., Figure 1 The control flow diagram of an example process for determining the start time of sleep preparation in the HMD device 10.

[0149] refer to Figure 11 In operation 1101, electronic device 10 can connect to external device (e.g., wireless short-range communication) via a communication link based on a predetermined communication scheme (e.g., wireless short-range communication scheme). Figure 1 (Smartphone 30 in the device). Electronic device 10 can perform authentication processes (e.g., account identification) to protect personal information associated with external device 30.

[0150] If connection and / or authentication with external device 30 is completed, electronic device 10 can receive sleep data from external device 30 for a predetermined time period (e.g., 3 months) in operation 1103. Electronic device 10 can analyze the sleep data. For example, electronic device 10 can analyze the average sleep time of user 20 each day of the week based on the sleep data. For example, electronic device 10 can analyze the time user 20 falls asleep each day of the week based on the sleep data.

[0151] In operation 1105, electronic device 10 can obtain a predicted target time for falling asleep based on the average daily sleep time of user 20 over a week or the analyzed daily sleep onset time of user 20 over a week. The predicted target time for falling asleep can be the predicted sleep onset time that user 20 expects.

[0152] In operation 1107, electronic device 10 can monitor changes in the fatigue level of user 20. Electronic device 10 can take into account changes in fatigue levels from external devices (e.g., Figure 1The electronic device 10 uses biometric information provided by a smartphone 30 or wearable device 40 to measure changes in the user 20's fatigue level. The electronic device 10 can measure the user 20's fatigue level periodically or non-periodically and evaluate a weighted value based on the amount of change. For periodic fatigue measurement, the electronic device 10 can preset a period (e.g., 30 minutes) for measuring fatigue. For non-periodic fatigue measurement, the electronic device 10 can monitor the occurrence of non-periodic events. A non-periodic event could be, for example, a sudden up-and-down movement of the user 20's head. This can be detected by at least one sensor through sensing signals. As an example, the electronic device 10 can evaluate a real-time weighted fatigue level based on detected changes in fatigue.

[0153] In operation 1109, electronic device 10 can determine whether the amount of change in fatigue level is equal to or greater than a threshold. For example, electronic device 10 can determine whether the blinking frequency of user 20 has increased by at least a predetermined number of times. For example, electronic device 10 can determine whether the duration of eye closure of user 20 has increased by a predetermined time. For example, electronic device 10 can determine whether the pupil diameter of user 20 has increased by a predetermined size. For example, electronic device 10 can determine whether the ocular surface temperature of user 20 has increased by at least a predetermined value. For example, electronic device 10 can determine whether the heart rate variability (HRV) of user 20 has increased by at least a preset level. If at least one or more indicators used to measure fatigue level (e.g., blinking frequency, duration of eye closure, or pupil diameter) show a change that can predict an abnormal level, electronic device 10 can determine that fatigue level has increased.

[0154] If it is determined that the increase in fatigue level exceeds (or is greater than) a threshold level, the electronic device 10 can adjust the predicted target sleep time in operation 1111 based on a real-time weighted value (or degree) corresponding to the increase in fatigue level. As an example, the electronic device 10 can change the target sleep time proportionally to the increase or decrease in fatigue level. For example, if the adjustment unit is 10 minutes, the electronic device 10 can decrease the target sleep time every 10 minutes as the fatigue level increases by the threshold. For example, if the target sleep time is 9:40 PM and the fatigue level increases by the threshold, the electronic device 10 can change the target sleep time to 9:30 PM. For example, if the target sleep time is 9:40 PM and the fatigue level decreases by the threshold or more, the electronic device 10 can change the target sleep time to 9:50 PM.

[0155] In operation 1113, the electronic device 10 can determine the start time for sleep preparation based on a predetermined target sleep time. The electronic device 10 can then begin sleep preparation at the designated start time.

[0156] Figure 12aand Figure 12b This illustrates an electronic device according to an embodiment (e.g., Figure 1 The control flow diagram of the example process of preparing to fall asleep in the HMD device 10.

[0157] refer to Figure 12a or Figure 12b In operation 1211, electronic device 10 can determine whether the sleep preparation start time has been reached. If the time has been reached, in operation 1213, electronic device 10 can activate the mode to be performed according to sleep preparation step 1. For example, electronic device 10 can display a sleep preparation guidance message on the display. Electronic device 10 can output, for example, "Ready to sleep. Would you like to run the sleep preparation mode?" as a sleep preparation guidance message on the display. Electronic device 10 can display, along with the sleep preparation guidance message, a button for confirming execution (e.g., a "Yes" button) and / or a button for rejecting execution (e.g., a "No" button). Electronic device 10 can determine whether to execute the sleep preparation mode based on the interaction with user 20, in response to the selection of one (1) button from two (2) buttons.

[0158] If sleep preparation step 1 is activated, in operation 1215, electronic device 10 can determine whether a full-screen display mode (e.g., VR mode) is applied, in which a virtual image is displayed across the entire area of ​​the display according to the content service. If it is determined that a full-screen display mode is not applied, electronic device 10 can determine that a partial display mode (e.g., VST mode) is applied and proceed to operation 1225 to switch steps.

[0159] If it is determined that a full-screen display mode is applied, the electronic device 10 can determine in operation 1217 whether a partial display mode (e.g., VST mode) is applicable. For example, the electronic device 10 can determine whether a switchable partial display mode (e.g., VST mode) option exists in the content service providing the virtual screen through the full-screen display mode.

[0160] If a partial display mode is applicable, the electronic device 10 can output a screen according to the partial display mode (e.g., a VST screen) via a display during operation 1223. The VST screen according to the partial display mode can be a composite screen (e.g., Figure 4 The composite image 441 in the image is created by using a virtual image (e.g., Figure 4 The virtual scene 431 in the image and the real scene as the background (e.g., Figure 4 It was obtained by synthesizing real footage (411).

[0161] If a partial display mode is unavailable, electronic device 10 can switch to virtual display mode in operation 1219. If switched to virtual display mode, electronic device 10 can display a partial virtual image within the virtual display area in operation 1221. For example, electronic device 10 can display the image based on the user 20's head position information (e.g., ...). Figure 4 The head position information 421 in the image determines the field of view of user 20. Electronic device 10 can obtain a portion of the virtual image based on the field of view of the entire virtual image. Electronic device 10 can display a portion of the virtual image within the virtual display area.

[0162] In operation 1225, electronic device 10 can determine whether the step switching time has been reached. The step switching time can be reached after a predetermined time (e.g., N minutes, where N is a positive integer) has elapsed after sleep preparation step 1 is activated. Electronic device 10 can perform operations according to sleep preparation step 1 until the step switching time is reached.

[0163] If the time reaches the step switching time, then in operation 1227, electronic device 10 can activate the mode to perform the sleep preparation operation according to sleep preparation step 2. For example, electronic device 10 can adjust the layout of the virtual image in the VST screen according to a partial display mode in sleep preparation step 2. Layout adjustment may include adjusting the distance between the virtual image in the VST screen and user 20 (see [link to relevant documentation]). Figure 14a Layout adjustments may include resizing the virtual image within the VST screen (see [link]). Figure 14b ).

[0164] More specifically, if sleep preparation step 2 is activated, in operation 1229, electronic device 10 can determine whether a partial display mode (e.g., VST mode) is applied. If a partial display mode is not applied, electronic device 10 can determine that the corresponding content service does not have an option that supports partial display modes (e.g., VST mode) and can proceed to operation 1239 to switch steps.

[0165] If a partial display mode is applied, the electronic device 10 can determine in operation 1231 whether the layout of the partial display screen (VST screen) can be changed according to the partial display mode. For example, if it is determined that the distance between the user 20 and the virtual screen is within a threshold distance, the electronic device 10 can determine that the layout can be changed.

[0166] If the layout of the virtual screen can be changed, electronic device 10 can determine in operation 1233 whether the virtual screen includes text. In the example, electronic device 10 can analyze the attributes of the content providing the virtual screen. If it is determined that the analyzed content attributes do not include text, then in operation 1235, electronic device 10 can change the layout by applying a display effect that reduces the size of the virtual screen or increases the distance from the user 20 (see [link to relevant documentation]). Figure 14a or Figure 14b If the content attribute being analyzed includes text, then in operation 1237, electronic device 10 can increase the size of the text included in the virtual screen and change the layout by applying a display effect that reduces the size of the virtual screen or increases the distance from the user 20.

[0167] In operation 1239, electronic device 10 can determine whether the step switching time has been reached. The step switching time can be reached after a predetermined time (e.g., N minutes, where N is a positive integer) has elapsed after sleep preparation step 2 is activated. Electronic device 10 can perform operations according to sleep preparation step 2 until the step switching time is reached.

[0168] If the step switching time is reached, the electronic device 10 can perform a sleep preparation operation according to sleep preparation step 3. For example, in operation 1241, the electronic device 10 can detect objects included in the screen and analyze the attributes of each detected object. The electronic device 10 can then perform the analysis based on the results (e.g., see...). Figure 15 The electronic device 10 can apply visual effects differently to virtual objects (e.g., objects detected on a virtual screen (rendered image)) or real objects (e.g., objects detected on a real screen (captured image)). For example, in operation 1243, the electronic device 10 can adjust the visual effects on the virtual object (e.g., blue light and / or illuminance) (see, for example, see...). Figure 16d For example, in operation 1245, electronic device 10 can adjust the visual effects (e.g., blue light and / or illuminance) differently for each real object (see...). Figure 16b For example, electronic device 10 can divide a real object into luminous objects and adjust the visual effects (e.g., blue light and / or illuminance) of the shaded area differently for each type of luminous object (e.g., natural light source, artificial light source, or real display). The shaded area can be a region used to apply visual effects (e.g., adjust blue light or illuminance) to each luminous object. Electronic device 10 can apply different visual effects (e.g., blue light and / or illuminance) to the shaded areas of the real object other than the luminous objects.

[0169] Figures 13a to 13c This illustrates an electronic device according to one or more embodiments (e.g., Figure 1The image shows an example of a visual effect that can be applied to each sleep preparation step in the HMD device 10.

[0170] refer to Figure 13a If sleep preparation step 1 is activated (e.g., sleep preparation start time), electronic device 10 can display a full-screen display 131a on the monitor. This full-screen display includes an operation guide 1313a “Ready to sleep. Do you want to run sleep preparation mode?” and buttons for indicating execution (e.g., “Yes” button) 1315a and / or buttons for indicating non-execution (e.g., “No” button) 1317a, which overlap with a virtual screen 1311a. Electronic device 10 can select one of the “Yes” button 1315a or the “No” button 1317a through interaction (1319a) with user 20. In the example, the “Yes” or “No” button can be activated by user operation of an external device connected to electronic device 10, by user providing a verbal command, or by user mimicking the action of selecting the “Yes” or “No” button.

[0171] If user 20 selects the "Yes" button 1315a, electronic device 10 can switch from a full-screen display 131a (e.g., a VR screen) to a partial display 131b (e.g., a VST screen). The partial display 131b can be a screen obtained by compositing (e.g., merging or combining) a virtual screen 1311b based on a content service with a real screen 1312b based on a captured image. The real screen 1312b or the virtual screen 1311b can include a guidance message 1314b: "Execute sleep preparation mode in a step-by-step manner." The guidance message 1314b can be displayed on both the real screen 1312b and the virtual screen 1311b.

[0172] refer to Figure 13bIf sleep preparation step 2 is activated (e.g., when a predetermined time has elapsed since the sleep preparation start time), the electronic device 10 can switch the first partial display screen 132a to the second partial display screen 132b. The second partial display screen 132b can be a screen of the first partial display screen 132a with predetermined visual effects applied. For example, the first partial display screen 132a can be a screen obtained by compositing a first virtual screen 1321a according to a content service with a first real screen 1322a based on a captured image. For example, the second partial display screen 132b can be a screen obtained by compositing a second virtual screen 1321b according to a content service with a second real screen 1322b based on a captured image. For example, a visual effect b used to reduce the size of the virtual screen 1321a in the first partial display screen 132a can be applied to the second partial display screen 132b. Visual effects used to increase the distance to users 1327a and 1327b compared to the first virtual screen 1321a included in the first partial display screen 132a can be applied to the second virtual screen 1321b included in the second partial display screen 132b. However, the size of the text included in the first virtual screen 1321a (e.g., NOWHERE 2050) can remain unchanged on the second virtual screen 1321b.

[0173] refer to Figure 13c If sleep preparation step 3 is activated (e.g., after a predetermined time has elapsed since sleep preparation step 2 was activated), electronic device 10 can switch the third part display screen 133a to the fourth part display screen 133b. The fourth part display screen 133b can be a screen to which predetermined visual effects are applied to the third part display screen 133a. For example, the third part display screen 133a can be a screen obtained by compositing a third virtual screen 1331a based on a content service with a third real screen 1332a based on a captured image. For example, the fourth part display screen 133b can be a screen obtained by compositing a fourth virtual screen 1331b based on a content service with a fourth real screen 1332b based on a captured image. For example, visual effects (e.g., blue light and / or illuminance) can be applied differently to the fourth part display screen 133b for each occlusion area T1, T2, T3, and T4. The occlusion regions T1, T2, T3, and T4 can be determined for each luminous object (e.g., bracket, window, door, and TV) among the real objects included in the third real image 1332a within the third partial display image 133a. The fourth partial display image 133b may include, for example, a fourth virtual image 1331b, in which visual effects (e.g., blue light and / or illuminance) are applied to the third partial display image 133a within the third virtual image 1331a.

[0174] Figure 14a It shows the implementation for changing Figure 13b An example of the visual effect of image depth (a) in the image. Figure 14b It shows the implementation for changing Figure 13b The image shows an example of the visual effect of screen size (b).

[0175] refer to Figure 14a After applying visual effect a, the separation distance between the virtual image 1411b composited onto the real space 1412b on the partial display screen 141b and the user (1417a and 1417b) may be relatively larger than the separation distance between the virtual image 1411a composited onto the real space 1412a on the partial display screen 141a and the user before applying visual effect a. In this case, the immersion of the image may be reduced after applying visual effect a to the user (1417a and 1417b).

[0176] refer to Figure 14b After applying visual effect b, the size of the virtual image 1411b composited onto the real space 1412b on the partial display screen 141b may be smaller than the size of the virtual image 1411a composited onto the real space 1412a on the partial display screen 141a before applying visual effect a (b1>b2). In this case, since visual effect b is applied gradually, the user's immersion in the image may be reduced after applying visual effect b.

[0177] Figure 15 This illustrates an electronic device according to an embodiment (e.g., Figure 1 A diagram showing examples of the visual effects applied to each object in the HMD device 10.

[0178] refer to Figure 15In sleep preparation step 3, different visual effects (e.g., blue light and / or illuminance) can be applied to the occlusion areas (T1, T2, T3, T4, T5, and T6) of the target objects selected based on their attributes. For example, the first occlusion area T1 is a support (artificial light source) that is one of the luminous objects included in the real image 1502; the second occlusion area T2 is a window (e.g., natural light) that is one of the luminous objects included in the real image 1502; the third occlusion area T3 is an entrance door (other indoor space light source) that is one of the luminous objects included in the real image 1502; and the fourth occlusion area T4 is a TV (real display light source) that is one of the luminous objects included in the real image 1502. The fifth occlusion area T5 is the entire virtual area 1501; the sixth occlusion area T6 is the entire real image 1502; and the seventh occlusion area T7 is the area on the real image 1502 excluding the light source.

[0179] For example, illuminance and / or blue light can be applied differently to the first through fourth occlusion areas (T1, T2, T3, and T4) used to select luminous objects on the real screen 1502. For example, blue light and / or illuminance can be selectively applied in a progressive manner to the fifth occlusion area T5 (selecting the entire area of ​​the virtual screen 1501) and / or the sixth occlusion area T6 (selecting the entire area of ​​the real screen 1502). For example, blue light and illuminance can be applied in a progressive manner to the seventh occlusion area T7 (selecting the area on the real screen 1502 excluding the light source). As a result of occlusion, illuminance and / or blue light can be gradually reduced, thereby preventing activation of the user's sympathetic nervous system.

[0180] Figures 16a to 16f This illustrates an electronic device according to an embodiment (e.g., Figure 1 A diagram showing an example of applying visual effects in an HMD device 10.

[0181] Figure 16a Part of the display screen 160a can be a VST screen without applied visual effects (e.g., Blu-ray); and Figure 16b Part of the display screen 160b can be a VST screen, wherein visual effects (e.g., blue light) are applied to the occluded areas (T8, T9, and T10) of the luminous objects included in the selected real screen 1602b, and visual effects (e.g., blue light) are not applied to the virtual screen 1601b.

[0182] Figure 16c The partial display screen 160a can be a VST screen, in which visual effects (such as blue light) are applied to the occluded area T11 of the selected real screen 1602c, and visual effects (such as blue light) are not applied to the virtual screen 1601c.

[0183] Figure 16d The portion of the display screen 160d can be a VST screen, where visual effects (such as blue light) are applied to the occluded area T12 of the entire virtual screen 1601c, and visual effects (such as blue light) are not applied to the entire real screen 1602d.

[0184] Figure 16e The portion of the display screen 160e can be a VST screen, which is obtained by applying visual effects (such as blue light) differently to the occlusion areas (T8, T9 and T10) of the luminous objects included in the selected real screen 1602e and the occlusion area T13 of the entire virtual screen 1601e.

[0185] Figure 16f The portion of the display screen 160f can be a VST screen, where visual effects (such as Blu-ray) are applied to the occlusion area T11 of the entire real screen 1602f and the occlusion area T13 of the entire virtual screen 1601f.

[0186] Figure 17 This illustrates an electronic device according to an embodiment (e.g., Figure 1 The control flow diagram of an example process for inducing sleep in the HMD device 10 is shown.

[0187] refer to Figure 17 In operation 1711, electronic device 10 can output content images via a display. The content image can be a virtual image based on a content service. The content image can be a full-screen display, outputting the virtual image to the entire area of ​​the display.

[0188] In operation 1713, electronic device 10 can determine whether the content service that outputs the virtual image corresponds to content that requires managing the fatigue level of user 20. The fatigue level of user 20 can be, for example, eye fatigue. Content that requires fatigue management can include, for example, game content or movie content that provides high visual stimulation (e.g., flashing lights) to user 20.

[0189] If the content requires fatigue level management, the electronic device 10 can determine in operation 1715 whether a set time has elapsed. The set time can determine the period of time used to measure fatigue level. The set time can be, for example, 30 minutes. In this case, the electronic device 10 can obtain the change in fatigue level by measuring fatigue level every 30 minutes.

[0190] After the set time has elapsed, electronic device 10 can measure the fatigue level of user 20 in operation 1717. Electronic device 10 can adjust the fatigue level based on the measured fatigue level. Electronic device 10 can take into account external devices (e.g., Figure 1 The electronic device 10 uses biometric information provided by a smartphone 30 or wearable device 40 to measure changes in the user 20's fatigue level. The electronic device 10 can assess a weighted fatigue level in real time based on the amount of change in fatigue level. The electronic device 10 can measure fatigue level both periodically and non-periodically. For example, when abnormal behavior is detected (such as the user 20 suddenly looking up and down), the electronic device 10 can measure the user 20's fatigue level.

[0191] In operation 1719, electronic device 10 can determine whether the change in fatigue level is greater than or equal to a threshold. For example, electronic device 10 can determine whether the blinking frequency of user 20 has increased by a predetermined number of times or more. For example, electronic device 10 can determine whether the duration of eye closure of user 20 has increased by a predetermined time. For example, electronic device 10 can determine whether the pupil diameter of user 20 has increased by a predetermined size. For example, electronic device 10 can determine whether the ocular surface temperature of user 20 has increased by a predetermined value or more. For example, electronic device 10 can determine whether the heart rate variability (HRV) of user 20 has increased by at least a predetermined level. When at least one or more indicators used to measure fatigue level (e.g., blinking frequency, duration of eye closure, or pupil diameter) show a change that can predict an abnormal level, electronic device 10 can determine that fatigue level has increased.

[0192] If the change in fatigue level is determined to be equal to or greater than a threshold level, electronic device 10 can notify user 20 in operation 1721, suggesting switching to fatigue minimization mode, in which fatigue reduction operations are to be performed. Electronic device 10 can use visual and / or auditory information to notify user 20. For example, electronic device 10 can output a guiding message on the display: "Fatigue level is high. Would you like to run fatigue minimization mode?"

[0193] If user 20 responds to the guidance message 1813a requesting to switch to the fatigue minimization mode, electronic device 10 can switch to the fatigue minimization mode to perform the fatigue reduction operation in operation 1723. For example, electronic device 10 can output a button indicating to switch to the fatigue minimization mode via a display (e.g., Figure 18a The "Yes" button (1815a) and / or a button indicating not to switch to the fatigue minimization mode (e.g., Figure 18a (No button 1817a). If the "Yes" button 1815a is selected by interacting with the user 20, the electronic device 10 can determine that the user 20 has requested to switch to the fatigue minimization mode, and switch to the fatigue minimization mode to perform the operation of reducing fatigue.

[0194] According to an embodiment, if switched to a fatigue minimization mode, the electronic device 10 can activate sleep preparation step 1 (e.g., sleep preparation start time). If sleep preparation step 1 is activated, the electronic device 10 can switch from full-screen display mode (VR mode) to partial display mode (VST mode) (e.g., see...). Figure 18a Full-screen display mode (VR mode) is a display mode that displays a full-screen virtual image (VR image) based on a content service across the entire area of ​​the display. Partial display mode (VST mode) is a display mode that displays a composite image (VST image) obtained by combining virtual and real images based on a content service across the entire area of ​​the display.

[0195] According to an embodiment, when the time reaches the step switching time, the electronic device 10 can activate the sleep preparation step 2. If the sleep preparation step 2 is activated, the electronic device 10 can analyze the attributes of the objects included in the screen. The electronic device 10 can then analyze the attributes of the objects (e.g., see...) Figure 18b The electronic device 10 can apply different visual effects to virtual objects (e.g., objects detected on a virtual screen (rendered image)) or real objects (e.g., objects detected on a real screen (captured image)). The electronic device 10 can adjust the visual effects (e.g., blue light and / or illuminance) for virtual objects, or it can adjust the visual effects (e.g., blue light and / or illuminance) for real objects differently. For example, the electronic device 10 can divide a real object into luminous objects and adjust the visual effects (e.g., blue light and / or illuminance) of the occlusion area (T14, T15, T16, or T17) differently for each type of luminous object (e.g., natural light source, artificial light source, or real display). The occlusion area (T14, T15, T16, or T17) can be a region set for applying visual effects (e.g., adjusting blue light or illuminance) to each luminous object. The electronic device 10 can apply the occlusion to the occlusion area of ​​the real objects other than the luminous objects, or to the real screen (e.g., Figure 18b The entire area of ​​the real scene in 1822 (e.g., Figure 18b Visual effects (e.g., blue light and / or illuminance) can be applied to the shaded area T18 in the image. Electronic device 10 can also apply visual effects (e.g., blue light and / or illuminance) to virtual images. Figure 18b The entire area of ​​the virtual screen (1821) in the image (e.g., Figure 18b Visual effects (e.g., blue light and / or illuminance) are applied to the shaded area T19.

[0196] Figure 18a This illustrates an electronic device according to an embodiment (e.g., Figure 1 An example image of applying visual effects in step 1 of the sleep preparation process in the HMD device 10.

[0197] refer to Figure 18a If sleep preparation step 1 is activated (e.g., sleep preparation start time), electronic device 10 can display a full-screen display 181a on the monitor. This full-screen display includes operation guidance 1813a “High fatigue level. Do you want to run the fatigue minimization mode?” and buttons for indicating execution (e.g., “Yes” button) 1815a and / or buttons for indicating non-execution (e.g., “No” button) 1817a, which overlap with the virtual screen 1811a. Electronic device 10 can select one of the “Yes” button 1815a or the “No” button 1817a through interaction with user 20 (1819a).

[0198] If user 20 selects the "Yes" button 1815a, electronic device 10 can switch from a full-screen display 181a (e.g., a VR screen) to a partial display 181b (e.g., a VST screen). The partial display 181b can be a screen obtained by compositing (e.g., merging or combining) a virtual screen 1811b based on a content service with a real screen 1812b based on a captured image. The real screen 1812b or the virtual screen 1811b can include a guidance message 1814b: "Execute fatigue minimization mode in a step-by-step manner." The guidance message 1814b can be displayed on both the real screen 1812b and the virtual screen 1811b.

[0199] Figure 18b This illustrates an electronic device according to an embodiment (e.g., Figure 1 A diagram showing examples of the visual effects applied to each object in the HMD device 10.

[0200] refer to Figure 18b In sleep preparation step 3, visual effects (e.g., blue light and / or illuminance) can be applied differently to the occlusion areas (T14, T15, T16, T17, T18, and T19) of the target objects selected based on their attributes. For example, the first occlusion area T14 selects a television set (real display light source) as one of the luminous objects included in the real image 1822; the second occlusion area T15 selects the entrance door (other indoor space light source) as one of the luminous objects included in the real image 1822; and the third occlusion area T16 and the fourth occlusion area T17 select two (2) windows arranged side by side (natural light) as one of the luminous objects included in the real image 1822. The fifth occlusion area T18 selects the entire real image 1822; and the sixth occlusion area T19 selects the entire virtual area 1821.

[0201] For example, illuminance and / or blue light can be applied differently to the first to fourth occlusion areas (T14, T15, T16, and T17) used to select luminous objects on the real screen 1822. For example, blue light and / or illuminance can be selectively applied in a stepwise manner to the fifth occlusion area T5 (selecting the entire area of ​​the real screen 1822) and / or the sixth occlusion area T6 (selecting the entire area of ​​the virtual screen 1821).

[0202] The terminology used herein is provided only to describe some embodiments thereof and is not intended to limit this disclosure. The singular forms “a” and “the” used herein are also intended to include the plural forms unless the context explicitly indicates otherwise. As used herein, each of the phrases “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B or C” can include all possible combinations of the items listed in the corresponding phrase. As used herein, the term “and / or” should be understood to cover any and all possible combinations of one or more of the listed items. As used herein, the terms “comprising,” “having,” and “including” are used only to indicate the presence of a feature, component, part, or combination thereof described herein, but the use of such terms does not preclude the possibility of the presence or addition of one or more other features, components, parts, or combinations thereof. As used herein, the terms “first” and “second” can modify various components regardless of importance and / or order, and are used to distinguish one component from another without limiting the component.

[0203] As used herein, depending on the context, the term "configured as" may be used interchangeably with the terms "suitable for," "capable of," "designed to," "suitable for," "enable to," or "able to." The term "configured as" does not inherently mean "specifically designed in hardware." Rather, the term "configured as" can mean that a device is capable of performing operations with another device or component. For example, "a device configured (or set) to perform A, B, and C" can be a dedicated device performing the corresponding operations or can represent a general-purpose device capable of performing various operations including the corresponding operations.

[0204] Meanwhile, the terms “upper side”, “lower side” and “front and rear direction” used in this disclosure are defined relative to the drawings, and the shape and position of each component are not limited by these terms.

[0205] In this disclosure, the above description is primarily directed to specific embodiments, but this disclosure is not limited to these specific embodiments, but should be understood to cover all various modifications, equivalents and / or substitutions of various embodiments.

Claims

1. A method performed by a head-mounted display, HMD, device (10), the method comprising: determining a sleep preparation start time; and displaying, on a display, a sleep preparation screen, wherein a visual effect is applied to the sleep preparation screen in a stepwise manner during a sleep preparation time interval from the sleep preparation start time, wherein the visual effect applied in the stepwise manner comprises a visual effect of switching a virtual screen output through an entire display area of the display to a video see-through, VST, screen, wherein, in the entire display area of the display, the VST screen displays the virtual screen with a non-virtual screen as a background, wherein the non-virtual screen is based on an image captured by a front-facing camera, and wherein the virtual screen is based on content performed by the HMD device. Determining the sleep preparation start time comprises:

2. The method of claim 1, wherein, measuring a fatigue level of a user; and determining the sleep preparation start time based on determining that the fatigue level meets a predetermined threshold level. Measuring the fatigue level comprises:

3. The method of claim 2, wherein, performing a first measurement operation that measures a duration of time that the user has eyes closed; performing a second measurement operation that measures a number of blinks of the user; performing a third measurement operation that measures a pupil diameter of the user; performing a fourth measurement operation that measures a surface temperature of an eye of the user; performing a fifth measurement operation that measures a heart rate variability of the user; and obtaining a fatigue level value indicative of the fatigue level based on at least one of the duration of time that the user has eyes closed, the number of blinks, the pupil diameter, the surface temperature of the eye, or the heart rate variability. Determining the sleep preparation start time further comprises:

4. The method of any one of claims 1 to 3, wherein, obtaining the sleep preparation start time based on at least one of a sleep target time or a wake-up target time of the user and the sleep preparation time interval; or obtaining the sleep preparation start time based on sleep data of the user, and wherein the visual effect is applied in the stepwise manner during the sleep preparation time interval. The visual effect applied in the stepwise manner comprises at least one of: a visual effect that reduces a proportion of the virtual screen in the VST screen in a stepwise manner during the sleep preparation time interval; or a visual effect that displays a preset background screen instead of the non-virtual screen as time progresses during the sleep preparation time interval.

5. The method of any one of claims 1 to 4, wherein, The visual effect applied in the stepwise manner comprises:

6. The method of any one of claims 1 to 4, wherein, a visual effect that displays the VST screen in which a blue light is adjusted differently for each object in the VST screen based on a property of one or more objects included in the VST screen; or a visual effect that displays the VST screen in which an illuminance is adjusted differently for each object in the VST screen based on a property of one or more objects included in the VST screen, and ​ The attributes of the one or more objects included in the VST screen include at least one of the following: a first indicator indicating one of the non-virtual screen or the virtual screen including the corresponding object; a second indicator indicating whether the corresponding object is emitting light; or a third indicator indicating the light source type of the corresponding object.

7. The method of any one of claims 1 to 4, wherein, Applying the visual effects in a stepwise manner during the sleep preparation time interval includes: At the designated sleep preparation start time, a message instructing whether to begin sleep preparation is displayed on the monitor; and Based on the interaction with the user, determine whether to apply the visual effect in a step-by-step manner.

8. A head-mounted display (HMD) device (10), comprising: At least one sensor (310, 320, 330); At least one camera; Display (851); as well as At least one processor (861), including processing circuitry, is operatively coupled to the at least one sensor (310, 320, 330), the at least one camera, and the display (851). The at least one processor (861) is configured to: Determine when to start preparing for sleep. A sleep preparation screen is displayed on the display (851), wherein, starting from the start time of the sleep preparation, visual effects are applied to the sleep preparation screen in a progressive manner during the sleep preparation time interval. The visual effects applied in a step-by-step manner include the visual effect of switching the virtual image output through the entire display area of ​​the display to a video perspective (VST) image. In the entire display area of ​​the monitor, the VST screen displays the virtual image against a non-virtual image background. The non-virtual image is based on an image captured by the front-facing camera, and The virtual screen is based on content executed by the HMD device.

9. The HMD device of claim 8, wherein, The at least one processor (861) is configured to: The user's fatigue level is measured based on information collected from at least one of the at least one sensor (310, 320, 330) or at least one of the at least one camera, and The start time for sleep preparation is determined based on whether the level of fatigue meets a predetermined threshold.

10. The HMD device of claim 9, wherein, The at least one processor (861) is configured to: Measure the duration of the user's closed eyes. Measure the number of blinks of the user. Measure the user's pupil diameter. Measure the surface temperature of the user's eyeball, or Measure the user's heart rate variability, and A fatigue level value indicating the degree of fatigue is obtained based on at least one of the following: the duration of the user's closed eyes, the number of blinks, the pupil diameter, the surface temperature of the eyeball, or the heart rate variability.

11. The HMD device of any one of claims 8-10, wherein, The at least one processor (861) is configured to: Based on the user's target time for falling asleep or waking up, the sleep preparation start time is obtained, or The sleep preparation start time is determined based on at least one of the target sleep time or the target wake-up time, and the sleep preparation time interval; and The visual effects are applied in a stepwise manner during the sleep preparation time interval.

12. The HMD device of any one of claims 8-11, wherein, The at least one processor (861) is configured to obtain the sleep preparation start time based on the user's sleep data.

13. The HMD device of any one of claims 8-12, wherein, The at least one processor (861) is configured to: During the sleep preparation time interval, the display (851) is controlled to gradually reduce the proportion of the virtual image in the VST image, or During the sleep preparation time interval, a preset background image is displayed instead of the non-virtual image.

14. The HMD device of any one of claims 8-12, wherein, The at least one processor (861) is configured to: The system controls the display to show the VST screen, in which blue light is adjusted differently for each object based on the attributes of one or more objects included in the VST screen; or, the system displays the VST screen, in which illuminance is adjusted differently for each object based on the attributes of one or more objects included in the VST screen. The attributes of the one or more objects included in the VST screen include at least one of the following: a first indicator indicating one of the non-virtual screen or the virtual screen including the corresponding object; a second indicator indicating whether the corresponding object is emitting light; or a third indicator indicating the light source type of the corresponding object.

15. The HMD device of any one of claims 8-14, wherein, The at least one processor (861) is configured to: When the sleep preparation start time is reached, the display is controlled to show a message indicating whether to start preparing to fall asleep; as well as Based on the interaction with the user using the at least one sensor (310, 320, 330) or the at least one camera, determine whether to apply the visual effect in a stepwise manner.

Images