Electronic device, method, and non-transitory computer-readable storage medium for controlling screen displayed on display

Abstract

At least one processor of the wearable device is configured to identify, by means of at least one camera, whether the eyes of a user wearing the wearable device blink within a designated time period, and on the basis of identifying that the eyes of the user do not blink within the designated time period: while a first line caused to move from the top boundary toward the bottom boundary of a screen displayed on a display is displayed, display a first space between the first line and the top boundary on the basis of a second brightness which is lower than a first brightness; while a second line caused to move from the bottom boundary toward the top boundary of the screen is displayed, display a second space between the second line and the bottom boundary on the basis of the second brightness which is lower than the first brightness; and on the basis of identifying that the first line and the second line meet, display the screen on the basis of the first brightness.

Description

Electronic device, method, and non-transient computer-readable storage medium for controlling a screen displayed through a display

[0001] The following descriptions relate to an electronic device, a method, and a non-transient computer-readable storage medium for controlling a screen displayed through a display.

[0002] A wearable device may be a head-mounted device (HMD) that can be worn on a user's head. The distance between the screen displayed on the wearable device's display and the eyes of the user wearing the wearable device may be relatively close. Because the distance between the screen and the user's eyes is relatively close, the use of the wearable device may cause damage to the user's eyes.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] A wearable device is provided. The wearable device may include at least one camera. The wearable device may include a display. The wearable device may include a memory that stores instructions and includes one or more storage media. The wearable device may include at least one processor that includes a processing circuit. When the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the wearable device to identify whether the eyes of a user wearing the wearable device blink within a specified time interval using the at least one camera. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause a first line to be displayed moving from the top boundary to the bottom boundary of the screen displayed through the display, in accordance with the identification that the user's eye does not blink within the specified time interval, while a first line is displayed, and a first space between the first line and the top boundary is displayed based on a second brightness lower than the first brightness. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause a second line to be displayed moving from the bottom boundary to the top boundary of the screen displayed through the display, in accordance with the identification that the user's eye does not blink within the specified time interval, while a second space between the second line and the bottom boundary is displayed based on the second brightness lower than the first brightness.When the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause the screen displayed through the display to be displayed based on the first brightness, according to the identification that the first line and the second line are in contact.

[0005] A method performed by a wearable device is provided. The method may include an operation of identifying whether the eye of a user wearing the wearable device blinks within a specified time interval using at least one camera. The method may include an operation of displaying a first space between a first line and a top boundary based on a second brightness lower than a first brightness while a first line is displayed that causes movement from a top boundary to a bottom boundary of a screen displayed through the display, based on the identification that the user's eye does not blink within the specified time interval. The method may include an operation of displaying a second space between a second line and a bottom boundary based on a second brightness lower than the first brightness while a second line is displayed that causes movement from a bottom boundary to a top boundary of a screen displayed through the display, based on the identification that the user's eye does not blink within the specified time interval. The above method may include an operation of displaying a screen displayed through the display based on the first brightness, according to the identification that the first line and the second line are in contact.

[0006] A non-transient computer-readable storage medium is provided. The non-transient computer-readable storage medium may store one or more programs. The one or more programs may include instructions that cause the wearable device to identify, using at least one camera, whether the eye of a user wearing the wearable device blinks within a specified time interval when executed by at least one processor of the wearable device. The one or more programs may include instructions that cause the wearable device to display a first space between the first line and the top boundary based on a second brightness lower than a first brightness while a first line is displayed that moves from the top boundary of the screen displayed through the display, according to the identification that the user's eye does not blink within the specified time interval when executed by the at least one processor of the wearable device. The above one or more programs may include instructions that cause the wearable device to display a second space between the second line and the bottom boundary based on the second brightness lower than the first brightness, while a second line is displayed that is caused to move from the bottom boundary of the screen displayed through the display to the top boundary, upon identification that the user's eye does not blink within the specified time interval when executed by the at least one processor of the wearable device. The above one or more programs may include instructions that cause the wearable device to display a screen displayed through the display based on the first brightness, upon identification that the first line and the second line touch.

[0007] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0008] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0009] FIG. 2a illustrates an example of a perspective view of a wearable device.

[0010] FIG. 2b illustrates an example of one or more hardware components placed within a wearable device.

[0011] FIGS. 3A and FIGS. 3B illustrate an example of the appearance of a wearable device.

[0012] Figure 4 illustrates an example of a block diagram of a wearable device.

[0013] Figure 5 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0014] FIG. 6 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0015] Figure 7 illustrates examples of screens designed to induce a user to blink.

[0016] FIG. 8 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0017] FIG. 9 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0018] FIG. 10a illustrates examples of screens designed to induce a change in the user's pupil size.

[0019] FIG. 10b illustrates examples of screens designed to induce a change in the user's pupil size.

[0020] FIG. 11 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0021] FIG. 12 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0022] Figure 13 illustrates examples of screens designed to induce eye movements of a user.

[0023] FIG. 14 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0024] Figure 15 illustrates examples of screens designed to reduce eye strain in users.

[0025] FIG. 16 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display.

[0026] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of various embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0027] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0028] Terms used in the following description to refer to images (e.g., image, frame, camera frame, captured image, camera image), terms referring to a user's hand (e.g., hand object, candidate object, hand candidate object, bounding box, candidate hand object), terms referring to signals (e.g., signaling, control signal, data, control data, request signal, information), terms referring to locations (e.g., location information, area information, object information, object location, object coordinates, reference object, coordinate information, location, coordinate, relative coordinate, absolute coordinate, coordinate system), terms referring to values ​​(e.g., threshold value, reference value, reference area, reference range, level, threshold level, threshold, range, value, area), terms for operation states (e.g., step, operation, procedure), or terms referring to components of a device are examples provided for the convenience of explanation. Accordingly, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used. Additionally, terms such as '...part', '...device', '...object', '...body' used below may refer to at least one shape structure or a unit that processes a function.

[0029] Additionally, in this disclosure, expressions of "greater than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled; however, this is merely for the purpose of expressing an example and does not exclude descriptions of "greater than" or "less than." Conditions described as "greater than" may be replaced with "greater than," conditions described as "less than" may be replaced with "less than," and conditions described as "greater than and less than" may be replaced with "greater than and less than." Furthermore, "A" to "B" below refer to at least one of elements from A (including A) to B (including B). Below, "C" and / or "D" refers to including at least one of "C" or "D," i.e., {"C", "D", "C" and "D"}.

[0030] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0031] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0032] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0033] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0034] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0035] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0036] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0037] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0038] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0039] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0040] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0041] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0042] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0043] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0044] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0045] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0046] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0047] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0048] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0049] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0050] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0051] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0052] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0053] In embodiments of the present disclosure, an electronic device for displaying an image in a virtual space (e.g., the electronic device (101) of FIG. 1) may be a wearable device. The wearable device (101) may include a head-mounted display (HMD) that is wearable on a user's head. The wearable device (101) may be referred to as a head-mount device (HMD), a headgear electronic device, a glasses-type electronic device, a video see-through (VST) or visible see-through (VST) device, an extended reality (XR) device, a virtual reality (VR) device, and / or an augmented reality (AR) device. Although the external appearance of the wearable device (101) having the form of glasses is illustrated, embodiments of the present disclosure are not limited thereto. An example of a hardware configuration included within the wearable device (101) is described exemplarily with reference to FIG. 4. An example of the structure of a wearable device (101) that can be worn on the head of a user (110) is described with reference to FIG. 2a, FIG. 2b, FIG. 3a, and / or FIG. 3b. The wearable device (101) may be referred to as an electronic device. For example, the electronic device may be combined with an accessory (e.g., a strap) for attaching to the user's head to form an HMD.

[0054] In one embodiment, the wearable device (101) can perform functions related to augmented reality (AR) and / or mixed reality (MR). For example, while a user (e.g., user (110) of FIG. 8a) is wearing the wearable device (101), the wearable device (101) may include at least one lens positioned adjacent to the user's (110) eye. The wearable device (101) may combine light emitted from the display of the wearable device (101) with ambient light passing through the lens. The display area of ​​the display may be formed within the lens through which the ambient light passes. Since the wearable device (101) combines the ambient light and the light emitted from the display, the user (110) can see an image in which a real object (or physical object) recognized by the ambient light and a virtual object formed by the light emitted from the display are mixed. The augmented reality, mixed reality, and / or virtual reality described above may be referred to as extended reality (XR).

[0055] In one embodiment, the wearable device (101) may perform functions related to VST (video see-through or visible see-through) and / or virtual reality (VR). For example, while a user (110) is wearing the wearable device (101), the wearable device (101) may include a housing that covers the user's (110) eyes. The wearable device (101) may include a display disposed on a first surface of the housing facing the eyes while in the state. The wearable device (101) may include a camera disposed on a second surface opposite to the first surface. Using the camera, the wearable device (101) may acquire an image and / or video representing ambient light. A wearable device (101) may output the image and / or video within a display placed on the first surface, thereby allowing a user (110) to perceive the ambient light through the display. A displaying area (or displaying region) (or active area or active region) of the display placed on the first surface may be formed by one or more pixels included in the display. The wearable device (101) may composite a virtual object with the image and / or video output through the display, thereby allowing the user (110) to perceive the virtual object together with a real object perceived by the ambient light.

[0056] In one embodiment, the wearable device (101) can identify or recognize the position or location and / or direction or orientation of the wearable device (101) based on images and / or videos obtained or acquired using a camera. The wearable device (101) can obtain information about the external space using one or more cameras and / or one or more sensors. The information may include the geographic location of the external space (e.g., GPS (global positioning system) coordinates) identified by one or more sensors. The information may include images and / or videos of the external space identified by one or more cameras. The wearable device (101) can identify external objects contained in the external space from the images and / or videos by performing object recognition on the images and / or videos.

[0057] Hereinafter, with reference to FIGS. 2a, FIGS. 2b, FIGS. 3a, FIGS. 3b, and FIGS. 4, an example of a hardware configuration of a wearable device (101) is described.

[0058] FIG. 2a illustrates an example of a perspective view of a wearable device. FIG. 2b illustrates an example of one or more hardware components disposed within the wearable device. According to one embodiment, the wearable device (101) may have the form of glasses that are wearable on a body part (e.g., head) of a user (110). The wearable device (101) of FIG. 2a and FIG. 2b may be an example of the electronic device (101) of FIG. 1. The wearable device (101) may include a head-mounted display (HMD). For example, the housing of the wearable device (101) may include a flexible material such as rubber and / or silicone that has a shape that adheres to a part of the user's (110) head (e.g., a part of the face covering both eyes). For example, the housing of the wearable device (101) may include one or more straps that can be twined around the head of a user (110), and / or one or more temples that can be attached to the ears of the head.

[0059] Referring to FIG. 2a, a wearable device (101) according to one embodiment may include at least one display (250) and a frame (200) supporting at least one display (250).

[0060] According to one embodiment, a wearable device (101) may be worn on a part of the body of a user (110). The wearable device (101) may provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to the user (110) wearing the wearable device (101). For example, the wearable device (101) may display a virtual reality image provided by at least one optical device (282, 284) of FIG. 2b on at least one display (250) in response to a designated gesture of the user (110) obtained through the motion recognition camera (260-2, 260-3) of FIG. 2b.

[0061] According to one embodiment, at least one display (250) can provide visual information to a user (110). For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at locations corresponding to the left and right eyes of the user (110), respectively.

[0062] Referring to FIG. 2b, at least one display (250) may provide visual information transmitted from external light to the user (110) through a lens included in at least one display (250), and other visual information distinct from said visual information. The lens may be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) may include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area may be formed on the second surface (232) of at least one display (250). When the user (110) wears the wearable device (101), external light may be transmitted to the user (110) by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided by at least one optical device (282, 284) on a real image transmitted through external light in a display area formed on the second surface (232).

[0063] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits it to a user (110). At least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano pattern may be formed on the exterior or at least a portion of the interior of at least one waveguide (233, 234). The nano pattern may be formed based on a polygonal and / or curved grating structure. Light incident on one end of at least one waveguide (233, 234) may be propagated to the other end of at least one waveguide (233, 234) by the nano pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., DOE (diffractive optical element), HOE (holographic optical element)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be placed within a wearable device (101) to guide a screen displayed by at least one display (250) to the eyes of a user (110). For example, the screen may be transmitted to the eyes of the user (110) based on total internal reflection (TIR) ​​occurring within at least one waveguide (233, 234).

[0064] A wearable device (101) can analyze an object included in a real-world image collected through a camera (260-4), combine a virtual object corresponding to an object among the analyzed objects that is the target of augmented reality provision, and display it on at least one display (250). The virtual object may include at least one of text and an image regarding various information related to the object included in the real-world image. The wearable device (101) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (101) can perform spatial recognition (e.g., SLAM (simultaneous localization and mapping)) using a multi-camera and / or time-of-flight (ToF). A user (110) wearing the wearable device (101) can view the image displayed on at least one display (250).

[0065] According to one embodiment, the frame (200) may be formed as a physical structure that allows the wearable device (101) to be worn on the body of a user (110). According to one embodiment, the frame (200) may be configured so that when the user (110) wears the wearable device (101), the first display (250-1) and the second display (250-2) can be positioned corresponding to the left and right eyes of the user (110). The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) so that they are positioned corresponding to the left and right eyes of the user (110).

[0066] Referring to FIG. 2a, the frame (200) may include an area (220) in which at least a portion comes into contact with a part of the user's (110) body when the user (110) wears the wearable device (101). For example, the area (220) of the frame (200) in contact with a part of the user's (110) body may include an area in contact with a part of the user's (110) nose, a part of the user's (110) ear, and a part of the side of the user's (110) face that the wearable device (101) comes into contact with. According to one embodiment, the frame (200) may include a nose pad (210) that comes into contact with a part of the user's (110) body. When the wearable device (101) is worn by the user (110), the nose pad (210) may come into contact with a part of the user's (110) nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another part of the user's (110) body that is distinct from the part of the user's (110) body.

[0067] For example, the frame (200) may include a first rim (201) covering at least a portion of a first display (250-1), a second rim (202) covering at least a portion of a second display (250-2), a bridge (203) positioned between the first rim (201) and the second rim (202), a first pad (211) positioned along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) positioned along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) extending from the first rim (201) and fixed to a portion of the wearer's ear, and a second temple (205) extending from the second rim (202) and fixed to a portion of the ear opposite to the first. The first pad (211) and the second pad (212) may come into contact with a portion of the user's (110) nose, and the first temple (204) and the second temple (205) may come into contact with a portion of the user's (110) face and a portion of the user's (110) ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through a first hinge unit (206) positioned between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through a second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (101) may identify an external object touching the frame (200) (e.g., the fingertip of the user (110)) and / or a gesture performed by said external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of the surface of the frame (200).

[0068] According to one embodiment, the wearable device (101) may include hardware that performs various functions (e.g., hardware to be described later based on the block diagram of FIG. 4). For example, the hardware may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers (255-1, 255-2)), a microphone (e.g., microphones (265-1, 265-2, 265-3)), a light-emitting module (not shown), and / or a PCB (printed circuit board) (290) (e.g., a printed circuit board). The various hardware may be placed within a frame (200).

[0069] According to one embodiment, a microphone (e.g., microphones (265-1, 265-2, 265-3)) of a wearable device (101) is positioned on at least a portion of a frame (200) to acquire a sound signal. A first microphone (265-1) positioned on a bridge (203), a second microphone (265-2) positioned on a second rim (202), and a third microphone (265-3) positioned on a first rim (201) are shown in FIG. 2b, but the number and position of the microphones (265) are not limited to the embodiment of FIG. 2b. If there are two or more microphones (265) included in the wearable device (101), the wearable device (101) can identify the direction of the sound signal by using a plurality of microphones positioned on different portions of the frame (200).

[0070] According to one embodiment, at least one optical device (282, 284) may project a virtual object onto at least one display (250) to provide various image information to a user (110). For example, at least one optical device (282, 284) may be a projector. At least one optical device (282, 284) may be disposed adjacent to at least one display (250) or included within at least one display (250) as part of at least one display (250). According to one embodiment, a wearable device (101) may include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) positioned at the edge of a first display (250-1) and a second optical device (284) positioned at the edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) positioned on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) positioned on the second display (250-2).

[0071] In one embodiment, the camera (260) may include a shooting camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 206-3). The shooting camera (260-4), the eye tracking camera (260-1), and the motion recognition camera (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position of the eyes or the gaze of a user (110) wearing the wearable device (101). For example, the wearable device (101) may detect the gaze from an image containing the pupils of the user (110) obtained through the eye tracking camera (260-1). A wearable device (101) can identify an object (e.g., a real object, and / or a virtual object) focused by the user (110) by using the user's (110) gaze obtained through an eye-tracking camera (260-1). The wearable device (101), having identified the focused object, can perform a function (e.g., gaze interaction) for interaction between the user (110) and the focused object. The wearable device (101) can represent a part corresponding to the eyes of an avatar representing the user (110) in a virtual space by using the user's (110) gaze obtained through an eye-tracking camera (260-1). The wearable device (101) can render an image (or screen) displayed on at least one display (250) based on the position of the user's (110) eyes. For example, the visual quality of a first region related to the gaze within an image and the visual quality of a second region distinct from the first region (e.g., resolution, brightness, saturation, grayscale, PPI (pixels per inch)) may differ from each other.In the present disclosure, the term “resolution” is used to refer to the density of pixels of an image and / or display. The density of pixels and / or resolution may be measured based on units of PPI and / or dpi (dots per inch) or may be parameterized. The wearable device (101) may acquire an image having a visual quality of a first region and a visual quality of a second region that match the gaze of the user (110) using foveated rendering. For example, if the wearable device (101) supports an iris recognition function, user authentication may be performed based on iris information acquired using an eye-tracking camera (260-1). An example in FIG. 2b in which the eye tracking camera (260-1) is positioned toward the right eye of the user (110) is shown, but the embodiment is not limited thereto, and the eye tracking camera (260-1) may be positioned solely toward the left eye of the user (110) or toward both eyes.

[0072] In one embodiment, the camera (260-4) can capture a real image or background to be matched with a virtual image in order to implement augmented reality or mixed reality content. The camera (260-4) can be used to acquire high-resolution images based on HR (high resolution) or PV (photo video). The camera (260-4) can capture an image of a specific object located at a position viewed by the user (110) and provide the image to at least one display (250). The at least one display (250) can display a single image in which information regarding a real image or background including the image of the specific object acquired using the camera (260-4) and a virtual image provided through at least one optical device (282, 284) are superimposed. The wearable device (101) can compensate for depth information (e.g., the distance between the wearable device (101) and an external object acquired through a depth sensor) using the image acquired through the camera (260-4). The wearable device (101) can perform object recognition through an image acquired using a shooting camera (260-4). The wearable device (101) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the shooting camera (260-4). The wearable device (101) can perform a pass-through function to superimpose an image acquired through the shooting camera (260-4) onto at least a portion of a screen representing a virtual space while displaying the screen representing a virtual space on at least one display (250). In one embodiment, the shooting camera (260-4) may be placed on a bridge (203) positioned between a first rim (201) and a second rim (202).

[0073] The eye tracking camera (260-1) can achieve more realistic augmented reality by matching the user's (110) gaze with visual information provided to at least one display (250) by tracking the gaze of a user (110) wearing a wearable device (101). For example, when the user (110) looks straight ahead, the wearable device (101) can naturally display environmental information related to the user's (110) front on at least one display (250) at the location where the user (110) is situated. The eye tracking camera (260-1) can be configured to capture an image of the user's (110) pupil to determine the user's (110) gaze. For example, the eye tracking camera (260-1) can receive a gaze detection light reflected from the user's (110) pupil and track the user's (110) gaze based on the position and movement of the received gaze detection light. In one embodiment, the eye-tracking camera (260-1) may be positioned at locations corresponding to the left and right eyes of the user (110). For example, the eye-tracking camera (260-1) may be positioned to face the direction in which the user (110) wearing the wearable device (101) is located within the first rim (201) and / or the second rim (202).

[0074] A motion recognition camera (260-2, 260-3) can provide a specific event to a screen provided on at least one display (250) by recognizing the movement of the user (110)'s entire body or part thereof, such as the user's torso, hands, or face. The motion recognition camera (260-2, 260-3) can recognize the user's (110) gesture, acquire a signal corresponding to the gesture, and provide a display corresponding to the signal to at least one display (250). A processor can identify the signal corresponding to the gesture and, based on the identification, perform a designated function. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for a 6-degrees-of-freedom pose (6 dof pose). The processor can perform gesture recognition and / or object tracking functions using motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) may be placed on the first rim (201) and / or the second rim (202).

[0075] The camera (260) included in the wearable device (101) is not limited to the eye-tracking camera (260-1) and motion recognition camera (260-2, 260-3) described above. For example, the wearable device (101) can identify external objects included within the field of view (FoV) by using a camera positioned toward the field of view (FoV) of the user (110). The identification of external objects by the wearable device (101) can be performed based on a sensor for identifying the distance between the wearable device (101) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus (AF) function and / or an optical image stabilization (OIS) function. For example, the wearable device (101) may include a camera (260) (e.g., a face tracking camera) positioned toward the face to acquire an image including the face of a user (110) wearing the wearable device (101).

[0076] Although not illustrated, according to one embodiment, the wearable device (101) may further include a light source (e.g., LED) that emits light toward a subject (e.g., the user's (110) eyes, face, and / or an object outside the FoV) being photographed using a camera (260). The light source may include an LED of infrared wavelength. The light source may be placed in at least one of the frame (200) and hinge units (206, 207).

[0077] According to one embodiment, the battery module (270) can supply power to the electronic components of the wearable device (101). In one embodiment, the battery module (270) may be placed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may each be placed in the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be placed at the end of the first temple (204) and / or the second temple (205).

[0078] The antenna module (275) can transmit a signal or power to the outside of the wearable device (101) or receive a signal or power from the outside. In one embodiment, the antenna module (275) may be placed within the first temple (204) and / or the second temple (205). For example, the antenna module (275) may be placed close to one side of the first temple (204) and / or the second temple (205).

[0079] The speaker (255) can output an acoustic signal to the outside of the wearable device (101). The acoustic output module may be referred to as the speaker. In one embodiment, the speaker (255) may be placed within a first temple (204) and / or a second temple (205) to be placed adjacent to the ear of a user (110) wearing the wearable device (101). For example, the speaker (255) may include a second speaker (255-2) placed adjacent to the left ear of the user (110) by being placed within the first temple (204), and a first speaker (255-1) placed adjacent to the right ear of the user (110) by being placed within the second temple (205).

[0080] A light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (101) to the user (110). For example, if the wearable device (101) requires charging, it may emit red light at a constant frequency. In one embodiment, the light-emitting module may be placed on the first rim (201) and / or the second rim (202).

[0081] Referring to FIG. 2b, a wearable device (101) according to one embodiment may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of a first temple (204) or a second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardware components included in the wearable device (101) (e.g., hardware components illustrated by different blocks in FIG. 4) may be disposed on the PCB (290). The wearable device (101) may include a flexible PCB (FPCB) for interconnecting the hardware components.

[0082] According to one embodiment, a wearable device (101) may include at least one of a gyroscope sensor, a gravity sensor, and / or an acceleration sensor for detecting the posture of the wearable device (101) and / or the posture of a body part (e.g., head) of a user (110) wearing the wearable device (101). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on designated three-dimensional axes (e.g., x-axis, y-axis, and z-axis) that are perpendicular to each other. The gyroscope sensor may measure the angular velocity of each of the designated three-dimensional axes (e.g., x-axis, y-axis, and z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyroscope sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (101) can identify motions and / or gestures of a user (110) performed to execute or stop specific functions of the wearable device (101) based on an IMU.

[0083] FIGS. 3A and 3B illustrate an example of the appearance of a wearable device. The wearable device (101) of FIGS. 3A and 3B may be an example of the electronic device (101) of FIG. 1, or the wearable device (101) of FIGS. 2A and 2B. According to one embodiment, an example of the appearance of a first surface (310) of the housing of the wearable device (101) may be illustrated in FIG. 3A, and an example of the appearance of a second surface (320) opposite to the first surface (310) may be illustrated in FIG. 3B.

[0084] Referring to FIG. 3a, according to one embodiment, a first surface (310) of a wearable device (101) may have a shape that is attachable to a body part of a user (110) (e.g., the face of the user (110)). Although not illustrated, the wearable device (101) may further include a strap for fixing to a body part of the user (110) and / or one or more temples (e.g., a first temple (204) and / or a second temple (205) of FIG. 2a and FIG. 2b). A first display (250-1) for outputting an image to the left eye among the two eyes of the user (110) and a second display (250-2) for outputting an image to the right eye among the two eyes may be disposed on the first surface (310). The wearable device (101) may further include rubber or silicone packing formed on the first surface (310) to prevent interference by light different from light emitted from the first display (250-1) and the second display (250-2) (e.g., ambient light).

[0085] According to one embodiment, a wearable device (101) may include cameras (260-1) for photographing and / or tracking both eyes of a user (110) adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referenced to the eye-tracking camera (260-1) of FIG. 2B. According to one embodiment, a wearable device (101) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of a user (110). The cameras (260-5, 260-6) may be referenced to FT cameras. The wearable device (101) can control an avatar representing the user (110) in a virtual space based on the motion of the user's (110) face identified using cameras (260-5, 260-6). For example, the wearable device (101) can change the texture and / or shape of a part of the avatar (e.g., a part of the avatar representing a human face) using information obtained by cameras (260-5, 260-6) (e.g., FT cameras) and representing the facial expression of the user (110) wearing the wearable device (101).

[0086] Referring to FIG. 3b, on a second surface (320) opposite to the first surface (310) of FIG. 3a, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)), and / or a sensor (e.g., a depth sensor (330)) may be placed to acquire information related to the external environment of the wearable device (101). For example, cameras (260-7, 260-8, 260-9, 260-10) may be placed on the second surface (320) to recognize external objects. The cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2b.

[0087] For example, using cameras (260-11, 260-12), the wearable device (101) can acquire images and / or videos to be transmitted to each of the user's (110) eyes. Camera (260-11) may be placed on the second surface (320) of the wearable device (101) to acquire an image to be displayed through a second display (250-2) corresponding to the right eye among the two eyes. Camera (260-12) may be placed on the second surface (320) of the wearable device (101) to acquire an image to be displayed through a first display (250-1) corresponding to the left eye among the two eyes. Cameras (260-11, 260-12) may be referenced to the shooting camera (260-4) of FIG. 2B.

[0088] According to one embodiment, the wearable device (101) may include a depth sensor (330) disposed on a second surface (320) to identify the distance between the wearable device (101) and an external object. Using the depth sensor (330), the wearable device (101) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user (110) wearing the wearable device (101). Although not illustrated, a microphone may be disposed on the second surface (320) of the wearable device (101) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0089] Hereinafter, with reference to FIG. 4, the hardware or software configuration of the wearable device (101) will be described.

[0090] FIG. 4 illustrates an example of a block diagram of a wearable device. The wearable device (101) of FIG. 4 may be an example of the electronic device (101) of FIG. 1 and the wearable device (101) of FIG. 2a to FIG. 3b.

[0091] Referring to FIG. 4, a wearable device (101) according to one embodiment may include a processor (410), memory (415), a display (250) (e.g., a first display (250-1) and / or a second display (250-2) of FIG. 2a, FIG. 2b, FIG. 3a, and FIG. 3b), and / or a sensor (420). The processor (410), memory (415), display (250) and / or sensor (420) may be electrically and / or operationally connected to each other by an electronic component such as a communication bus (402). In the present disclosure, the operational connection of the electronic components may include a direct connection established between the electronic components and / or an indirect connection established between the electronic components such that a first electronic component among the electronic components is controlled by a second electronic component among the electronic components. The type and / or number of electronic components included in the wearable device (101) are not limited to those shown in FIG. 4. For example, the wearable device (101) may include only some of the electronic components shown in FIG. 4.

[0092] A processor (410) of a wearable device (101) according to one embodiment may include a circuit (e.g., a processing circuit) for processing data based on one or more instructions. The circuit for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (101) may include one or more processors. According to one embodiment, the structure of the processor (410) is not limited to one embodiment of the present disclosure, and at least one circuit may be formed as a separate processor (e.g., an embedded secure element (eSE), a secure processor) physically separated from the processor (410). The processor (410) may have a structure of a multi-core processor such as a dual core, quad core, hexa core, and / or octa core. The multi-core processor structure of the processor (410) may include a structure based on multiple core circuits (e.g., big-little structure) distinguished by power consumption, clock, and / or computational amount per unit time. In one embodiment comprising a processor (410) having a multi-core processor structure, the operations and / or functions of the present disclosure may be performed individually or collectively by one or more cores included in the processor (410).

[0093] A memory (415) of a wearable device (101) according to one embodiment may include electronic components for storing data and / or instructions that are input to or output from a processor (410). The memory (415) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, and embedded multi-media card (eMMC). In one embodiment, the memory (415) may be referred to as storage.

[0094] In one embodiment, a display (250) of a wearable device (101) can output visualized information to a user (110) of the wearable device (101). A display (250) arranged in front of the eyes of a user (110) wearing the wearable device (101) may be placed in at least a part of the housing of the wearable device (101) (e.g., a first display (250-1) and / or a second display (250-2) of FIG. 2a, FIG. 2b, FIG. 3a, and FIG. 3b). For example, the display (250) may be controlled by a processor (410) including circuits such as a CPU, a GPU (graphic processing unit), and / or a DPU (display processing unit) to output visualized information to the user (110). The display (250) may include a flexible display, a flat panel display (FPD), and / or electronic paper. The display (250) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). Embodiments are not limited thereto, for example, if the wearable device (101) includes a lens for transmitting external light (or ambient light), the display (250) may include a projector (or projection assembly) for projecting light onto the lens. In one embodiment, the display (250) may be referred to as a display panel and / or a display module. The pixels included in the display (250) can be positioned toward either of the user's (110) eyes when the wearable device (101) is worn by the user (110).For example, the display (250) may include display areas (or active areas) corresponding to each of the user's (110) two eyes.

[0095] In one embodiment, a sensor (420) of a wearable device (101) may generate electrical information that can be processed by a processor (410) and / or a memory (415) from non-electronic information associated with the wearable device (101). For example, the sensor (420) may include a global positioning system (GPS) sensor for detecting the geographic location of the wearable device (101). In addition to the GPS method, the sensor (420) may generate information indicating the geographic location of the wearable device (101) based on a global navigation satellite system (GNSS), such as Galileo or Beidou (compass). This information may be stored in the memory (415), processed by the processor (410), and / or transmitted to another electronic device distinct from the wearable device (101) via a communication circuit.

[0096] Referring to FIG. 4, an image sensor (421) and / or a motion sensor (422) are illustrated as examples of sensors (420) included in a wearable device (101). The sensor (420) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate electrical signals representing the color and / or brightness of light. The image sensor (421) may be referred to as a camera. A plurality of light sensors included in the image sensor (421) may be arranged in the form of a two-dimensional grid (2 dimensional array). The image sensor (421) may acquire the electrical signals of each of the plurality of light sensors substantially simultaneously to generate two-dimensional frame data corresponding to the light reaching the light sensors of the two-dimensional grid. For example, photo data captured using the image sensor (421) may refer to one (a) two-dimensional frame data acquired from the image sensor (421). For example, video data captured using an image sensor (421) may refer to a sequence of multiple two-dimensional frame data obtained from the image sensor (421) along a frame rate. The image sensor (421) may further include a flash light for outputting light in the direction in which the image sensor (421) receives light.

[0097] According to one embodiment, the wearable device (101) may include a plurality of image sensors arranged toward different directions as an example of an image sensor (421). As described above with reference to FIGS. 2A, 2B, 3A, and 3B, the plurality of image sensors may include an eye-tracking camera (e.g., the eye-tracking camera (260-1) of FIGS. 2B and 3A) configured to be arranged toward the eyes of a user (110) wearing the wearable device (101). The plurality of image sensors may include an outward camera. A processor (410) may identify the direction of the user's (110) gaze using an image and / or video obtained from the eye-tracking camera. The eye-tracking camera may include an infrared (IR) sensor. The eye-tracking camera may be referred to as an eye sensor and / or an eye tracker.

[0098] An external camera may be positioned facing the front of a user (110) wearing the wearable device (101) (e.g., a direction in which both eyes may face). The wearable device (101) may include a plurality of external cameras. The embodiments are not limited thereto, and the external camera may be positioned facing the external space. Using images and / or videos obtained from the external camera, the processor (410) may identify external objects. For example, the processor (410) may identify the position, shape, and / or gesture (e.g., hand gesture) of the hand of the user (110) wearing the wearable device (101) based on images and / or videos obtained from the external camera. Using images and / or videos of the external environment obtained from the external camera, the processor (410) may recognize or track one or more objects within the external environment.

[0099] According to one embodiment, the motion sensor (422) may output an electrical signal representing gravitational accelerations, accelerations, and / or angular velocities of a plurality of axes (e.g., x-axis, y-axis, and z-axis) that are perpendicular to each other and are based on a designated origin within the wearable device (101) and / or the motion sensor (422). For example, the processor (410) may repeatedly receive or acquire sensor data from the motion sensor (422), including accelerations, angular velocities, and / or magnitudes of the magnetic field of a number of the plurality of axes, based on a designated period (e.g., 1 millisecond). In one embodiment, the motion sensor (422) may be referred to as an inertial measurement unit (IMU). The sensor (420) included in the wearable device (101) is not limited to the above and may include a grip sensor, a proximity sensor, a heart rate sensor, a fingerprint sensor, an ambient light sensor, and / or a ToF sensor. By using a motion sensor (422), the processor (410) can detect motion of the wearable device (101) (e.g., motion of the wearable device (101) caused by a user (110) wearing the wearable device (101).

[0100] In one embodiment, a communication circuit (not shown) of a wearable device (101) may include a hardware component for supporting the transmission and / or reception of a signal between the wearable device (101) and an external electronic device (e.g., electronic device (102), electronic device (104)). The communication circuit may include, for example, at least one of a modem, an antenna, and an O / E (optic / electronic) converter. The communication circuit (430) may support the transmission and / or reception of an electrical signal based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (Bluetooth low energy), Zigbee, LTE (long term evolution), and 5G NR (new radio).

[0101] According to one embodiment, within the memory (415) of the wearable device (101), one or more instructions (or commands) representing data to be processed by the processor (410) of the wearable device (101), calculations to be performed, and / or operations may be stored. A set of one or more instructions may be referred to as a program, firmware, operating system, process, routine, sub-routine, and / or software application (hereinafter, application). For example, the wearable device (101) and / or processor (410) may perform at least one of the operations of FIGS. 5 through 16 when a set of a plurality of instructions distributed in the form of an operating system, firmware, driver, program, and / or software application is executed. In the following, the statement that a software application is installed in a wearable device (101) may mean that one or more instructions provided in the form of a software application (or package) are stored in memory (415), and that the one or more applications are stored in an executable format (e.g., a file having an extension specified by the operating system of the wearable device (101)) by the processor (410). For example, the application may include a program and / or library related to a service provided to a user (110).

[0102] Referring to FIG. 4, programs installed on a wearable device (101) may be included in any one of different layers, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480), based on the target. For example, within the hardware abstraction layer (480) (e.g., an Android system HAL, and / or an XR HAL), programs (e.g., modules, or drivers) designed to target the hardware of the wearable device (101) (e.g., a display (250), and / or a sensor (420)) may be included. The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing XR (extended reality) services. For example, the layers illustrated in FIG. 4 are logically (or for convenience of explanation) separated, and may not imply that the address space of memory (415) is separated by said layers.

[0103] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or the application layer (440) (e.g., a location tracker (471), a spatial recognizer (472), a gesture tracker (473), an eye tracker (474), a face tracker (475), and / or a renderer (490)) may be included. The programs included in the framework layer (450) may provide an application programming interface (API) that is executable (or invokeable) based on other programs.

[0104] For example, within the application layer (440), a program designed to target a user (110) of the wearable device (101) may be included. Examples of programs included in the application layer (440) include an XR (extended reality) system UI (user interface) (441) and / or an XR application (442), but embodiments are not limited thereto. For example, programs included in the application layer (440) (e.g., software applications) may call an API to cause the execution of a function supported by programs included in the framework layer (450).

[0105] For example, the wearable device (101) may display one or more visual objects on the display (250) to perform interaction with the user (110) based on the execution of the XR system UI (441). A visual object may mean an object that can be placed on the screen for the transmission of information and / or interaction, such as text, images, icons, videos, buttons, checkboxes, radio buttons, text boxes, sliders, and / or tables. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (101) may provide the user (110) with functions available in a virtual space based on the execution of the XR system UI (441).

[0106] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plugin (444) are depicted within the XR system UI (441), but are not limited thereto. For example, based on the XR system UI (441), the processor (410) may execute a lightweight renderer (443) and / or an XR plugin (444) within the framework layer (450).

[0107] For example, a wearable device (101) may acquire resources (e.g., APIs, system processes and / or libraries) used to define, create, and / or execute a rendering pipeline, which is permitted to be partially modified, based on the execution of a lightweight renderer (443). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline, which is permitted to be partially modified. The lightweight renderer (443) may include a renderer built prior to the execution of a software application (e.g., a pre-built renderer). For example, the wearable device (101) may acquire resources (e.g., APIs, system processes and / or libraries) used to define, create, and / or execute the entire rendering pipeline based on the execution of an XR plugin (444). The XR plugin (444) may be referred to as an open XR native client in terms of defining (or setting) the entire rendering pipeline.

[0108] For example, the wearable device (101) may display a screen representing at least a portion of a virtual space on the display (250) based on the execution of the XR application (442). The XR plugin (444-1) included in the XR application (442) may include instructions that support functions similar to the XR plugin (444) of the XR system UI (441). Descriptions of the XR plugin (444-1) that overlap with descriptions of the XR plugin (444) may be omitted. The wearable device (101) may trigger the execution of the virtual space manager (451) based on the execution of the XR application (442).

[0109] For example, the wearable device (101) may display an image on the display (250) in a virtual space based on the execution of an application (445). The application (445) may be configured to output image information for displaying a two-dimensional image. The wearable device (101) may trigger the execution of a virtual space manager (451) based on the execution of the application (445). The wearable device (101) may generate dual image information to display the two-dimensional image in a three-dimensional virtual space based on the execution of the application (445). Here, the dual image information may include a first image information for the left eye and a second image information for the right eye, taking into account binocular parallax. To display the two-dimensional image in a three-dimensional virtual space, the wearable device (101) may generate the dual image information based on the image information for displaying the two-dimensional image.

[0110] According to one embodiment, the wearable device (101) can provide a virtual space service based on the execution of a virtual space manager (451). For example, the virtual space manager (451) may include a platform for supporting the virtual space service. Based on the execution of the virtual space manager (451), the wearable device (101) can identify a virtual space formed based on the location of a user (110) indicated by data acquired through a sensor (420), and can display at least a portion of the virtual space on a display (250). The virtual space manager (451) may be referred to as a composition presentation manager (CPM).

[0111] For example, the virtual space manager (451) may include a runtime service (452). For example, the runtime service (452) may be referred to as an OpenXR runtime module (or OpenXR runtime program). The wearable device (101) may execute at least one of a pose prediction function, a frame timing function, and / or a spatial input function of the user (110) based on the execution of the runtime service (452). For example, the wearable device (101) may perform rendering for the virtual space service for the user (110) based on the execution of the runtime service (452). For example, a virtual space-related function that is executable by the application layer (440) based on the execution of the runtime service (452) may be supported.

[0112] For example, the virtual space manager (451) may include a pass-through manager (453). The wearable device (101) may display an image and / or video representing a real space acquired through an external camera superimposed on at least a portion of the screen while displaying a screen representing a virtual space (e.g., the screen (120) of FIG. 1) on the display (250) based on the execution of the pass-through manager (453).

[0113] For example, the virtual space manager (451) may include an input manager (454). The wearable device (101) may identify acquired data (e.g., sensor data) by executing one or more programs included within the recognition service layer (470) based on the execution of the input manager (454). The wearable device (101) may identify user inputs associated with the wearable device (101) using the acquired data. The user inputs may be associated with motions (e.g., hand gestures), gazes, and / or speech of the user (110) identified by a sensor (420) (e.g., an image sensor (421) such as an external camera). The user inputs may be identified based on an external electronic device connected (or paired) via a communication circuit.

[0114] For example, the perception abstract layer (460) can be used for data exchange between the virtual space manager (451) and the perception service layer (470). In terms of being used for data exchange between the virtual space manager (451) and the perception service layer (470), the perception abstract layer (460) can be referred to as an interface. As an example, the perception abstract layer (460) can be referred to as OpenPX. The perception abstract layer (460) can be used for a perception client and a perception service.

[0115] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data obtained from the sensor (420). The one or more programs may include at least one of a location tracker (471), a spatial recognizer (472), a gesture tracker (473), an eye tracker (474), a face tracker (475), and / or a renderer (490). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those shown in FIG. 4.

[0116] For example, the wearable device (101) can identify the posture of the wearable device (101) using the sensor (420) based on the execution of the position tracker (471). The wearable device (101) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (101) using data acquired using an external camera (e.g., image sensor (421)) and / or an IMU (e.g., motion sensor (422) including a gyroscope, accelerometer, and / or geomagnetic sensor) based on the execution of the position tracker (471). The position tracker (471) may be referred to as a head tracking (HeT) module (or head tracker, head tracking program).

[0117] For example, the wearable device (101) may acquire information to provide a three-dimensional virtual space corresponding to the surrounding environment (e.g., external space) of the wearable device (101) (or the user (110) of the wearable device (101)) based on the execution of the spatial recognizer (472). The wearable device (101) may reproduce the surrounding environment of the wearable device (101) in three dimensions using data acquired using an external camera (e.g., image sensor (421)) based on the execution of the spatial recognizer (472). The wearable device (101) may identify at least one of a plane, an incline, and a staircase based on the surrounding environment of the wearable device (101) reproduced in three dimensions based on the execution of the spatial recognizer (472). The spatial recognizer (472) may be referred to as a scene understanding (SU) module (or scene understanding program).

[0118] For example, the wearable device (101) can identify (or recognize) the pose and / or gesture of the user (110) of the wearable device (101) based on the execution of the gesture tracker (473). For example, the wearable device (101) can identify the pose and / or gesture of the user (110)'s hand using data acquired from an external camera (e.g., image sensor (421)) based on the execution of the gesture tracker (473). For example, the wearable device (101) can identify the pose and / or gesture of the user (110)'s hand based on data (or images) acquired using an external camera based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module (or hand tracking program) and / or a gesture tracking module.

[0119] For example, the wearable device (101) can identify (or track) the eye movements of the user (110) of the wearable device (101) based on the execution of the eye tracker (474). For example, the wearable device (101) can identify the eye movements of the user (110) using data obtained from an eye tracking camera (e.g., image sensor (421)) based on the execution of the eye tracker (474). The eye tracker (474) may be referred to as an eye tracking (ET) module (or eye tracking program) and / or a gaze tracking module.

[0120] For example, the recognition service layer (470) of the wearable device (101) may further include a face tracker (475) for tracking the face of the user (110). For example, the wearable device (101) may identify (or track) the movement of the user (110)'s face and / or the facial expression of the user (110) based on the execution of the face tracker (475). The wearable device (101) may estimate the facial expression of the user (110) based on the movement of the user (110)'s face based on the execution of the face tracker (475). For example, the wearable device (101) can identify the movement of the user's (110) face and / or the user's (110) facial expressions based on data (e.g., images and / or videos) acquired using a camera (e.g., a face tracking (FT) camera, a camera facing at least a part of the user's (110) face, an image sensor (421)) based on the execution of the face tracker (475). The face tracker (475) may be referred to as face tracking (FT) (or face tracking program) and / or a face tracking module.

[0121] Referring to FIG. 4, the renderer (490) may include instructions for rendering images in a three-dimensional virtual space. The processor (410) executing the renderer (490) may obtain at least one image to be displayed at least partially in a display area of ​​the display (250) in a software application. For example, the processor (410) executing the renderer (490) may determine the location of the area where an application (e.g., XR application (442), application (445)) will be rendered. The processor (410) executing the renderer (490) may generate an image of said application to be displayed on the display (250). The renderer (490) may synthesize images to generate a composite image to be displayed on the display (250).

[0122] For example, a processor (410) that executes a renderer (490) can divide the display area of ​​a display (250) into a foveated portion (or may be referred to as a foveated area) and a peripheral portion (or may be referred to as a residual area) using a gaze position calculated using a position tracker (471) and / or a gaze tracker (474). For example, a processor (410) that detects coordinate values ​​of the gaze position can determine the portion of the display area containing said coordinate values ​​as the foveated area. A DPU that executes a renderer (490) can acquire at least one image corresponding to each of said foveated area and said residual area, having a size smaller than the size of the entire display area of ​​the display (250) or having a resolution less than the resolution of the display area.

[0123] For example, the processor (410) that executes the renderer (490) can obtain or generate a composite image to be displayed on the display (250) by synthesizing an image corresponding to the foveated area and an image corresponding to the surrounding area. For example, the processor (410) can perform upscaling to enlarge the image corresponding to the surrounding area to the size of the entire display area of ​​the display (250). On the enlarged image, the processor (410) can combine the image corresponding to the foveated area to generate a composite image to be displayed on the display (250). Along the boundary line of the image corresponding to the foveated area, the processor (410) can mix the enlarged image and the image corresponding to the foveated area by applying a visual effect such as blur.

[0124] The wearable device (101) may be a head-mount device (HMD) that can be worn on a user's head. The distance between the screen displayed through the display (250) of the wearable device (101) and the eyes of the user wearing the wearable device (101) may be relatively close. Since the distance between the screen and the user's eyes is relatively close, the use of the wearable device (101) may cause damage to the user's eyes. Below, a wearable device (101) that performs actions for the user's eye health to reduce eye damage caused by the use of the wearable device (101) is described.

[0125] FIG. 5 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 5 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by the processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations (e.g., operation 503 and operation 504) may be performed in parallel.

[0126] Referring to FIG. 5, in operation 501, a wearable device (101) according to one embodiment can identify whether the eyes of a user wearing the wearable device (101) blink within a specified time interval. For example, the wearable device (101) can monitor the movement of the user's eyelids (e.g., upper eyelid and lower eyelid) using a camera (260). Based on monitoring the user's eyelids, the wearable device (101) can identify whether the user's eyes blink within a specified time interval. A timer may be used to identify whether the user's eyes blink within a specified time interval. In one example, the specified time interval may be one minute. However, this is merely an example and the present disclosure is not limited thereto. For example, the length of the specified time interval may be set differently from one minute by user input.

[0127] In operation 502, a wearable device (101) according to one embodiment may display a screen displayed through a display (250) based on a first brightness. If the eyes of a user wearing the wearable device (101) blink within a specified time interval, control to induce the user's eye blinking may not be required. Since control to induce the user's eye blinking is not required, the wearable device (101) may maintain the screen displayed based on the first brightness. For example, the wearable device (101) may display a screen displayed through the display (250) based on the first brightness upon identification that the user's eyes blink within a specified time interval. For example, the first brightness may be determined based on the brightness outside the wearable device (101) identified by a sensor (420) (e.g., an illuminance sensor). In one example, the first brightness may be referred to as default brightness or another term having an equivalent technical meaning. The wearable device (101) may restart or reset a timer for identifying whether the user's eye blinks within a specified time interval upon identification that the user's eye blinks within a specified time interval.

[0128] In operation 503, a wearable device (101) according to one embodiment may display a first space (or, first area, first dim layer) between a first line and a top boundary based on a second brightness. If the eyes of a user wearing the wearable device (101) do not blink within a specified time interval, control may be required to induce the user to blink. For example, the wearable device (101) may display text and / or an image on the screen to guide the start of control to induce the user to blink. In one example, the text may be 'no blink detected for a long time'. However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) can control the display (250) to display a screen that simulates, imitates, or replicates the movement of the upper eyelid during the user's eye-closing movement after text and / or an image has been displayed.

[0129] In one embodiment, the wearable device (101) may cause a first line to move from the top boundary of the screen displayed through the display (250) toward the bottom boundary upon identifying that the eyes of the user wearing the wearable device (101) do not blink within a specified time interval. The top boundary of the screen is an edge located at the top part of the screen displayed through the display (250) and may refer to the boundary between the screen and a part of the wearable device (101). The bottom boundary of the screen is an edge located at the bottom part of the screen displayed through the display (250) and may refer to the boundary between the screen and a part of the wearable device (101). For example, the first line may be caused to move until the first line touches the second line. For example, the first line may be caused to move from the top boundary of the screen toward the bottom boundary according to a specified speed. In one example, the first line may be a virtual line that is not displayed on the screen. In one example, the shape of the first line may change between the shape of the line forming the top boundary of the screen and the shape of the horizontal line of the screen as the first line moves from the top boundary toward the bottom boundary. However, this is merely an example and the present disclosure is not limited thereto.

[0130] In one embodiment, the wearable device (101) may display a first space (or, first region, first dim layer) between the first line and the upper boundary based on a second brightness lower than the first brightness while the first line moves from the upper boundary toward the lower boundary. For example, the second brightness may be defined as a ratio to the first brightness. In one example, the second brightness may correspond to 70% of the first brightness. However, this is merely an example and the present disclosure is not limited thereto. In one example, the second brightness may be referred to as dim brightness or other terms having an equivalent technical meaning. By displaying the first space (or, first region, first dim layer) that expands due to the movement of the first line based on a second brightness lower than the first brightness, the movement of the upper eyelid during the user's eye-closing movement may be simulated, imitated, or replicated.

[0131] In operation 504, a wearable device (101) according to one embodiment may display a second space (or, second area, second dim layer) between a second line and a lower boundary based on a second brightness. If the eyes of a user wearing the wearable device (101) do not blink within a specified time interval, control may be required to induce the user to blink. For example, the wearable device (101) may control the display (250) to display a screen that simulates, imitates, or replicates the movement of the lower eyelid during the user's eye-closing movement, after text and / or an image is displayed to guide the start of control to induce the user to blink.

[0132] In one embodiment, the wearable device (101) may cause a second line to move from the bottom boundary of the screen displayed through the display (250) toward the top boundary upon identifying that the eyes of the user wearing the wearable device (101) do not blink within a specified time interval. For example, the second line may be caused to move until the second line touches the first line. For example, the second line may be caused to move from the bottom boundary of the screen toward the top boundary according to a specified speed. In one example, the speed specified for the second line may be the same as the speed specified for the first line. In another example, the speed specified for the second line may be different from the speed specified for the first line. In one example, the second line may be a virtual line that is not displayed on the screen. In one example, the shape of the second line may change between the shape of the line forming the bottom boundary of the screen and the shape of the horizontal line of the screen while the second line moves from the bottom boundary toward the top boundary. However, this is merely an example, and the present disclosure is not limited thereto.

[0133] In one embodiment, the wearable device (101) may display a second space (or, second region, second dim layer) between the second line and the bottom boundary based on a second brightness lower than the first brightness while the second line moves from the bottom boundary toward the top boundary. By displaying the second space (or, second region, second dim layer) that expands due to the movement of the second line based on a second brightness lower than the first brightness, the movement of the lower eyelid during the user's eye-closing movement may be simulated, imitated, or replicated.

[0134] In one embodiment, the wearable device (101) may display a screen displayed through the display (250) based on a blur effect while the first line and the second line are moving. For example, a first space between the first line and the top boundary, a second space between the second line and the bottom boundary, and / or a third space between the first line and the second line may be displayed based on a blur effect. For example, the first space and the second space may be displayed based on a second brightness and a blur effect. For example, the third space may be displayed based on a first brightness and a blur effect.

[0135] In operation 505, a wearable device (101) according to one embodiment may display a screen displayed through a display (250) based on a first brightness, upon identifying that a first line and a second line are in contact. For example, a first line that causes movement from an upper boundary toward a lower boundary and a second line that causes movement from a lower boundary toward an upper boundary may be in contact with each other. The first line and the second line may be in contact with each other on a virtual line on the screen. In an example that is not limited, the wearable device (101) may identify that the first line and the second line are in contact upon identifying that the distance between the first line and the second line is less than a specified distance. In one example, the specified distance may be defined based on the number of pixels on the screen. However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) can control the display (250) to display a screen simulating the movement of eyelids to open the eyes, based on the identification that the first line and the second line are in contact. For example, the wearable device (101) can change the brightness of the screen displayed through the display (250) based on the identification that the first line and the second line are in contact. For example, the wearable device (101) can change the brightness of the screen displayed through the display (250) from a second brightness to a first brightness.

[0136] FIG. 6 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 6 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations (e.g., operation 605 and operation 606) may be performed in parallel.

[0137] Referring to FIG. 6, in operation 601, a wearable device (101) according to one embodiment can identify whether the eyes of a user wearing the wearable device (101) blink within a specified time interval. For example, the wearable device (101) can monitor the movement of the user's eyelids (e.g., upper eyelid and lower eyelid) using a camera (260). Based on monitoring the user's eyelids, the wearable device (101) can identify whether the user's eyes blink within a specified time interval. A timer may be used to identify whether the user's eyes blink within a specified time interval. In one example, the specified time interval may be one minute. However, this is merely an example and the present disclosure is not limited thereto. For example, the length of the specified time interval can be set differently from 1 minute by user input.

[0138] In operation 602, a wearable device (101) according to one embodiment may display a screen displayed through a display (250) based on a first brightness. If the eyes of a user wearing the wearable device (101) blink within a specified time interval, control to induce the user's eye blinking may not be required. Since control to induce the user's eye blinking is not required, the wearable device (101) may maintain the screen displayed based on the first brightness. For example, the wearable device (101) may display a screen displayed through the display (250) based on the first brightness upon identification that the user's eyes blink within a specified time interval. For example, the first brightness may be determined based on the brightness outside the wearable device (101) identified by a sensor (420) (e.g., an illuminance sensor). In one example, the first brightness may be referred to as default brightness or another term having an equivalent technical meaning.

[0139] In operation 603, a wearable device (101) according to one embodiment can identify whether the current situation corresponds to a specified situation. For example, the wearable device (101) can identify whether the current situation corresponds to a specified situation based on the identification that the eyes of a user wearing the wearable device (101) do not blink within a specified time interval. A specified situation may mean a situation defined to avoid interfering with a user using the wearable device (101). In one example, a specified situation may be referred to as an exception situation, an exception handling situation, a situation requiring exception handling, or other terms having an equivalent technical meaning.

[0140] In one embodiment, whether the current situation corresponds to a specified situation may be identified based on the type of object displayed through the display (250), the movement of the user's eyes identified using the camera (260), the type of application executed by the wearable device (101), and whether user input is obtained. For example, if the object displayed through the display (250) is text, the wearable device (101) may identify that the current situation corresponds to a specified situation. In one example, if text is displayed on the screen displayed through the display (250), the wearable device (101) may identify that the current situation corresponds to a specified situation. However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) may identify that the movement of the user's eyes identified using the camera (260) tracks the movement of the object on the screen. The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the movement of the user's eyes tracks the movement of an object on the screen. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that a specified application (e.g., a game application, a video application) is running. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that user input is obtained.

[0141] In operation 604, the wearable device (101) may display a screen displayed through the display (250) based on a first brightness. If the current situation is a designated situation to avoid disturbing the user using the wearable device (101), control to induce the user's eye blinking may not be required. Since control to induce the user's eye blinking is not required, the wearable device (101) may maintain a screen displayed based on the first brightness to avoid disturbing the user. For example, the wearable device (101) may display a screen displayed through the display (250) based on the first brightness according to the identification that the current situation corresponds to a designated situation. For example, the first brightness may be determined based on the brightness outside the wearable device (101) identified by a sensor (420) (e.g., an illuminance sensor).

[0142] In operation 605, the wearable device (101) may display a first space (or, first area, first dim layer) between a first line and a top boundary based on a second brightness. Control may be required to induce the user's eyes to blink when the user's eyes do not blink within a specified time interval and when there is no specified situation to avoid disturbing the user. For example, the wearable device (101) may display text and / or an image on a screen displayed via a display (250) to guide the start of control to induce the user's eyes to blink. In one example, the text may be 'no eye blink detected for a long time.' However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) can control the display (250) to display a screen that simulates, imitates, or replicates the movement of the upper eyelid during the user's eye-closing movement after text and / or an image has been displayed.

[0143] In one embodiment, the wearable device (101) may cause a first line to move from the top boundary of the screen displayed through the display (250) toward the bottom boundary upon identifying that the eyes of the user wearing the wearable device (101) do not blink within a specified time interval. The top boundary of the screen is an edge located at the top part of the screen displayed through the display (250) and may refer to the boundary between the screen and a part of the wearable device (101). The bottom boundary of the screen is an edge located at the bottom part of the screen displayed through the display (250) and may refer to the boundary between the screen and a part of the wearable device (101). For example, the first line may be caused to move until the first line touches the second line. For example, the first line may be caused to move from the top boundary of the screen toward the bottom boundary according to a specified speed. In one example, the first line may be a virtual line that is not displayed on the screen. In one example, the shape of the first line may change between the shape of the line forming the top boundary of the screen and the shape of the horizontal line of the screen as the first line moves from the top boundary toward the bottom boundary. However, this is merely an example and the present disclosure is not limited thereto.

[0144] In one embodiment, the wearable device (101) may display a first space (or, first region, first dim layer) between the first line and the upper boundary based on a second brightness lower than the first brightness while the first line moves from the upper boundary toward the lower boundary. For example, the second brightness may be defined as a ratio to the first brightness. In one example, the second brightness may correspond to 70% of the first brightness. However, this is merely an example and the present disclosure is not limited thereto. In one example, the second brightness may be referred to as dim brightness or other terms having an equivalent technical meaning. By displaying the first space (or, first region, first dim layer) that expands due to the movement of the first line based on a second brightness lower than the first brightness, the movement of the upper eyelid during the user's eye-closing movement may be simulated, imitated, or replicated.

[0145] In operation 606, a wearable device (101) according to one embodiment may display a second space (or, second area, second dim layer) between a second line and a lower boundary based on a second brightness. Control may be required to induce the user's eye to blink when the user's eye wearing the wearable device (101) does not blink within a specified time interval and when there is no specified situation to avoid disturbing the user. For example, the wearable device (101) may control the display (250) to display a screen that simulates, imitates, or replicates the movement of the lower eyelid during the user's eye-closing movement after text and / or an image is displayed to guide the start of the control to induce the user's eye to blink.

[0146] In one embodiment, the wearable device (101) may cause a second line to move from the bottom boundary of the screen displayed through the display (250) toward the top boundary upon identifying that the eyes of the user wearing the wearable device (101) do not blink within a specified time interval. For example, the second line may be caused to move until the second line touches the first line. For example, the second line may be caused to move from the bottom boundary of the screen toward the top boundary according to a specified speed. In one example, the speed specified for the second line may be the same as the speed specified for the first line. In another example, the speed specified for the second line may be different from the speed specified for the first line. In one example, the second line may be a virtual line that is not displayed on the screen. In one example, the shape of the second line may change between the shape of the line forming the bottom boundary of the screen and the shape of the horizontal line of the screen while the second line moves from the bottom boundary toward the top boundary. However, this is merely an example, and the present disclosure is not limited thereto.

[0147] In one embodiment, the wearable device (101) may display a second space (or, second region, second dim layer) between the second line and the bottom boundary based on a second brightness lower than the first brightness while the second line moves from the bottom boundary toward the top boundary. By displaying the second space (or, second region, second dim layer) that expands due to the movement of the second line based on a second brightness lower than the first brightness, the movement of the lower eyelid during the user's eye-closing movement may be simulated, imitated, or replicated.

[0148] In one embodiment, the wearable device (101) may display a screen displayed through the display (250) based on a blur effect while the first line and the second line are moving. For example, a first space between the first line and the top boundary, a second space between the second line and the bottom boundary, and / or a third space between the first line and the second line may be displayed based on a blur effect. For example, the first space and the second space may be displayed based on a second brightness and a blur effect. For example, the third space may be displayed based on a first brightness and a blur effect.

[0149] In operation 607, a wearable device (101) according to one embodiment may display a screen displayed through a display (250) based on a first brightness, upon identifying that a first line and a second line are in contact. For example, a first line that causes movement from an upper boundary toward a lower boundary and a second line that causes movement from a lower boundary toward an upper boundary may be in contact with each other. The first line and the second line may be in contact with each other on a virtual line on the screen. In an example that is not limited, the wearable device (101) may identify that the first line and the second line are in contact upon identifying that the distance between the first line and the second line is less than a specified distance. In one example, the specified distance may be defined based on the number of pixels on the screen. However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) can control the display (250) to display a screen simulating the movement of eyelids to open the eyes, based on the identification that the first line and the second line are in contact. For example, the wearable device (101) can change the brightness of the screen displayed through the display (250) based on the identification that the first line and the second line are in contact. For example, the wearable device (101) can change the brightness of the screen displayed through the display (250) from a second brightness to a first brightness.

[0150] Figure 7 illustrates examples of screens designed to induce a user to blink.

[0151] In the example illustrated in FIG. 7, the wearable device (101) can identify that the eyes of a user wearing the wearable device (101) have not blinked for a specified time interval (e.g., 1 minute). In the example illustrated in FIG. 7, the wearable device (101) can identify that the current situation is not a specified situation to avoid interfering with the user using the wearable device (101). The wearable device (101) can control the display (250) to display a screen to induce the user to blink, based on the identification that the user's eyes have not blinked for a specified time interval and the current situation is not a specified situation.

[0152] Referring to FIG. 7, for example, at a first time (710), the wearable device (101) may display objects (701) on the screen to guide the start of a control to induce a user's eye blink. For example, the objects (701) may include images and text to notify the user of the start of a control to induce a user's eye blink. In one example, the text may be "No eye blink detected for a long time." However, this is merely an example and the present disclosure is not limited thereto.

[0153] Referring to FIG. 7, for example, at the second time (720) and the third time (730), the wearable device (101) can cause the first line (702) and the second line (705) to move.

[0154] For example, the wearable device (101) may cause the first line (702) to move from the top boundary (703) of the screen displayed through the display (250) toward the bottom boundary (706). The first line (702) may be caused to move until the first line (702) comes into contact with the second line (705). While the first line (702) is moving, the wearable device (101) may display the first space (704) (or first area, first dim layer) between the first line (702) and the top boundary (703) based on a second brightness (or dim brightness) lower than the first brightness (or default brightness). By displaying the first space (704) based on a second brightness lower than the first brightness, the movement of the upper eyelid during the user's eye-closing movement can be simulated, imitated, or replicated.

[0155] For example, the wearable device (101) may cause the second line (705) to move from the bottom boundary (706) of the screen displayed through the display (250) toward the top boundary (703). The second line (705) may be caused to move until the second line (705) comes into contact with the first line (702). While the second line (705) is moving, the wearable device (101) may display a second space (707) (or, second area, second dim layer) between the second line (705) and the bottom boundary (706) based on a second brightness lower than the first brightness. By displaying the second space (707) based on a second brightness lower than the first brightness, the movement of the lower eyelid during the user's eye-closing movement may be simulated, imitated, or replicated.

[0156] For example, the wearable device (101) can display the screen displayed through the display (250) based on a blur effect while the first line (702) and the second line (705) are moving. For example, the first space (704) between the first line (702) and the upper boundary (703), the second space (707) between the second line (705) and the lower boundary (706), and the third space (708) between the first line (702) and the second line (705) can be displayed based on a blur effect. For example, the first space (704) and the second space (707) can be displayed based on a second brightness and a blur effect. For example, the third space (708) can be displayed based on a first brightness and a blur effect.

[0157] For example, the wearable device (101) may display a progress bar (709) on the screen corresponding to the time intervals in which the first line (702) and the second line (705) move. For example, the progress bar may be displayed along the edges of the screen. For example, the length of the progress bar displayed on the screen may indicate the degree of progress of movements to simulate eye blinking. In the example illustrated in FIG. 7, the progress bar (709) is displayed along the edges of the screen, but this is merely an example for illustrative purposes and the present disclosure is not limited thereto. The shape of the progress bar (709) and / or the position where the progress bar (709) is displayed on the screen is not limited to the example illustrated in FIG. 7. In one example, the progress bar (709) may be displayed within the first space (704) and / or the second space (707). In one example, the progress bar (709) may be displayed on a portion of the screen (e.g., an area where objects (701) are displayed). The progress bar (709) may have various shapes that indicate the progress of a specific action and / or may be placed at various locations.

[0158] Referring to FIG. 7, for example, at the fourth time (740), the wearable device (101) can identify that the first line (702) and the second line (705) are in contact. In the example illustrated in FIG. 7, the first line (702), which is caused to move from the upper boundary (703) toward the lower boundary (706), and the second line (705), which is caused to move from the lower boundary (706) toward the upper boundary (703), may be in contact on a virtual line on the screen. However, the present disclosure is not limited to the first line (702) and the second line (705) being in contact on a virtual line. For example, the wearable device (101) may identify that the first line (702) and the second line (705) are in contact based on the identification that the distance between the first line (702) and the second line (705) is less than a specified distance.

[0159] Referring to FIG. 7, for example, at the fifth time (750), the wearable device (101) may display a screen displayed through the display (250) based on a first brightness according to the identification that the first line (702) and the second line (705) are in contact. For example, the first area (705) and the second area (707) displayed based on the second brightness may be removed. For example, the blur effect applied to the screen during the second time (720) to the fourth time (740) may be removed. By changing the brightness of the screen displayed through the display (250) from the second brightness to the first brightness, a screen simulating the movement of eyelids to open the eyes may be displayed.

[0160] FIG. 8 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 8 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0161] Referring to FIG. 8, in operation 801, a wearable device (101) according to one embodiment can identify whether the pupil size of a user wearing the wearable device (101) is fixed. For example, the wearable device (101) can monitor the user's pupil size over a time interval (e.g., 30 seconds) using a camera (260). In one example, the pupil size may represent the diameter or area of ​​the pupil. Based on the monitoring, the wearable device (101) can identify whether the amount of change in the user's pupil size is below a threshold. In one example, the threshold may be 1 millimeter. However, this is merely an example and the present disclosure is not limited thereto. For example, the threshold may be preconfigured or set by user input. For example, the wearable device (101) can identify that the user's pupil size is fixed based on the identification that the amount of change in the user's pupil size is less than a threshold value. In another example, the wearable device (101) can identify that the user's pupil size is not fixed based on the identification that the amount of change in the user's pupil size is greater than or equal to a threshold value.

[0162] In operation 802, a wearable device (101) according to one embodiment may reduce the size of content (or image) displayed on a screen from a first size to a second size. If the pupil size of a user wearing the wearable device (101) is fixed, control to change the user's pupil size may be required. For example, the wearable device (101) may display text and / or an image on the screen to guide the start of control to change the user's pupil size. In one example, the text may be "No change in focus for a long time." However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) may display text on the screen to guide control to change the user's pupil size. In one example, the text may be "The screen will be temporarily reduced to change the pupil's focus." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) may initiate control to change the user's pupil size after text and / or an image is displayed. For example, the wearable device (101) may reduce the size of the content displayed on the screen. While the size of the content is being reduced, the wearable device (101) may monitor the user's pupil size using a camera (260). Based on the monitoring, the wearable device (101) may identify that the amount of change in the user's pupil size corresponds to a specified value (e.g., 2 mm). The wearable device (101) may stop reducing the size of the content upon identifying that the amount of change in the user's pupil size corresponds to a specified value. The size of the content reduced from the first size may be a second size.

[0163] In operation 803, a wearable device (101) according to one embodiment may display content based on a second size for a specified time interval (e.g., 10 seconds). By displaying content based on a second size reduced from a first size for a specified time interval, a change in the user's pupil size may be caused.

[0164] In one embodiment, the wearable device (101) can generate content (or images) for a space (or area) other than the content reduced to a second size using an artificial intelligence model. The artificial intelligence model used to generate the content may be a generative artificial intelligence model. For example, the generative artificial intelligence model may generate content for a space (or area) other than the content reduced to a second size based on the content reduced to a second size. The wearable device (101) can display the content generated for the space (or area) other than the content reduced to a second size through a display (250).

[0165] In one embodiment, the wearable device (101) may display a progress bar corresponding to a specified time interval on the screen. For example, the progress bar may be displayed along the edges of the screen. For example, the length of the progress bar displayed on the screen may indicate the progress of actions to change the user's pupil size.

[0166] In operation 804, a wearable device (101) according to one embodiment can increase the size of content displayed on the screen from a second size to a first size after a specified time interval.

[0167] In operation 805, a wearable device (101) according to one embodiment may refrain from reducing the size of the content. If the pupil size of the user wearing the wearable device (101) is not fixed, control to change the pupil size of the user may not be necessary. For example, the wearable device (101) may refrain from reducing the size of the content displayed on the screen upon identifying that the pupil size of the user is not fixed.

[0168] FIG. 9 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 9 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be performed by the processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0169] Referring to FIG. 9, in operation 901, a wearable device (101) according to one embodiment can identify whether the pupil size of a user wearing the wearable device (101) is fixed. For example, the wearable device (101) can monitor the user's pupil size over a time interval (e.g., 30 seconds) using a camera (260). In one example, the pupil size may represent the diameter or area of ​​the pupil. Based on the monitoring, the wearable device (101) can identify whether the amount of change in the user's pupil size is below a threshold. In one example, the threshold may be 1 millimeter. However, this is merely an example and the present disclosure is not limited thereto. For example, the threshold may be preconfigured or set by user input. For example, the wearable device (101) can identify that the user's pupil size is fixed based on the identification that the amount of change in the user's pupil size is less than a threshold value. In another example, the wearable device (101) can identify that the user's pupil size is not fixed based on the identification that the amount of change in the user's pupil size is greater than or equal to a threshold value.

[0170] In operation 902, a wearable device (101) according to one embodiment can identify whether the current situation corresponds to a specified situation. For example, the wearable device (101) can identify whether the current situation corresponds to a specified situation based on the identification that the user's pupil size is fixed. A specified situation may mean a situation defined to avoid interfering with the user using the wearable device (101). In one example, a specified situation may be referred to as an exception situation, an exception handling situation, an exception handling required situation, or other terms having an equivalent technical meaning.

[0171] In one embodiment, whether the current situation corresponds to a specified situation can be identified based on whether an object on a screen displayed through a display (250) is moving, whether a user wearing the wearable device (101) is moving, and whether user input is obtained. For example, the wearable device (101) can identify whether the distance of movement of an object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the object is moving based on the identification that the distance of movement of an object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the object is moving. For example, the wearable device (101) can identify whether a user wearing the wearable device (101) is moving by using a sensor (420) (e.g., an IMU (inertial measurement unit) sensor). The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the user is moving. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that user input is obtained.

[0172] In operation 903, the wearable device (101) according to one embodiment may refrain from reducing the size of the content (or image) displayed on the screen. If the current situation is a situation designated so as not to disturb the user using the wearable device (101), control to change the size of the user's pupil may not be necessary. Since control to change the size of the user's pupil is not necessary, the wearable device (101) may refrain from reducing the size of the content displayed on the screen so as not to disturb the user.

[0173] In operation 904, a wearable device (101) according to one embodiment may reduce the size of content displayed on a screen from a first size to a second size. If the pupil size of a user wearing the wearable device (101) is fixed and the current situation is not a specified situation, control to change the user's pupil size may be required. For example, the wearable device (101) may display text and / or an image on a screen displayed through a display (250) to guide the start of control to change the user's pupil size. In one example, the text may be "No change in focus for a long time." However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) may display text on a screen to guide control to change the user's pupil size. In one example, the text may be "The screen will be temporarily reduced to change the pupil's focus." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) may initiate control to change the user's pupil size after text and / or an image is displayed. For example, the wearable device (101) may reduce the size of the content displayed on the screen. While the size of the content is being reduced, the wearable device (101) may monitor the user's pupil size using a camera (260). Based on the monitoring, the wearable device (101) may identify that the amount of change in the user's pupil size corresponds to a specified value (e.g., 2 mm). The wearable device (101) may stop reducing the size of the content upon identifying that the amount of change in the user's pupil size corresponds to a specified value. The size of the content reduced from the first size may be a second size.

[0174] In operation 905, a wearable device (101) according to one embodiment may display content based on a second size for a specified time interval (e.g., 10 seconds). By displaying content based on a second size reduced from a first size for a specified time interval, a change in the user's pupil size may be caused.

[0175] In one embodiment, the wearable device (101) can generate content (or images) for a space (or area) other than the content reduced to a second size using an artificial intelligence model. The artificial intelligence model used to generate the content may be a generative artificial intelligence model. For example, the generative artificial intelligence model may generate content for a space (or area) other than the content reduced to a second size based on the content reduced to a second size. The wearable device (101) may display the content generated for a space (or area) other than the content reduced to a second size through a display (250).

[0176] In one embodiment, the wearable device (101) may display a progress bar corresponding to a specified time interval on the screen. For example, the progress bar may be displayed along the edges of the screen. For example, the length of the progress bar displayed on the screen may indicate the progress of actions to change the user's pupil size.

[0177] In operation 906, a wearable device (101) according to one embodiment can increase the size of content displayed on a screen from a second size to a first size after a specified time interval.

[0178] In operation 907, the wearable device (101) according to one embodiment may refrain from reducing the size of the content displayed on the screen. If the user's pupil size is not fixed during a time interval (e.g., 30 seconds), control to change the user's pupil size may not be necessary. Since control to change the user's pupil size is not necessary, the wearable device (101) may refrain from reducing the size of the content displayed on the screen so as not to disturb the user.

[0179] FIG. 10a illustrates examples of screens designed to induce a change in the user's pupil size.

[0180] In the example illustrated in FIG. 10a, the wearable device (101) can identify that the size of the pupil (111) of the user wearing the wearable device (101) has been fixed for a time interval (e.g., 30 seconds). In the example illustrated in FIG. 10a, the wearable device (101) can identify that the current situation is not a designated situation to avoid interfering with the user using the wearable device (101). The wearable device (101) can control the display (250) to display a screen to induce a change in the size of the user's pupil (111) based on the identification that the size of the user's pupil (111) has been fixed for a time interval (e.g., 30 seconds) and that the current situation is not a designated situation.

[0181] Referring to FIG. 10a, for example, at a first time (1010), the wearable device (101) may display objects (1011) on a screen to guide the start of control for changing the size of the pupil (111). For example, the objects (1011) may include images and text to notify the user of the start of control for changing the size of the pupil (111). In one example, the text may be "No change in focus for a long time." However, this is merely an example and the present disclosure is not limited thereto.

[0182] Referring to FIG. 10a, for example, at a second time (1020), the wearable device (101) may display text (1021) on the screen to guide control for changing the size of the pupil (111). In one example, the text (1021) may be, "The screen will be briefly reduced for a change in pupil focus." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) may display a progress bar (1022) on the screen indicating the progress of the wearable device (101)'s actions for changing the size of the pupil (111).

[0183] Referring to FIG. 10a, for example, at a third time (1030), the wearable device (101) may reduce the size of the content displayed on the screen. While the size of the content is being reduced, the wearable device (101) may monitor the size of the pupil (111) using a camera (260). Based on the monitoring, the wearable device (101) may identify that the amount of change in the size of the pupil (111) corresponds to a specified value (e.g., 2 mm). The wearable device (101) may stop reducing the size of the content upon identifying that the amount of change in the size of the pupil (111) corresponds to a specified value. The size of the content reduced from the first size may be a second size. The wearable device (101) may display the content based on the second size for a specified time interval (e.g., 10 seconds). By displaying content based on a second size reduced from a first size during a specified time interval, a change in the size of the pupil (111) may be caused. The wearable device (101) may display a progress bar (1031) on the screen indicating the progress of the actions of the wearable device (101) to change the size of the pupil (111).

[0184] Referring to FIG. 10a, for example, at the fourth time (1040), after a specified time interval, the size of the content displayed on the screen can be increased from the second size to the first size.

[0185] FIG. 10b illustrates examples of screens designed to induce a change in the user's pupil size.

[0186] In the example illustrated in FIG. 10b, the wearable device (101) can identify that the pupil size of the user wearing the wearable device (101) has been fixed for a time interval (e.g., 30 seconds). In the example illustrated in FIG. 10b, the wearable device (101) can identify that the current situation is not a designated situation to avoid interfering with the user using the wearable device (101). The wearable device (101) can control the display (250) to display a screen to induce a change in the user's pupil size based on the identification that the user's pupil size has been fixed for a time interval (e.g., 30 seconds) and the current situation is not a designated situation.

[0187] Referring to FIG. 10b, for example, at a first time (1050), the wearable device (101) may display objects (1051) on the screen to guide the start of control for changing pupil size. For example, the objects (1051) may include images and text to notify the user of the start of control for changing pupil size. In one example, the text may be 'no change in focus for a long time'. However, this is merely an example and the present disclosure is not limited thereto.

[0188] Referring to FIG. 10b, for example, at the second time (1060), the wearable device (101) may display text (1061) on the screen to guide control for changing the pupil size. In one example, the text (1061) may be, "The screen will be briefly reduced for a change in pupil focus." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) may display a progress bar (1062) on the screen indicating the progress of the wearable device (101)'s actions for changing the pupil size.

[0189] Referring to FIG. 10b, for example, at a third time (1070), the wearable device (101) may reduce the size of the content displayed on the screen. While the size of the content is being reduced, the wearable device (101) may monitor the pupil size using a camera (260). Based on the monitoring, the wearable device (101) may identify that the amount of change in pupil size corresponds to a specified value (e.g., 2 mm). The wearable device (101) may stop reducing the size of the content upon identifying that the amount of change in pupil size corresponds to a specified value. The size of the content reduced from the first size may be the second size. The wearable device (101) may display a progress bar (1071) on the screen indicating the progress of the wearable device (101)'s actions to change the pupil size.

[0190] Referring to FIG. 10b, for example, at the fourth time (1080), the wearable device (101) can generate content (or image) for a space (1082) (or area) outside the content (1081) reduced to a second size using an artificial intelligence model. The artificial intelligence model used to generate the content may be a generative artificial intelligence model. For example, the generative artificial intelligence model may generate content for a space (1082) (or area) outside the content (1081) based on the content (1081) reduced to a second size. The wearable device (101) may display a progress bar (1083) on the screen indicating the progress of the actions of the wearable device (101) to change the pupil size.

[0191] Referring to FIG. 10b, for example, at the fifth time (1090), the wearable device (101) may display content (1091) reduced to a second size and content (1092) for the space outside the content (1091) together on the screen. The wearable device (101) may display a progress bar (1093) on the screen indicating the progress of the wearable device (101)'s actions for changing the pupil size. Meanwhile, although not illustrated, the wearable device (101) may increase the size of the content (1091) from the second size to the first size after a designated time interval.

[0192] FIG. 11 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 11 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0193] Referring to FIG. 11, in operation 1101, a wearable device (101) according to one embodiment can identify whether the gaze of a user wearing the wearable device (101) is fixed. For example, the wearable device (101) can perform eye tracking using a camera (260) and / or a sensor (420). The wearable device (101) can identify the position of the user's gaze on the screen based on the eye tracking. The wearable device (101) can identify whether the position of the gaze on the screen is within a threshold range within a time interval (e.g., 20 seconds). For example, the wearable device (101) can identify that the user's gaze is fixed based on the identification that the position of the gaze on the screen is within a threshold range within the time interval. In one example, the wearable device (101) can identify that the user's gaze is fixed when the distance between the position on the screen at the start of the gaze tracking and the position of the gaze within the time interval is within a threshold range. In another example, the wearable device (101) can identify that the user's gaze is not fixed when the position on the screen of the gaze within the time interval is outside the threshold range.

[0194] In operation 1102, in one embodiment, another wearable device (101) may display points to guide the user's gaze on a screen displayed through a display (250). When the user's gaze is fixed, control to guide the user's eye movement may be required. For example, the wearable device (101) may display text and / or an image on a screen displayed through the display (250) to guide the start of control to guide eye movement. In one example, the text may be "No eye movement for a long time." However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) may display a first point to guide the user's gaze on a screen displayed through the display (250). The wearable device (101) may display text on the screen to guide the user's gaze to the first point. In one example, the text to guide the user's gaze may be "Move your eyes to follow the dot. It disappears upon success." However, this is merely an example, and the present disclosure is not limited thereto. The wearable device (101) may perform eye tracking using a camera (260) and / or a sensor (420) while text is displayed to guide the user's gaze to a first point. The wearable device (101) may identify, based on eye tracking, whether the position of the gaze on the screen corresponds to the position of the first point. The wearable device (101) may display a second point on the screen to guide the user's gaze based on the identification that the position of the gaze on the screen corresponds to the position of the first point. The wearable device (101) may display text on the screen to guide the user's gaze to the second point. The wearable device (101) may identify, based on eye tracking performed while the text is displayed, whether the position of the gaze on the screen corresponds to the position of the second point.The wearable device (101) may terminate the procedure for inducing the user's eye movement upon identifying that the position of the gaze on the screen corresponds to the position of the second point. In the description above, an example in which points including the first point and the second point are displayed on the screen has been described, but the present disclosure is not limited thereto. For example, the number of points may be one or three or more.

[0195] In operation 1103, the wearable device (101) according to one embodiment may refrain from displaying points to guide the user's gaze on a screen displayed through a display (250). If the user's gaze is not fixed during a time interval, control to guide the user's eye movement may not be necessary. Since control to guide the user's eye movement is not necessary, the wearable device (101) may refrain from displaying points to guide the user's gaze on the screen so as not to disturb the user.

[0196] FIG. 12 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 12 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0197] Referring to FIG. 12, in operation 1201, a wearable device (101) according to one embodiment can identify whether the gaze of a user wearing the wearable device (101) is fixed. For example, the wearable device (101) can perform eye tracking using a camera (260) and / or a sensor (420). The wearable device (101) can identify the position of the user's gaze on the screen based on the eye tracking. The wearable device (101) can identify whether the position of the gaze on the screen is within a threshold range within a time interval (e.g., 20 seconds). For example, the wearable device (101) can identify that the user's gaze is fixed based on the identification that the position of the gaze on the screen is within a threshold range within the time interval. In one example, the wearable device (101) can identify that the user's gaze is fixed when the distance between the position on the screen at the start of the gaze tracking and the position of the gaze within the time interval is within a threshold range. In another example, the wearable device (101) can identify that the user's gaze is not fixed when the position on the screen of the gaze within the time interval is outside the threshold range.

[0198] In operation 1202, a wearable device (101) according to one embodiment can identify whether the current situation corresponds to a specified situation. For example, the wearable device (101) can identify whether the current situation corresponds to a specified situation based on the identification that the user's gaze is fixed. A specified situation may mean a situation defined to avoid interfering with the user using the wearable device (101). In one example, a specified situation may be referred to as an exception situation, an exception handling situation, a situation requiring exception handling, or other terms having an equivalent technical meaning.

[0199] In one embodiment, whether the current situation corresponds to a specified situation can be identified based on whether an object on a screen displayed through a display (250) is moving, whether a user wearing the wearable device (101) is moving, and whether user input is obtained. For example, the wearable device (101) can identify whether the distance of movement of an object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the object is moving based on the identification that the distance of movement of an object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the object is moving. For example, the wearable device (101) can identify whether a user wearing the wearable device (101) is moving by using a sensor (420) (e.g., an IMU (inertial measurement unit) sensor). The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the user is moving. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that user input is obtained.

[0200] In operation 1203, the wearable device (101) according to one embodiment may refrain from displaying points to guide the user's gaze on a screen displayed through a display (250). If the current situation is a designated situation to avoid disturbing the user using the wearable device (101), control to guide the user's eye movements may not be necessary. Since control to guide the user's eye movements is not necessary, the wearable device (101) may refrain from displaying points to guide the user's gaze on a screen to avoid disturbing the user.

[0201] In operation 1204, a wearable device (101) according to one embodiment may display points to guide the user's gaze on a screen displayed through a display (250). When the user's gaze is fixed, control to guide the user's eye movement may be required. For example, the wearable device (101) may display text and / or an image on the screen to guide the start of control to guide eye movement. In one example, the text may be "No eye movement for a long time." However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) may display a first point to guide the user's gaze on the screen. The wearable device (101) may display text on the screen to guide the user's gaze to the first point. In one example, the text to guide the user's gaze to the first point may be "Move your eyes to follow the dot. It disappears upon success." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) can perform eye tracking using a camera (260) and / or a sensor (420) while text is displayed to guide the user's gaze to a first point. The wearable device (101) can identify, based on the eye tracking, whether the position of the gaze on the screen corresponds to the position of the first point. The wearable device (101) can display a second point on the screen to guide the user's gaze based on the identification that the position of the gaze on the screen corresponds to the position of the first point. The wearable device (101) can display text on the screen to guide the user's gaze to the second point. The wearable device (101) can identify, based on the eye tracking performed while the text is displayed, whether the position of the gaze on the screen corresponds to the position of the second point.The wearable device (101) may terminate the procedure for inducing the user's eye movement upon identifying that the position of the gaze on the screen corresponds to the position of the second point. In the description above, an example in which points including the first point and the second point are displayed on the screen has been described, but the present disclosure is not limited thereto. For example, the number of points may be one or three or more.

[0202] In operation 1205, the wearable device (101) according to one embodiment may refrain from displaying points to guide the user's gaze on a screen displayed through a display (250). If the user's gaze is not fixed during a time interval or if the current situation is a designated situation to avoid interfering with the user using the wearable device (101), control to guide the user's eye movement may not be necessary. Since control to guide the user's eye movement is not necessary, the wearable device (101) may refrain from displaying points to guide the user's gaze on a screen to avoid interfering with the user.

[0203] Figure 13 illustrates examples of screens designed to induce eye movements of a user.

[0204] In the example illustrated in FIG. 13, the wearable device (101) can identify that the gaze of a user wearing the wearable device (101) has been fixed for a time interval (e.g., 20 seconds). In the example illustrated in FIG. 13, the wearable device (101) can identify that the current situation is not a designated situation to avoid interfering with the user using the wearable device (101). The wearable device (101) can control the display (250) to display a screen to induce the user's eye movement based on the identification that the user's gaze has been fixed for a time interval (e.g., 20 seconds) and the current situation is not a designated situation.

[0205] Referring to FIG. 13, for example, at a first time (1310), the wearable device (101) may display objects (1311) on the screen to guide the start of control to induce eye movement of the user. For example, the objects (1311) may include text and / or images. In one example, the text may be 'no eye movement for a long time'. However, this is merely an example and the present disclosure is not limited thereto.

[0206] Referring to FIG. 13, for example, at a second time (1320), the wearable device (101) may display a first point (1321) on the screen to guide the user's gaze. The wearable device (101) may display text (1322) on the screen to guide the user's gaze to the first point (1321). In one example, the text to guide the user's gaze to the first point may be, "Move your eyes to follow the dot. It will disappear upon success." However, this is merely an example and the present disclosure is not limited thereto. While the text (1322) to guide the user's gaze to the first point (1321) is displayed, the wearable device (101) may perform gaze tracking using a camera (260) and / or a sensor (420). The wearable device (101) can identify, based on eye tracking, whether the position of the gaze on the screen corresponds to the position of the first point (1321). The wearable device (101) can display a second point (1331) on the screen to guide the user's gaze, based on the identification that the position of the gaze on the screen corresponds to the position of the first point (1321). The wearable device (101) can display a progress bar (1323) on the screen indicating the progress of the movements of the wearable device (101) to guide the user's eye movements.

[0207] Referring to FIG. 13, for example, at a third time (1330), the wearable device (101) may display a second point (1331) on the screen to guide the user's gaze. The wearable device (101) may display text (1332) on the screen to guide the user's gaze to the second point (1332). In one example, the text to guide the user's gaze to the second point may be, "Move your eyes to follow the dot. It will disappear upon success." However, this is merely an example and the present disclosure is not limited thereto. While the text (1333) to guide the user's gaze to the second point (1332) is displayed, the wearable device (101) may perform eye tracking using a camera (260) and / or a sensor (420). The wearable device (101) can identify, based on eye tracking, whether the position of the gaze on the screen corresponds to the position of the second point (1331). The wearable device (101) can terminate the procedure for inducing the user's eye movement upon identifying that the position of the gaze on the screen corresponds to the position of the second point (1331). In the example illustrated in FIG. 13, an example in which points including the first point (1321) and the second point (1331) are displayed on the screen has been described, but the present disclosure is not limited thereto. For example, the number of points may be one or three or more. The wearable device (101) may display a progress bar (1333) on the screen indicating the progress of the movements of the wearable device (101) for inducing the user's eye movement.

[0208] FIG. 14 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 14 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0209] Referring to FIG. 14, in operation 1401, the wearable device (101) can identify whether the user's eyes are bloodshot while displaying the screen based on a first brightness. For example, a camera (260) may be used to identify whether the user's eyes are bloodshot. The camera (260) may be a camera positioned to photograph the eyes of a user wearing the wearable device (101). The wearable device (101) can acquire an image of the user's eyes using the camera (260). The wearable device (101) can identify whether the user's eyes are bloodshot based on the image of the user's eyes. In one example, the wearable device (101) can identify whether the user's eyes are bloodshot based on the color of the image of the user's eyes (e.g., RGB (red green blue) values ​​and / or HSV (hue saturation brightness).

[0210] In operation 1402, the wearable device (101) may reduce the screen brightness from a first brightness to a second brightness. If the eyes of a user wearing the wearable device (101) are bloodshot, control may be required to reduce the user's eye fatigue. For example, the wearable device (101) may display text and / or an image on a screen displayed via the display (250) to guide the start of control to reduce the user's eye fatigue upon identification that the user's eyes are bloodshot. In one example, the text may be, 'Eyes are bloodshot. Starting screen adjustment for relief.' However, this is merely an example and the present disclosure is not limited thereto. After the text and / or image is displayed, the wearable device (101) may start control to reduce the user's eye fatigue. For example, the wearable device (101) may reduce the brightness of the screen displayed via the display (250) from a first brightness to a second brightness. The wearable device (101) can display a screen based on a second brightness for a specified time interval (e.g., 20 seconds, 1 minute). By displaying the screen based on a second brightness lower than the first brightness for the specified time interval, the user's eye strain can be reduced. While displaying the screen based on the second brightness, the wearable device (101) can display text on the screen to guide control. In one example, the text may be, "Adjusting the brightness of the screen for a while." However, this is merely an example, and the present disclosure is not limited thereto. After the specified time interval, the wearable device (101) can increase the brightness of the screen displayed through the display (250) from the second brightness to the first brightness.

[0211] In operation 1403, the wearable device (101) may refrain from reducing screen brightness. If the eyes of the user wearing the wearable device (101) are not bloodshot, control to reduce the user's eye fatigue may not be necessary. For example, the wearable device (101) may refrain from reducing the brightness of the screen displayed through the display (250) upon identifying that the user's eyes are not bloodshot.

[0212] Figure 15 illustrates examples of screens designed to reduce eye strain in users.

[0213] In the example illustrated in FIG. 15, the wearable device (101) can identify that the eyes of a user wearing the wearable device (101) are bloodshot. The wearable device (101) can control the display (250) to display a screen to reduce the user's eye fatigue based on the identification that the user's eyes are bloodshot.

[0214] Referring to FIG. 15, for example, at a first time (1510), the wearable device (101) may display objects (1511) on a screen to guide the start of a control to reduce the user's eye fatigue. For example, the objects (1511) may include images and text to notify the user of the start of a control to reduce the user's eye fatigue. In one example, the text may be, "Your eyes are bloodshot. Starting screen adjustment for relief." However, this is merely an example and the present disclosure is not limited thereto.

[0215] Referring to FIG. 15, for example, at a second time (1520), the wearable device (101) may reduce the brightness of the screen displayed through the display (250) from a first brightness to a second brightness. The wearable device (101) may display the screen based on the second brightness for a specified time interval (e.g., 20 seconds, 1 minute). While displaying the screen based on the second brightness, the wearable device (101) may display text on the screen to guide control. In one example, the text may be, "Adjusting the brightness of the screen for a while." However, this is merely an example and the present disclosure is not limited thereto. The wearable device (101) may display a progress bar (1522) on the screen indicating the progress of the wearable device (101)'s actions to reduce the user's eye fatigue.

[0216] Referring to FIG. 15, for example, at a third time (1530) after a specified time interval, the wearable device (101) can increase the brightness of the screen displayed through the display (250) from a second brightness to a first brightness.

[0217] FIG. 16 is a flowchart illustrating the operations of a wearable device for controlling a screen displayed through a display. The operations of FIG. 16 may be performed by the electronic device (101) of FIG. 1 or the wearable device (101) of FIG. 2a through FIG. 4. For example, at least some of the operations may be controlled by a processor (410) of the wearable device (101). In the following, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed. For example, at least two operations may be performed in parallel.

[0218] Referring to FIG. 16, a wearable device (101) according to one embodiment can identify whether a trigger condition for eye health of a user wearing the wearable device (101) is detected.

[0219] In one embodiment, the trigger condition may include first to fourth conditions. For example, the wearable device (101) may detect the first condition when the user's eyes do not blink for a time interval (e.g., 1 minute). The details for detecting the first condition may be substantially the same as those described in FIGS. 5 to 7. For example, the wearable device (101) may detect the second condition when the amount of change in the user's pupil size is less than a threshold value within a time interval (e.g., 30 seconds). The details for detecting the second condition may be substantially the same as those described in FIGS. 8 to 10b. For example, the wearable device (101) may detect the third condition when the position of the user's gaze on the screen is within a threshold range within a time interval (e.g., 20 seconds). The details for detecting the third condition may be substantially the same as those described in FIGS. 11 to 13. For example, the wearable device (101) can detect a fourth condition when the user's eyes are bloodshot. The details for detecting the fourth condition may be substantially the same as those described in FIGS. 14 and 15.

[0220] In one embodiment, the wearable device (101) can detect trigger conditions based on priority. For example, the wearable device (101) can detect trigger conditions based on the priority of the first to fourth conditions. In one example, priority may be assigned in the order of the first to fourth conditions. The wearable device (101) can identify whether the first condition is detected. Based on the identification that the first condition is not detected, the wearable device (101) can identify whether the second condition is detected. Based on the identification that the second condition is not detected, the wearable device (101) can identify whether the third condition is detected. Based on the identification that the third condition is not detected, the wearable device (101) can identify whether the fourth condition is detected. However, this is merely an example and the present disclosure is not limited thereto. For example, the priority of the first through fourth conditions may be assigned in an order different from the order described above.

[0221] In operation 1602, a wearable device (101) according to one embodiment can identify whether the current situation corresponds to a specified situation. For example, the wearable device (101) can identify whether the current situation corresponds to a specified situation based on the identification that a trigger condition for the user's eye health is detected. A specified situation may mean a situation defined to avoid interfering with the user using the wearable device (101). In one example, a specified situation may be referred to as an exception situation, an exception handling situation, an exception handling required situation, or other terms having an equivalent technical meaning.

[0222] In one embodiment, whether the current situation corresponds to a specified situation may be identified based on the type of object displayed through the display (250), the movement of the user's eyes identified using the camera (260), the type of application executed by the wearable device (101), whether the object on the screen displayed through the display (250) is moving, whether the user wearing the wearable device (101) is moving, and / or whether user input is obtained. For example, if the object displayed through the display (250) is text, the wearable device (101) may identify that the current situation corresponds to a specified situation. In one example, if text is displayed on the screen displayed through the display (250), the wearable device (101) may identify that the current situation corresponds to a specified situation. However, this is merely an example and the present disclosure is not limited thereto. For example, the wearable device (101) can identify that the movement of the user's eyes, identified using the camera (260), tracks the movement of an object on the screen. The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the movement of the user's eyes tracks the movement of an object on the screen. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that a specified application (e.g., a game application, a video application) is running. For example, the wearable device (101) can identify whether the distance of movement of an object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the object is moving based on the identification that the distance of movement of the object on the screen is greater than or equal to a threshold value. The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the object is moving.For example, the wearable device (101) can identify whether the user wearing the wearable device (101) is moving by using a sensor (420) (e.g., an inertial measurement unit (IMU) sensor). The wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that the user is moving. For example, the wearable device (101) can identify that the current situation corresponds to a specified situation based on the identification that user input is obtained.

[0223] In operation 1603, a wearable device (101) according to one embodiment may wait until the next cycle to detect a trigger condition for the user's eye health. For example, the wearable device (101) may wait until the next cycle to detect a trigger condition based on the identification that the current situation corresponds to a specified situation. In the next cycle, the wearable device (101) may perform the operations illustrated in FIG. 16.

[0224] In operation 1604, a wearable device (101) according to one embodiment can perform actions for the user's eye health. For example, the wearable device (101) can perform actions corresponding to a detected trigger condition based on the identification that the current situation does not correspond to a specified situation.

[0225] In one embodiment, the detected trigger condition may be a first condition. The wearable device (101) may control the display (250) to display a screen to induce the user to blink their eyes upon identifying that the first condition is detected. The content for inducing the user to blink their eyes may be substantially the same as that described in FIGS. 5 through 7.

[0226] In one embodiment, the detected trigger condition may be a second condition. The wearable device (101) may control the display (250) to display a screen for changing the user's pupil size upon identification that the second condition is detected. The content for changing the user's pupil size may be substantially the same as that described in FIGS. 8 to 10b.

[0227] In one embodiment, the detected trigger condition may be a third condition. The wearable device (101) may control the display (250) to display a screen to induce eye movement of the user upon identification that the third condition is detected. The content for inducing eye movement of the user may be substantially the same as that described in FIGS. 11 to 13.

[0228] In one embodiment, the detected trigger condition may be a fourth condition. The wearable device (101) may control the display (250) to display a screen for reducing the user's eye fatigue upon identification that the fourth condition is detected. The content for reducing the user's eye fatigue may be substantially the same as that described in FIGS. 14 and 15.

[0229] In operation 1605, the wearable device (101) according to one embodiment may wait until the next cycle to detect a trigger condition for the user's eye health. For example, the wearable device (101) may wait until the next cycle to detect a trigger condition, depending on the identification that no trigger condition is detected or that the current situation is a designated situation to prevent interference with the user using the wearable device (101). In the next cycle, the wearable device (101) may perform the operations illustrated in FIG. 16.

[0230] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs.

[0231] The wearable device (101) described above may include at least one camera (260). The wearable device (101) may include a display (250). The wearable device (101) may include a memory (415) that stores instructions and includes one or more storage media. The wearable device (101) may include at least one processor (410) that includes a processing circuit. When the instructions are executed individually or collectively by the at least one processor (410), the wearable device (101) may cause the wearable device (101) to identify whether the eyes of a user wearing the wearable device (101) are blinking within a specified time interval using the at least one camera (260). When the above instructions are executed individually or collectively by the at least one processor (410), the wearable device (101) may be caused to display a first space between the first line and the top boundary based on a second brightness lower than the first brightness while a first line is displayed that is caused to move from the top boundary of the screen displayed through the display (250) toward the bottom boundary, depending on the identification that the user's eye does not blink within the specified time interval. When the above instructions are executed individually or collectively by the at least one processor (410), the wearable device (101) may be caused to display a second space between the second line and the bottom boundary based on the second brightness lower than the first brightness while a second line is displayed that is caused to move from the bottom boundary of the screen displayed through the display (250) toward the top boundary, depending on the identification that the user's eye does not blink within the specified time interval.When the above instructions are executed individually or collectively by the at least one processor (410), the wearable device (101) may cause the display screen shown through the display (250) to be displayed based on the first brightness according to the identification that the first line and the second line are in contact.

[0232] For example, when the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the screen displayed through the display to be displayed based on a blur effect while the first line and the second line are moving. For example, when the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the third space between the first line and the second line to be displayed based on a first brightness.

[0233] For example, the wearable device may further include at least one sensor. When the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the wearable device to determine the first brightness based on the brightness outside the wearable device identified by the at least one sensor.

[0234] For example, when the instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to identify whether the current situation corresponds to a specified situation based on the type of object displayed through the display, the movement of the eye identified using the at least one camera, the type of application executed by the wearable device, and whether user input is obtained. When the instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to refrain from displaying the first space and the second space based on the second brightness lower than the first brightness, depending on the identification that the current situation corresponds to the specified situation.

[0235] For example, the first line and the second line may be displayed during a first designated time interval. When the instructions are executed individually or collectively by the at least one processor, the wearable device may cause a progress bar for the first designated time interval to be displayed on at least one part of the screen displayed through the display.

[0236] For example, when the instructions are executed individually or collectively by the at least one processor, the wearable device may use the at least one camera to identify whether the amount of change in the pupil size of the eye is less than a threshold value within a second designated time interval. When the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the size of the content displayed on the screen to be reduced based on the identification that the amount of change in the pupil size is less than the threshold value. When the instructions are executed individually or collectively by the at least one processor, the wearable device may cause the content to be displayed based on the second size during a third designated time interval based on the identification that the amount of change in the pupil size corresponds to a specified value while the size of the content is reduced from a first size to a second size. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause the content to be displayed based on the first size after the third specified time interval.

[0237] For example, when the above instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to generate content for a space excluding the content in the screen using an artificial intelligence model based on the content reduced to the second size. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to display the generated content through the display.

[0238] For example, when the instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to identify whether the current situation corresponds to a specified situation based on whether at least one object within the screen displayed through the display is moving, whether the user wearing the wearable device is moving, and whether user input is obtained. When the instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to refrain from reducing the size of the content displayed on the screen based on the identification that the current situation corresponds to the specified situation.

[0239] For example, when the above instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to identify whether the position on the screen of the user’s gaze, identified using the at least one camera, is located within a threshold range during a fourth designated time interval. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to display a first point for guiding the user’s gaze at a first position on the screen upon identification that the position on the screen of the user’s gaze is located within the threshold range during the fourth designated time interval. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may be caused to display a second point for guiding the user’s gaze at a second position on the screen upon identification that the position on the screen of the user’s gaze is located at the first point.

[0240] For example, when the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause the user to identify whether the user’s eye is bloodshot using the at least one camera. When the above instructions are executed individually or collectively by the at least one processor, the wearable device may cause the screen displayed through the display during a fifth designated time interval to be displayed based on a brightness lower than the first brightness, depending on the identification that the user’s eye is bloodshot.

[0241] A method performed by a wearable device comprising at least one sensor, at least one camera, and a display as described above may include an operation of identifying whether the eye of a user wearing the wearable device blinks within a specified time interval using the at least one camera. The method may include an operation of displaying a first space between the first line and the top boundary based on a second brightness lower than a first brightness while a first line is displayed that causes the user's eye to move from the top boundary to the bottom boundary of the screen displayed through the display, based on the identification that the user's eye does not blink within the specified time interval. The method may include an operation of displaying a second space between the second line and the bottom boundary based on a second brightness lower than the first brightness while a second line is displayed that causes the user's eye to move from the bottom boundary to the top boundary of the screen displayed through the display, based on the identification that the user's eye does not blink within the specified time interval. The above method may include an operation of displaying a screen displayed through the display based on the first brightness, according to the identification that the first line and the second line are in contact.

[0242] For example, the above method may include an operation of displaying a screen displayed through the display based on a blur effect while the first line and the second line are moving. The above method may include an operation of displaying a third space between the first line and the second line based on a first brightness.

[0243] For example, the above method may include an operation to determine the first brightness based on the brightness outside the wearable device identified by the at least one sensor.

[0244] For example, the above method may include an operation of identifying whether the current situation corresponds to a specified situation based on the type of object displayed through the display, the movement of the eye identified using the at least one camera, the type of application executed by the wearable device, and whether user input is obtained. The above method may include an operation of refraining from displaying the first space and the second space based on the second brightness lower than the first brightness, upon identifying that the current situation corresponds to the specified situation.

[0245] For example, the first line and the second line may be displayed during a first designated time interval. The method may include the operation of displaying a progress bar for the first designated time interval on at least one part of the screen displayed through the display.

[0246] For example, the above method may include an operation of identifying whether the amount of change in the pupil size of the eye is less than a threshold value within a second designated time interval using the at least one camera. The above method may include an operation of reducing the size of the content displayed on the screen based on the identification that the amount of change in the pupil size is less than the threshold value. The above method may include an operation of displaying the content based on the second size during a third designated time interval based on the identification that the amount of change in the pupil size corresponds to a specified value while the size of the content is reduced from a first size to a second size. The above method may include an operation of displaying the content based on the first size after the third designated time interval.

[0247] For example, the above method may include an operation of generating content for a space excluding the content within the screen using an artificial intelligence model, based on the content reduced to the second size. The above method may include an operation of displaying the generated content through the display.

[0248] For example, the above method may include an operation of identifying whether the current situation corresponds to a specified situation based on whether at least one object within the screen displayed through the display is moving, whether the user wearing the wearable device is moving, and whether user input is obtained. The above method may include an operation of refraining from reducing the size of the content displayed on the screen based on the identification that the current situation corresponds to the specified situation.

[0249] For example, the method may include an operation of identifying whether the position on the screen of the user’s gaze, identified using at least one camera, is located within a threshold range during a fourth designated time interval. The method may include an operation of displaying a first point to guide the user’s gaze at a first position on the screen, based on the identification that the position on the screen of the user’s gaze is located within the threshold range during the fourth designated time interval. The method may include an operation of displaying a second point to guide the user’s gaze at a second position on the screen, based on the identification that the position on the screen of the user’s gaze is located at the first point.

[0250] For example, the above method may include an operation of identifying whether the user's eye is bloodshot using the at least one camera. The above method may include an operation of displaying the screen displayed through the display during a fifth designated time interval based on a brightness lower than the first brightness, depending on the identification that the user's eye is bloodshot.

[0251] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

[0252] Any of the embodiments described above may be combined with any other embodiment (or combination of embodiments) unless otherwise explicitly stated. The foregoing description of one or more embodiments is for illustrative and explanatory purposes only, and is not intended to limit or exhaust the scope of the embodiments in the exact form disclosed. Modifications and variations are possible in light of the foregoing teachings or may be obtained from the practice of various embodiments.

[0253] For one or more embodiments, at least one of the components described in one or more of the prior art drawings may be configured to perform one or more operations, techniques, processes and / or methods as described in the present disclosure. For example, a processor (e.g., a baseband processor) described in the present disclosure in relation to one or more of the prior art drawings may be configured to operate according to one or more examples described in the present disclosure. As another example, circuits associated with user equipment (UE), a base station, a network element, etc., as described above in relation to one or more of the prior art drawings may be configured to operate according to one or more examples described herein.

[0254] Any of the embodiments described above may be combined with any other embodiment (or combination of embodiments) unless otherwise explicitly stated. The foregoing description of one or more embodiments is for illustrative and explanatory purposes only, and is not intended to limit or exhaust the scope of the embodiments in the exact form disclosed. Modifications and variations are possible in light of the foregoing teachings or may be obtained from the practice of various embodiments.

[0255] The electronic devices according to the various embodiments disclosed in this document may be of various forms. The electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or consumer electronics. The electronic devices according to the embodiments of this document are not limited to the devices described above.

[0256] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0257] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0258] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0259] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0260] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable device, At least one camera; display; Memory for storing instructions and including one or more storage media; and It includes at least one processor comprising a processing circuit, and When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Using at least one camera, identify whether the eye of a user wearing the wearable device blinks within a specified time interval, and Based on the identification that the user's eye does not blink within the specified time interval: While a first line is displayed that causes movement from the top boundary to the bottom boundary of the screen displayed through the above display, a first space between the first line and the top boundary is displayed based on a second brightness lower than a first brightness, and While a second line is displayed that causes movement from the bottom boundary of the screen displayed through the display toward the top boundary, a second space between the second line and the bottom boundary is displayed based on the second brightness which is lower than the first brightness, and Causing the screen displayed through the display to be displayed based on the first brightness, according to the identification that the first line and the second line are in contact. Wearable device.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, While the first line and the second line are moving, the screen displayed through the display is displayed based on a blur effect, and Causing to display the third space between the first line and the second line based on the first brightness, Wearable device.

3. In Paragraph 1, It includes at least one additional sensor, and When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Causing to determine the first brightness based on the brightness outside the wearable device identified by the at least one sensor. Wearable device.

4. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Based on the type of object displayed through the above display, the eye movement identified using the at least one camera, the type of application executed by the wearable device, and whether user input is acquired, identifying whether the current situation corresponds to a specified situation, and Causing to refrain from displaying the first space and the second space based on the second brightness lower than the first brightness, in accordance with the identification that the above current situation corresponds to the above specified situation. Wearable device.

5. In Paragraph 1, The above first line and the above second line are displayed during a first designated time interval, and When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Causing to display a progress bar for the first designated time interval on at least one part of the screen displayed through the display. Wearable device.

6. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Using the above at least one camera, it is determined whether the amount of change in the pupil size of the eye is less than a threshold value within a second designated time interval, and Based on the identification that the change in the above pupil size is less than the above threshold value: Reduce the size of the content displayed on the screen above, and Based on the identification that the amount of change in the pupil size corresponds to a specified value while the size of the content decreases from a first size to a second size, the content is displayed based on the second size during a third specified time interval, and Causing to display the content based on the first size after the third designated time interval above, Wearable device.

7. In Paragraph 6, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Based on the content reduced to the second size mentioned above, content for the space excluding the content within the screen is generated using an artificial intelligence model, and Causing the above-mentioned generated content to be displayed through the above-mentioned display, Wearable device.

8. In Paragraph 6, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Identifying whether the current situation corresponds to a specified situation based on whether at least one object within the screen displayed through the above display is moving, whether the user wearing the wearable device is moving, and whether user input is obtained, and Causing to refrain from reducing the size of the content displayed on the screen, based on the identification that the above current situation corresponds to the above specified situation, Wearable device.

9. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Identifying whether the position on the screen of the user's gaze, identified using at least one camera, is located within a threshold range during a fourth designated time interval, and Based on the identification that the position of the user's gaze on the screen is located within the threshold range during the fourth designated time interval, a first point for guiding the user's gaze is displayed at a first position on the screen, and Causing to display a second point to guide the user's gaze to a second position on the screen, based on the identification that the position of the user's gaze on the screen is located at the first point. Wearable device.

10. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the wearable device, Using the above at least one camera, identify whether the user's eye is bloodshot, and, Causing the screen displayed through the display during a fifth designated time interval to be displayed based on a brightness lower than the first brightness, upon identification that the user's eye is bloodshot. Wearable device.

11. A method performed by a wearable device comprising at least one sensor, at least one camera, and a display, An operation of identifying whether the eye of a user wearing the wearable device blinks within a specified time interval using at least one camera; Based on the identification that the user's eye does not blink within the specified time interval: An operation of displaying a first space between the first line and the top boundary based on a second brightness lower than a first brightness while a first line is displayed that causes movement from the top boundary to the bottom boundary of the screen displayed through the above display; An operation of displaying a second space between the second line and the bottom boundary based on the second brightness lower than the first brightness while a second line is displayed that causes the second line to move from the bottom boundary toward the top boundary of the screen displayed through the display; and Based on the identification that the first line and the second line are in contact, the operation of displaying the screen displayed through the display based on the first brightness, method.

12. In Paragraph 11, An operation of displaying a screen displayed through the display based on a blur effect while the first line and the second line are moving; and Further including the operation of displaying a third space between the first line and the second line based on a first brightness. method.

13. In Paragraph 11, The method further includes the operation of determining the first brightness based on the brightness outside the wearable device identified by the at least one sensor. method.

14. In Paragraph 11, An operation to identify whether the current situation corresponds to a specified situation based on the type of object displayed through the display, the movement of the eye identified using the at least one camera, the type of application executed by the wearable device, and whether user input is acquired; and A further operation of refraining from displaying the first space and the second space based on the second brightness lower than the first brightness, upon identification that the above current situation corresponds to the above specified situation. method.

15. In Paragraph 11, The above first line and the above second line are displayed during a first designated time interval, and The above method is, The method further includes the operation of displaying a progress bar for the first designated time interval on at least one part of the screen displayed through the display. method.

Images