Electronic device and method for providing notification information based on gaze of user

Abstract

This wearable device may comprise: a display; a sensor; a communication circuit; a memory storing instructions; and at least one processor. When executed by the at least one processor, the instructions can cause the wearable device to: identify, on the basis of information obtained through the sensor and the communication circuit, a context, which indicates the situation of a user, and the location of an object, which is related to the context; display, on the basis of identifying that the object is located in a first area within an FoV area of the wearable device, a first visual object overlapping the edge of the object; flicker and display the first visual object on the basis of identifying that the object is located within a second area of the FoV area; and, on the basis of identifying that the object is located outside the FoV area, display, within the FoV area, a second visual object indicating the object.

Description

Electronic device and method for providing notification information based on a user's gaze

[0001] The descriptions below relate to electronic devices and methods for providing notification information based on a user's gaze.

[0002] An electronic device can display computer-generated information. The information can be linked to external objects in the real world (or virtual objects in the virtual world). The electronic device can include a wearable device. For example, the electronic device can include user equipment, AR glasses, VR glasses, and / or a head-mounted device (HMD) (e.g., a video see-through (VST) HMD, an optical see-through (OST) HMD).

[0003] A wearable device may include a display. The wearable device may include a sensor. The wearable device may include a communication circuit. The wearable device may include a memory storing instructions and including one or more storage media. The wearable device may include at least one processor including a processing circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a context representing a situation of a user wearing the wearable device based on information obtained through the sensor and the communication circuit. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to identify a location of an object related to the context based on the information. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display a first visual object that is at least partially superimposed on and overlaps a periphery of the object based on determining that the object is located within a first region corresponding to a gaze direction of the user within a field of view (FoV) region of the wearable device.The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, by blinking, the first visual object at least partially overlapping the object based on determining that the object is located within a second region excluding the first region within the FoV region. The instructions, when individually or collectively executed by the at least one processor, may cause the wearable device to display, at a point within the FoV region, a second visual object representing the object based on determining that the object is located within a third region outside the FoV region.

[0004] A vehicle may include a sensor. The vehicle may include a communication circuit. The vehicle may include a memory storing instructions and including one or more storage media. The vehicle may include at least one processor including a processing circuit. The instructions, when individually or collectively executed on the at least one processor, may cause the vehicle to receive, from a wearable device worn by a user of the vehicle connected through the communication circuit, FoV information of the wearable device and gaze information of the user. The instructions, when individually or collectively executed on the at least one processor, may cause the vehicle to identify, with respect to the gaze, FoV of the wearable device based on the FoV information and the gaze information. The instructions, when individually or collectively executed on the at least one processor, may cause the vehicle to identify a recognition area of ​​the vehicle outside the FoV based on information acquired through the sensors and the communication circuit. The instructions, when individually or collectively executed on the at least one processor, may cause the vehicle to provide a notification for an object related to a context representing a situation of the user within the recognition area and to transmit a signal to the wearable device indicating the display of a visual object to notify of the object.

[0005] FIG. 1 is a block diagram of an electronic device within a network environment according to embodiments of the present disclosure.

[0006] FIG. 2A illustrates an example of a perspective view of a wearable device according to embodiments of the present disclosure.

[0007] FIG. 2b illustrates an example of one or more hardware elements arranged within a wearable device according to embodiments of the present disclosure.

[0008] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to embodiments of the present disclosure.

[0009] Figure 4 illustrates an example of a method for providing notification information based on the location of an object.

[0010] Figure 5 illustrates an exemplary block diagram of a wearable device.

[0011] Figure 6 illustrates an example of an operation flow for a method of identifying an object according to a user's context and providing notification information based on the location of the identified object.

[0012] Figures 7a and 7b illustrate examples of a method for identifying a first region based on a user's gaze.

[0013] Figures 8a and 8b illustrate examples of visual effects provided depending on the location and type of an object.

[0014] Figure 9a illustrates an example of a recognition area identified when a user wearing a wearable device uses a vehicle.

[0015] FIG. 9b illustrates an example of a wearable device that provides notification information based on information provided from a vehicle.

[0016] FIGS. 10A and 10B illustrate examples of a method for providing notification information based on the space in which an external electronic device is located.

[0017] Figure 11 illustrates an example of an operational flow for a method of providing notification information based on the location of an object.

[0018] Figure 12 illustrates an example of an operational flow for providing notification information based on the location of an object and transmitting a signal to an external electronic device.

[0019] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0020] One or more embodiments of the present disclosure described below illustrate a hardware-based approach. However, since one or more embodiments of the present disclosure include techniques utilizing both hardware and software, one or more embodiments of the present disclosure do not exclude a software-based approach.

[0021] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled. However, this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." Conditions described as "more than" may be replaced with "more than," conditions described as "less than," and conditions described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B).

[0022] FIG. 1 is a block diagram of an electronic device within a network environment according to embodiments of the present disclosure.

[0023] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). The electronic device (101) may communicate with the electronic device (104) via the server (108). The electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0024] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). The processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0025] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. The auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). The auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, the artificial intelligence model may include a software structure. The term "processor" in this document may be understood to mean both a single context and multiple contexts.

[0026] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0027] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. The receiver can be implemented separately from the speaker or as part of the speaker.

[0030] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. The display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0031] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. The audio module (170) can acquire sound through the input module (150), or output sound through an audio output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0032] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. The sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) to an external electronic device (e.g., the electronic device (102)). The interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). The connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. The haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0036] The camera module (180) can capture still images and videos. The camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). The power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0038] A battery (189) can power at least one component of the electronic device (101). The battery (189) can include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. The communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). The wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for realizing eMBB, a loss coverage (e.g., 164 dB or less) for realizing mMTC, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for realizing URLLC.

[0041] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). The antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). The antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and the external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0042] According to one or more embodiments, the antenna module (197) may form a mmWave antenna module. The mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. The external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045] FIG. 2A illustrates an example of a perspective view of a wearable device according to embodiments of the present disclosure. FIG. 2B illustrates an example of one or more hardware components arranged within a wearable device according to embodiments of the present disclosure.

[0046] According to one embodiment, the wearable device (103) may have the form of glasses that are wearable on a body part of the user (e.g., the head). The wearable device (103) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (103) may include a head-mounted display (HMD). For example, the housing of the wearable device (103) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (103) may include one or more straps that are capable of being twined around the user's head, and / or one or more temples that are detachably attachable to the ears of the head.

[0047] Referring to FIG. 2A, according to one embodiment, a wearable device (103) may include at least one display (250) and a frame (200) supporting at least one display (250).

[0048] According to one embodiment, a wearable device (103) can be worn on a part of a user's body. The wearable device (103) can provide extended reality (XR) to a user wearing the wearable device (103). For example, the extended reality can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality. For example, the wearable device (103) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.

[0049] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0050] Referring to FIG. 2B, at least one display (250) can provide visual information transmitted from external light to a user through a lens included in at least one display (250), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of at least one display (250). When a user wears the wearable device (103), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0051] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (103) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0052] The wearable device (103) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (103) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (103) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (103) can view an image displayed on at least one display (250).

[0053] According to one embodiment, the frame (200) may be configured as a physical structure that allows the wearable device (103) to be worn on the user's body. The frame (200) may be configured so that, when the user wears the wearable device (103), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0054] Referring to FIG. 2A, the frame (200) may include a region (220) that is at least partially in contact with a part of the user's body when the user wears the wearable device (103). For example, the region (220) of the frame (200) that is in contact with a part of the user's body may include a region that is in contact with a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (103) makes contact with. The frame (200) may include a nose pad (210) that is in contact with a part of the user's body. When the wearable device (103) is worn by the user, the nose pad (210) may be in contact with a part of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another part of the user's body that is distinct from the part of the user's body.

[0055] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). The wearable device (103) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of the surface of the frame (200).

[0056] The wearable device (103) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 5). For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).

[0057] The microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (103) can be arranged on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) arranged on the bridge (203), a second microphone (265-2) arranged on the second rim (202), and a third microphone (265-3) arranged on the first rim (201) are illustrated in FIG. 2B, but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B. When the number of microphones (265) included in the wearable device (103) is two or more, the wearable device (103) can identify the direction of the sound signal by using a plurality of microphones arranged on different portions of the frame (200).

[0058] According to one embodiment, at least one optical device (282, 284) may project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) may be a projector. At least one optical device (282, 284) may be disposed adjacent to at least one display (250) or may be included within at least one display (250) as a part of at least one display (250). The wearable device (103) may include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0059] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (103). For example, the wearable device (103) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (103) can perform gaze interaction with at least one object using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can express a portion corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a first region related to the gaze and a second region distinguished from the first region within the image may be different from each other. For example, if the wearable device (103) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1).An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or may be positioned toward both eyes.

[0060] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image in order to implement augmented reality or mixed reality content. The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including an image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (103) can compensate for depth information (e.g., the distance between the wearable device (103) and an external object acquired using a depth sensor) using the image acquired using the capturing camera (260-4). The wearable device (103) can perform object recognition using the image acquired using the capturing camera (260-4). The wearable device (103) may perform a pass-through function to display an image acquired through a photographing camera (260-4) by overlaying it on at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) may be positioned on a bridge (203) positioned between a first rim (201) and a second rim (202).

[0061] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (103) and thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (103) looks straight ahead, the wearable device (103) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (103) is positioned.

[0062] The motion recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The motion recognition camera (260-2, 260-3) can recognize the user's motion (gesture recognition), obtain a signal corresponding to the motion, and provide a display corresponding to the signal on at least one display (250). The wearable device (103) can identify the signal corresponding to the motion, and perform a designated function based on the identification. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The wearable device (103) can perform gesture recognition and / or object tracking functions using motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) can be positioned on the first rim (201) and / or the second rim (202).

[0063] The camera (260) included in the wearable device (103) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (103) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (103) can identify an external object based on a sensor for identifying the distance between the wearable device (103) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (103) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (103).

[0064] Although not shown, in one embodiment, the wearable device (103) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).

[0065] The battery module (270) can supply power to electronic components of the wearable device (103). In one embodiment, the battery module (270) can be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) can be a plurality of battery modules (270). The plurality of battery modules (270) can be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) can be disposed at an end of the first temple (204) and / or the second temple (205).

[0066] The antenna module (275) can transmit signals or power to the outside of the wearable device (103), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0067] The speaker (255) can output an audio signal to the outside of the wearable device (103). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (103). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus positioned adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus positioned adjacent to the user's right ear.

[0068] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state, in order to visually provide information regarding a specific state of the wearable device (103) to the user. For example, when the wearable device (103) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0069] Referring to FIG. 2B, according to one embodiment, a wearable device (103) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (103) (e.g., hardwares illustrated by different blocks in FIG. 5) may be disposed on the PCB (290). The wearable device (103) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0070] According to one embodiment, the wearable device (103) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (103) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (103). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (103) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (103) based on the IMU.

[0071] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to embodiments of the present disclosure.

[0072] The wearable device (103) of FIGS. 3A and 3B may be an example of the electronic device (101) of FIG. 1. According to one embodiment, an example of the appearance of a first side (310) of a housing of the wearable device (103) may be illustrated in FIG. 3A, and an example of the appearance of a second side (320) opposite to the first side (310) may be illustrated in FIG. 3B.

[0073] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (103) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (103) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (103) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0074] The wearable device (103) may include cameras (260-1) for photographing and / or tracking the user's two eyes adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the eye tracking camera (260-1) of FIG. 2B. The wearable device (103) may include cameras (260-5, 260-6) for photographing and / or recognizing the user's face. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (103) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6).

[0075] Referring to FIG. 3b, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor (330)) for obtaining information related to the external environment of the wearable device (103) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3a. For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0076] For example, using cameras (260-11, 260-12), the wearable device (103) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0077] The wearable device (103) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (103) and an external object. Using the depth sensor (330), the wearable device (103) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (103). Although not illustrated, a microphone may be disposed on the second face (320) of the wearable device (103) to obtain sound output from an external object. The number of microphones may be one or more depending on the embodiment.

[0078] A wearable device according to one or more embodiments of the present disclosure may provide VR or AR. In one or more embodiments, the wearable device may enhance a user's awareness of the surrounding environment.

[0079] Figure 4 illustrates an example of a method for providing notification information based on the location of an object.

[0080] FIG. 4 illustrates an example (400) of a user (490) wearing a wearable device (103) driving a vehicle. For example, the wearable device (103) of FIG. 4 may represent an example of the electronic device (101) of FIG. 1 or the wearable device (103) of FIGS. 2A to 3B.

[0081] According to one or more embodiments, the wearable device (103) may determine a context associated with the user, such as an action in which the user is engaged. The wearable device (103) may identify one or more objects associated with the action in which the user is engaged. Different visual effects may be used based on whether the objects are located within a region corresponding to the user's line of sight within the FoV (a first region), within the FoV but outside the user's line of sight (a second region), or outside the FoV (a third region).

[0082] A user (490) can drive while wearing a wearable device (103). The wearable device (103) can identify a context representing a situation of the user (490) based on information acquired through sensors and communication circuits included in the wearable device (103). For example, the context can define an action or situation of the user (490). For example, the context can be referred to as a routine, situational information, an action in which the user participates, or a behavior pattern. In the example of FIG. 4, the context of the user (490) can be driving. However, the embodiments of the present disclosure are not limited thereto. For example, the context of the user (490) can be driving in a section with a crosswalk and a traffic light. In other words, the context can be defined so that the wearable device (103) can interpret the actions or situations of the user (490) and provide the user (490) with necessary information or services.

[0083] The wearable device (103) can identify an object related to the context. The object related to the context can be referred to as an alarm object, an object of interest, a point of interest (POI), a visual element related to the context, a visual object related to the context, a graphical object related to the context, or a target object. The object can be an object considered to be related to the action performed by the user. For example, when the user is driving, the objects can be other vehicles or pedestrians. For example, the wearable device (103) can identify an object around the wearable device (103) based on the information acquired through the sensor and the communication circuit. For example, the wearable device (103) can identify an object related to the context among the objects around the wearable device (103) based on the information and the identified context. For example, the wearable device (103) can identify a first object (440-1), which is a vehicle driving in front of the user's (490) vehicle, a second object (440-2), which is a traffic light located in the section of the user's (490) vehicle, and a third object (440-3), which is a pedestrian trying to cross a crosswalk in the section. FIG. 4 illustrates a case where the user (490) actually drives while wearing the wearable device (103), and the first object (440-1), the second object (440-2), and the third object (440-3) may be real objects. However, the embodiments of the present disclosure are not limited thereto. For example, when the wearable device (103) provides VR, the objects may include rendered images rather than real objects.In the following, for convenience of explanation, the description focuses on real objects, but the objects of the embodiments of the present disclosure may include the rendered images provided in VR.

[0084] The wearable device (103) can provide a visual effect based on the location of an object. The visual effect can be used to provide a notification (or alarm) regarding the object related to the context. For example, the visual effect can be referred to as an alarm, alarm information, notification, or notification information.

[0085] For example, the wearable device (103) may provide a first visual effect (451) to a first object (440-1) within a first region (410), and the first region (410) may correspond to the gaze of the user (490). That is, the first region (410) may be included in a range (401) recognizable to the user (490) based on the direction of the gaze. Specific details related thereto are further described in FIGS. 7A and 7B below.

[0086] The first region (410) may be referred to as a user recognition region, a gaze region, or a gaze region. According to one embodiment, the first visual effect (451) may include the display of a first visual object. For example, the first visual object may include a line corresponding to the periphery of the first object (440-1). For example, the first visual object may be at least partially superimposed on the first object (440-1). However, the present disclosure is not limited thereto. For example, the first visual object may be displayed at a position spaced apart from the periphery of the first object (440-1) by a specified distance without at least partially superimposing on the first object (440-1). The first visual effect (451) is not limited to the first visual object. For example, the first visual effect (451) may include displaying the first object (440-1) by highlighting it with a first brightness. Alternatively, for example, the first visual effect (451) may include displaying an indicator having a first size indicating the first object (440-1) in a surrounding area of ​​the first object (440-1).

[0087] For example, the wearable device (103) may provide a second visual effect (452) to a second object (440-2) within a second area (420) identified based on a field of view (FoV) (402) of the wearable device (103). The first area (410) may be included in the FoV (402). The second area (420) may represent at least a portion of the FoV (402) that is distinct from the first area (410). For example, the second area (420) may be an area outside the user's line of sight. The second area (420) may be referred to as a display area of ​​the wearable device (103). According to one embodiment, the second visual effect (452) may include blinking of the first visual object. For example, the first visual object for the second object (440-2) may include a line corresponding to the periphery of the second object (440-2) and having a specified thickness.

[0088] However, the present disclosure is not limited thereto. For example, the first visual object for the second object (440-2) may be displayed at a position that is spaced apart from the edge of the second object (440-2) by a specified length without at least partially overlapping the second object (440-2). For example, the first visual object for the second object (440-2) may be at least partially superimposed on the second object (440-2). In this case, the first visual object for the second object (440-2) may blink a specified number of times (or for a specified time). However, the second visual effect (452) is not limited to the display and blinking of the first visual object. For example, the second visual effect (452) may include highlighting and displaying the second object (440-2) with a second brightness that is brighter than the first brightness. Alternatively, for example, the second visual effect (452) may include displaying an indicator having a second size larger than the first size indicating the second object (440-2) in a surrounding area of ​​the second object (440-2).

[0089] For example, the wearable device (103) may provide a third visual effect (453) to an object (440-3) within a third area (430). For example, the third area (430) may represent an area outside the FoV (402) and may represent an area within a recognizable range (403) based on information acquired by the wearable device (103). The third area (430) may be referred to as a recognition area of ​​the wearable device (103) or an identification area of ​​the wearable device (103). For example, the range (403) may be identifiable based on the information acquired by the wearable device (103). For example, if the information is communication information obtained from an external electronic device (e.g., a beacon or an access point (AP)) connected via a communication circuit, the range (403) may be identified based on the location of other external electronic devices that can be identified based on the communication information. The third visual effect (453) may include the display of a second visual object. For example, the second visual object may include a rendered image representing the third object (440-3). For example, the user (490) may not recognize the third object (440-3) outside of the FoV (402), and only the wearable device (103) may recognize the third object (440-3). In addition, since the third object (440-3) is located outside of the FoV (402), it may not be displayed through the display of the wearable device (103). Accordingly, the wearable device (103) can display the second visual object, which is a rendered image of the third object (440-3), through a portion of the FoV (402). For example, the portion of the area can represent an area within the FoV (402) adjacent to an area outside the FoV (402) where the third object (440-3) is located.For example, the partial region may be referred to as a corner, an edge, a corresponding region of the FoV, an adjacent region, or a region of interest. In the example (400) of FIG. 4, the partial region may include the lower right. In this case, the third object (440-3) may be positioned to the right (or rear of the right) with respect to the user (490). However, the third visual effect (453) according to the embodiment of the present disclosure is not limited to the second visual object. For example, the third visual effect (453) may further include, in addition to the display of the second visual object, the display of an indicator to indicate the direction in which the third object (440-3) is positioned.

[0090] In example (400), a method of providing a visual effect to a real object and a user (490) is exemplified, but the embodiments of the present disclosure are not limited thereto. The embodiments of the present disclosure can be applied to a virtual object in a virtual environment and an avatar of a user (490). For example, the real object can correspond to a virtual object in the virtual environment, and the user (490) can correspond to the avatar of the user (490). Accordingly, a first region (410) can be identified according to the gaze of the avatar, and a second region (420) can be identified within a range (or FoV) in which at least a portion of the virtual environment is displayed through the wearable device (103). In addition, an area within a range recognizable by the wearable device (103) among the virtual environment other than at least a portion of the virtual environment can be identified as a third region (430).

[0091] The wearable device (103) may not simply display an object within the XR through the display (or display area) of the wearable device (103), but may identify information (e.g., an object) necessary for a situation or action performed by the user (490) and provide an alarm for the identified object. The electronic device and method according to embodiments of the present disclosure may effectively provide the user (490) with information about a situation (or context) recognized by the wearable device (103) and an object according to the situation. Accordingly, the electronic device and method according to embodiments of the present disclosure may provide the alarm to increase the user's (490's) level of attention to the object when the user's (490's) level of attention to the object is low. For example, if a user (490) does not look at an obstacle in front of him / her when walking on the road, does not look at an approaching bus when using public transportation, does not look at traffic lights or pedestrians when driving, or cannot find a destination near the destination when using a navigation system, the wearable device (103) can provide information appropriate to the user's (490) situation and provide convenience by providing the alarm.

[0092] Figure 5 illustrates an exemplary block diagram of a wearable device.

[0093] The wearable device (103) of FIG. 5 may be an example of the electronic device (101) of FIG. 1 and the wearable device (103) of FIGS. 2A to 3B.

[0094] Referring to FIG. 5, an exemplary situation is illustrated in which a wearable device (103), an external electronic device (570), and a server (580) are connected to each other based on a wired network and / or a wireless network. For example, the wired network may include a network such as the Internet, a local area network (LAN), a wide area network (WAN), or a combination thereof. For example, the wireless network may include a network such as long term evolution (LTE), 5g new radio (NR), wireless fidelity (WiFi), Zigbee, near field communication (NFC), Bluetooth, Bluetooth low-energy (BLE), or a combination thereof. Although the wearable device (103), the external electronic device (570), and the server (580) are illustrated as being directly connected, the wearable device (103), the external electronic device (570), and the server (580) may be indirectly connected via one or more routers and / or access points (APs). In other words, the wearable device (103) is exemplified as being directly connected to the communication circuit (571) of the external electronic device (570) (or, the communication circuit (581) of the server (580)) through the communication circuit (540), but the embodiments of the present disclosure are not limited thereto.

[0095] Referring to FIG. 5, a wearable device (103) may include a processor (510), a sensor (520), a display (530), a communication circuit (540), and a memory (550). The processor (510), the sensor (520), the display (530), the communication circuit (540), and the memory (550) may be electrically and / or operably coupled with each other by a communication bus. Hereinafter, operably coupled hardware components may mean that a direct connection or an indirect connection is established between the hardware components, either wired or wireless, so that a second hardware component is controlled by a first hardware component among the hardware components. Although illustrated based on different blocks, the present disclosure is not limited thereto, and some of the hardware components illustrated in FIG. 5 (at least a portion of the processor (510), the sensor (520), the display (530), the communication circuit (540), and the memory (550)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware components included in the wearable device (103) is not limited to those illustrated in FIG. 5. For example, the wearable device (103) may include only some of the hardware components illustrated in FIG. 5. It will be appreciated that the wearable device (103) may include additional components as well as omit some of the aforementioned components.

[0096] The processor (510) of the wearable device (103) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), and a field programmable gate array (FPGA). For example, the hardware component for processing data may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), and / or a neural processing unit (NPU). The number of processors (510) may be one or more. For example, the processor (510) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core. The processor (510) of FIG. 5 may include at least a part of the processor (120) of FIG. 1.

[0097] The sensor (520) of the wearable device (103) may include at least one sensor. For example, the sensor (520) may include at least a portion of the sensor module (176) of FIG. 1, the camera module (180) of FIG. 1, and the input module (150) of FIG. 1. For example, the sensor (520) may include an IMU (or IMU sensor), a bio sensor, an image sensor (or camera), an acoustic sensor (or microphone (or mic)), a gyro sensor, a gravity sensor, and / or an acceleration sensor.

[0098] The wearable device (103) can obtain an image of a surrounding area of ​​a user (e.g., user (490) of FIG. 4) wearing the wearable device (103) based on the sensor (520). For example, the wearable device (103) can obtain an image of the surrounding area through a camera included in the sensor (520). For example, the surrounding area can represent an omnidirectional view of the user (or the wearable device (103)). The wearable device (103) can identify at least one object within the image based on the image of the surrounding area.

[0099] The wearable device (103) can obtain acoustic information about the surrounding area of ​​the user based on the sensor (520). For example, the wearable device (103) can obtain the acoustic information through the microphone included in the sensor (520). For example, the acoustic information can include the user's voice. Based on the acoustic information, the wearable device (103) can identify the voice spoken by the user or a word (or sentence) required by the user.

[0100] The wearable device (103) can obtain the user's biometric information based on the sensor (520). For example, the wearable device (103) can obtain the biometric information based on the biometric sensor included in the sensor (520). For example, the biometric information may include at least one of the user's heart rate, body fat information, blood pressure, or pupil size. The wearable device (103) can identify the user's context by identifying the user's health status based on the biometric information. The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the biometric information.

[0101] The wearable device (103) can identify the user's moving direction, moving speed, or the direction in which the user's head is facing based on the sensor (520). For example, the wearable device (103) can identify the user's moving direction, moving speed, or the direction in which the user's head is facing based on the IMU sensor included in the sensor (520). The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the user's moving direction, moving speed, or the direction in which the user's head is facing.

[0102] The wearable device (103) can identify location information of the wearable device (103) based on the sensor (520). For example, the wearable device (103) can identify the location information of the wearable device (103) based on GPS information. Alternatively, for example, the wearable device (103) can identify current time information, temperature or air pressure information around the wearable device (103) based on the sensor (520).

[0103] The display (530) of the wearable device (103) can output visualized information to the user (e.g., the screen of FIG. 4, the screens of FIGS. 8A and 8B, the screen of FIG. 9B, and the screens of FIGS. 10A and 10B). The number of displays (530) included in the wearable device (103) may be one or more. For example, the display (530) may be controlled by a processor (510) and / or a graphic processing unit (GPU) (not shown) to output visualized information to the user. The display (530) may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), a digital mirror device (DMD), one or more light emitting diodes (LEDs), and / or a micro LED. The above LED may include an OLED (organic LED). The display (530) of FIG. 5 may include at least a portion of the display module (160) of FIG. 1.

[0104] In one embodiment, light transmission may occur through at least a portion of the display (530). The wearable device (103) may provide a user experience related to augmented reality by providing a combination of light output through the display (530) and light transmitted through the display (530) to the user. As described above with reference to FIGS. 2A and 2B and / or 3A and 3B , the display (530) of the wearable device (103) according to one embodiment may have a structure for covering the entire FoV of the user, or radiating light toward the FoV of the user, while being worn on a body part of the user, such as the head. Although not illustrated, the wearable device (103) may include other output means for outputting information in forms other than visual or auditory forms. For example, the wearable device (103) may include at least one speaker for outputting an audio signal, and / or a motor (or actuator) for providing haptic feedback based on vibration. In the above example, a display (530) for providing a user experience related to the augmented reality is exemplified, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may render an image for providing a user experience related to virtual reality, and display the rendered image through the display (530). In this case, the rendered image may include a visual object corresponding to an object related to the user's context.

[0105] The communication circuit (540) of the wearable device (103) may include hardware for supporting transmission and / or reception of electrical signals between the wearable device (103) and an external electronic device (570) (or a server (580)). The communication circuit (540) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (540) may support transmission and / or reception of electrical signals based on various types of communication means, such as Ethernet, Bluetooth (BT), Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G new radio (NR), and wireless-fidelity (WiFi). The communication circuit (540) of FIG. 5 may include at least a part of the communication module (190) and / or the antenna module (197) of FIG. 1.

[0106] The wearable device (103) can obtain communication information from another external electronic device connected through the communication circuit (540). For example, the communication circuit (540) can include a Wi-Fi module or a BT module. For example, the other external electronic device can include a router connected through the Wi-Fi module or a beacon connected through the BT module. For example, the communication information can include information about electronic devices connected to the router or the beacon. For example, the communication information can include the location of the electronic devices, information of the electronic devices (e.g., type, model name, etc.), information used by the electronic devices (e.g., traffic information, location information, etc.), and the location of the wearable device (103).

[0107] The wearable device (103) can identify a context and an object representing a situation of a user of the wearable device (103) based on the information acquired through the sensor (520) and the communication circuit (540). For example, the information can identify an image of the surrounding area, the sound information, the user's biometric information, the user's moving direction, the moving speed, the direction in which the user's head is facing, and the location information of the wearable device (103). It can include at least one of current time information, temperature or air pressure information around the wearable device (103), or the communication information. For example, based on the information, the wearable device (103) can identify the context. In addition, for example, based on the information, the wearable device (103) can identify an object located around the user (or located around an avatar corresponding to the user in a virtual space). The user's surroundings may be understood as being substantially identical to the surroundings of a wearable device (103) worn by the user. The objects located in the user's surroundings may represent one or more objects that can be identified based on the information. The objects located in the user's surroundings may be different from objects (or alarm objects) related to the context. Thereafter, the wearable device (103) may identify objects related to the context among the objects located in the user's surroundings. In other words, the information may be used to identify the context, objects located in the user's surroundings, and objects related to the context.

[0108] The memory (550) of the wearable device (103) may include a hardware component for storing data and / or instructions input to and / or output from the processor (510). The memory (550) may include, for example, volatile memory such as random-access memory (RAM), and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, and embedded multimedia card (eMMC). The memory (550) of FIG. 5 may include at least a portion of the memory (130) of FIG. 1.

[0109] Although not shown, the wearable device (103) may include an output means for outputting information in a form other than a visualized form. For example, the wearable device (103) may include a speaker for outputting an acoustic signal. For example, the wearable device (103) may include a motor for providing haptic feedback based on vibration.

[0110] The wearable device (103) may operate in conjunction with an external electronic device (570) and a server (580). In one or more embodiments, the wearable device (103) may establish a short-range communication link, such as a Bluetooth connection, with the external electronic device (570). The server (580) may also be utilized to provide specific information in response to an inquiry.

[0111] Referring to FIG. 5, according to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (510) of the wearable device (103) may be stored in the memory (550) of the wearable device (103). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or an application. Hereinafter, when an application is installed in an electronic device (e.g., a wearable device (103)), it may mean that one or more instructions provided in the form of an application are stored in the memory (550), and that the one or more applications are stored in a format executable by the processor of the electronic device (e.g., a file having an extension designated by the operating system of the wearable device (103)). The wearable device (103) can perform the operations of FIGS. 6, 11, and 12 by executing one or more instructions stored in the memory (550).

[0112] Referring to FIG. 5, one or more instructions included in the memory (550) may be classified into a context identifier (551), a gaze area identifier (553), an alarm object identifier (555), and / or a notification information generator (557). For example, each of the context identifier (551), the gaze area identifier (553), the alarm object identifier (555), and the notification information generator (557) may be implemented as a program or software. The wearable device (103) may identify a context indicating the user's situation using the context identifier (551). For example, the wearable device (103) may obtain the information through a sensor (520) and / or a communication circuit (540). For example, the wearable device (103) can identify the context using the context identifier (551) based on the acquired information. For example, the context can define the user's action or situation. For example, the context can be referred to as a routine, situational information, an action in which the user participates, or a behavior pattern. In other words, the context can be defined so that the wearable device (103) can interpret the user's action or situation and provide the user with necessary information or services. For example, the context can include the user walking, the user using public transportation, the user driving, the user watching a performance, the user ordering a menu at a restaurant, the user cooking, or the user resting. However, the embodiments of the present disclosure are not limited thereto.

[0113] The wearable device (103) may identify the context further based on the application the user is using via the wearable device (103). For example, if the user is using a map application, the wearable device (103) may identify the context as being related to movement. Or, for example, if the user is using a public transportation search application, the context may be identified as being related to using the public transportation. For example, if the user is using a recipe providing application, the context may be identified as being related to cooking. The wearable device (103) may store a table in the memory (550) that associates different contexts with different applications.

[0114] In one or more embodiments, the context identifier (551) may identify a context based on a comparison of information from a sensor (573) with a library of information corresponding to known contexts of the user.

[0115] In one or more embodiments, the user may provide the context as input to the context identifier (551).

[0116] In one or more embodiments, the wearable device (103) may capture an image, provide the image to a server that uses artificial intelligence to identify objects within the image, and determine an action in which the user engages based on the identified objects.

[0117] According to one embodiment, the wearable device (103) can identify the first region using the gaze region identifier (553). For example, the wearable device (103) can identify the direction of the user's gaze using an eye tracking camera (ET CAM). For example, the wearable device (103) can identify the first region based on the user's input based on the identified gaze direction. For example, the wearable device (103) can perform calibration to identify the first region. For example, the calibration can include identifying the size of the first region by obtaining the user's input for a calibration object displayed at an arbitrary location based on the gaze direction. For example, the user's input can include various gestures such as a touch input, a grip input, a press input, and a drag input. Specific details related thereto are described below in FIG. 7A.

[0118] The wearable device (103) can change the size of the first region using the gaze region identifier (553). For example, the wearable device (103) can change the size and location of the first region based on the information acquired through the sensor (520). For example, the size or location of the first region can be changed based on the size of the user's pupil or whether the user is moving. Specific details related to this are described below in FIG. 7B.

[0119] The wearable device (103) can identify an alarm object using an alarm object identifier (555). The alarm object can represent an object related to the context identified through the context identifier (551). The alarm object can be referenced as an object related to the context, an object of interest, a point of interest (POI), a visual element related to the context, a visual object related to the context, a graphical object related to the context, or a target object. The alarm object identifier (555) can include a library of objects known to be related to different contexts. For example, the alarm object identifier (555) can list objects related to the driver, such as other vehicles, pedestrians, traffic signals, and road signs.

[0120] For example, the wearable device (103) can identify objects around the wearable device (103) based on the information acquired through the sensor (520) and / or the communication circuit (540) using the alarm object identifier (555). For example, the wearable device (103) can identify an object (i.e., the alarm object) related to the context among the surrounding objects based on the identified context representing the user's situation. For example, if the context is moving toward a specific destination on foot or by public transportation, the alarm object may include public transportation to be boarded, the destination, obstacles, traffic lights, crosswalks, and external objects with a risk of collision (e.g., bicycles, etc.). Alternatively, if the context is driving a vehicle, the alarm object may include traffic lights, a crosswalk situation nearby along the route, and a blind spot of the vehicle. Alternatively, if the context is viewing a performance, such as a musical or play, the object may include an actor, a stage area, or an actor's lines. Alternatively, if the context is ordering a menu at a restaurant, the object may include order completion information or a waiter.

[0121] The wearable device (103) can identify the priority of the alarm object using the alarm object identifier (555). For example, the wearable device (103) can identify the priority of the alarm object based on the context. For example, the priority can represent an emergency level (or urgency) that defines the order in which notification information is to be provided among alarm objects related to the context. For example, the priority can be referenced as a notification level, a notification order, an urgency, or an emergency level. According to one embodiment, the wearable device (103) can display text describing the alarm object within the FoV of the wearable device (103) when the priority of the alarm object is equal to or higher than a reference level. Specific details related thereto are described below in FIG. 8B.

[0122] The wearable device (103) can use the alarm object identifier (555) to identify an available notification function along with notification information for the alarm object. For example, the notification function may be different from the notification information displayed through the display (530) of the wearable device (103) and may include an additional action performed along with the display of the notification information. For example, when the wearable device (103) is connected to a vehicle, the notification function may include vibration of the steering wheel of the vehicle or a change in the seat position. However, the embodiments of the present disclosure are not limited thereto. For example, the notification function may include vibration of the wearable device (103) or blinking of a light-emitting module included in the wearable device (103).

[0123] The wearable device (103) may generate the notification information using the notification information generator (557). For example, the notification information may include a visual effect for the alarm object (e.g., the first visual effect (451), the second visual effect (452), and the third visual effect (453) of FIG. 4). Or, for example, the notification information may include text related to the alarm object. Or, for example, the notification information may include voice information, which is the text converted into speech.

[0124] The wearable device (103) can identify whether the alarm object is visual information. The visual information can indicate the type of the alarm object. For example, the type can include visual information representing an image and non-visual information including text and voice. For example, if the alarm object is visual information (e.g., a specific object), the wearable device (103) can provide a visual effect according to the location of the alarm object. Alternatively, for example, if the alarm object is non-visual information (e.g., text or voice), the wearable device (103) can provide notification information according to the alarm object. If the non-visual information is text, the wearable device (103) can display a visual object representing the text through the display (530). Alternatively, if the non-visual information is voice, the wearable device (103) can output the voice. Specific details related to this are described in Figure 6 below.

[0125] According to one embodiment, the external electronic device (570) may include at least one of a communication circuit (571), a sensor (573), a display (575), a processor (577), and a memory (579). The communication circuit (571), the sensor (573), the display (575), the processor (577), or the memory (579) may be electrically and / or operably coupled with each other by an electronic component, such as a communication bus. The type and / or number of hardware components included in the external electronic device (570) is not limited to those illustrated in FIG. 5. For example, the external electronic device (570) may include only some of the hardware components illustrated in FIG. 5. For example, the communication circuit (571) may correspond to the communication circuit (540) of the wearable device (103). For example, the sensor (573) may correspond to the sensor (520) of the wearable device (103). For example, the display (575) may correspond to the display (575) of the wearable device (103). For example, the processor (577) may correspond to the processor (510) of the wearable device (103). For example, the memory (579) may correspond to the memory (550) of the wearable device (103). To reduce repetition of description, redundant descriptions may be omitted. For example, the external electronic device (570) may include a vehicle connected to the wearable device (103). For example, examples of the external electronic device (570) are described below in FIGS. 9A and 9B. The above vehicle external electronic device (570) can transmit and receive signals with other vehicles or other devices connected through a communication circuit (571).For example, the communication circuit (571) may include a module for V2X (vehicle to everything or vehicle to x). For example, an external electronic device (570) may obtain data such as surrounding traffic information or information about pedestrians based on the module for V2X.

[0126] According to one embodiment, the server (580) may include at least one of a communication circuit (581), a processor (583), and a memory (585). The communication circuit (581), the processor (583), or the memory (585) may be electrically and / or operably coupled with each other by an electronic component, such as a communication bus. The type and / or number of hardware components included in the server (580) is not limited to those illustrated in FIG. 5. For example, the server (580) may include only some of the hardware components illustrated in FIG. 5. For example, the communication circuit (581) may correspond to the communication circuit (540) of the wearable device (103). For example, the processor (583) may correspond to the processor (510) of the wearable device (103). For example, the memory (585) may correspond to the memory (550) of the wearable device (103). To reduce repetition of description, redundant descriptions may be omitted. For example, the server (580) may be connected to at least one Internet of Things (IoT) device. For example, the server (580) may represent an external electronic device that manages the at least one IoT device. For example, examples of the server (580) are described below in FIGS. 10A and 10B .

[0127] Figure 6 illustrates an example of an operation flow for a method of identifying an object according to a user's context and providing notification information based on the location of the identified object.

[0128] At least some of the methods of FIG. 6 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the methods may be controlled by the processor (510) of the wearable device (103). In the embodiments described below, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0129] As described in more detail below, the wearable device (103) can determine a first region corresponding to the user's gaze. Based on information from the sensor, the wearable device (103) can identify the user's context and objects related to the context. Depending on whether an object is located within the first region corresponding to the user's gaze, within a second region outside the first region and within the FoV, or within a third region outside the FoV, different visual effects can be provided.

[0130] Referring to FIG. 6, although not illustrated in FIG. 6, the wearable device (103) can execute an application including a plurality of instructions that cause the wearable device (103) to provide augmented reality. For convenience of explanation, the following description will be based on augmented reality, but embodiments of the present disclosure can also be applied to virtual reality or mixed reality. For example, the wearable device (103) can identify whether a function for providing notification information based on the location of the object is set when the instructions are executed. For example, the function for providing notification information based on the location of the object can be activated or deactivated in the global settings of the wearable device (103) or in the settings within the application including the plurality of instructions. In other words, when the function is activated, the wearable device (103) can provide the notification information based on the location of the object described below.

[0131] According to one embodiment, in operation (600), the wearable device (103) may identify a first region based on the user's gaze. For example, the wearable device (103) may perform calibration to identify the first region. The calibration may include a process for identifying the first region based on the direction of the user's gaze. Within the calibration, the wearable device (103) may display a visual object at an arbitrary location within a peripheral region centered on the direction (or point) of the gaze. The visual object may be referred to as a calibration object. For example, the wearable device (103) may obtain a user input for the calibration object. For example, based on the input, the wearable device (103) may identify a range recognizable by the user with respect to the direction of the gaze. For example, the range may represent a boundary of the first region. For example, the above range may be referred to as the first region. Specific details related thereto are described below in Fig. 7a.

[0132] In one embodiment, the size of the first region may be changed based on information obtained in operation (605). For example, the size of the first region may be changed based on biometric information contained in the information or information indicating whether the user is moving. Specific details related to this are described below in FIG. 7B.

[0133] According to one embodiment, in operation (605), the wearable device (103) may obtain information through the sensor (520) and the communication circuit (540). For example, the wearable device (103) may obtain the information through the sensor (520) and / or the communication circuit (540). For example, the information may be used to identify a context for the user or an object within a recognizable range of the wearable device (103). For example, the context may include an action in which the user participates.

[0134] The wearable device (103) can obtain an image of a surrounding area of ​​a user wearing the wearable device (103) based on a sensor (520). For example, the wearable device (103) can obtain an image of the surrounding area through a camera included in the sensor (520). For example, the surrounding area can represent an omnidirectional view of the user. The wearable device (103) can identify at least one object within the image based on the image of the surrounding area.

[0135] The wearable device (103) can obtain acoustic information about the surrounding area of ​​the user based on the sensor (520). For example, the wearable device (103) can obtain the acoustic information through the microphone included in the sensor (520). For example, the acoustic information can include the user's voice. Based on the acoustic information, the wearable device (103) can identify the voice spoken by the user or a word (or sentence) required by the user.

[0136] The wearable device (103) can obtain the user's biometric information based on the sensor (520). For example, the wearable device (103) can obtain the biometric information based on the biometric sensor included in the sensor (520). For example, the biometric information may include at least one of the user's heart rate, body fat information, blood pressure, or pupil size. The wearable device (103) can identify the user's context by identifying the user's health status based on the biometric information. The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the biometric information.

[0137] The wearable device (103) can identify the user's moving direction, moving speed, or the direction in which the user's head is facing based on the sensor (520). For example, the wearable device (103) can identify the user's moving direction, moving speed, or the direction in which the user's head is facing based on the IMU sensor included in the sensor (520). The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the user's moving direction, moving speed, or the direction in which the user's head is facing.

[0138] The wearable device (103) can identify location information of the wearable device (103) based on the sensor (520). For example, the wearable device (103) can identify the location information of the wearable device (103) based on GPS information. Alternatively, for example, the wearable device (103) can identify current time information, temperature or air pressure information around the wearable device (103) based on the sensor (520).

[0139] The wearable device (103) can obtain communication information from another external electronic device connected through the communication circuit (540). For example, the communication circuit (540) can include a Wi-Fi module or a BT module. For example, the other external electronic device can include a router connected through the Wi-Fi module or a beacon connected through the BT module. For example, the communication information can include information about electronic devices connected to the router or the beacon. For example, the communication information can include the location of the electronic devices, information of the electronic devices (e.g., type, model name, etc.), information used by the electronic devices (e.g., traffic information, location information, etc.), and the location of the wearable device (103).

[0140] In operation (610), the wearable device (103) can identify the context of the user. For example, the wearable device (103) can identify the context of the wearable device (103) and an object related to the context based on the information acquired through the sensor (520) and the communication circuit (540).

[0141] For example, the context may define the user's action or situation. For example, the context may be referred to as a routine, situational information, an action in which the user participates, or a behavior pattern. In other words, the context may be defined so that the wearable device (103) can interpret the user's action or situation and provide the user with information or services required. For example, the context may include the user walking, the user using public transportation, the user driving, the user watching a performance, the user ordering a menu at a restaurant, the user cooking, or the user resting. However, the embodiments of the present disclosure are not limited thereto.

[0142] The wearable device (103) may identify the context further based on the application being used by the user via the wearable device (103). For example, if the user is using a map application, the wearable device (103) may identify the context as being related to movement. Alternatively, if the user is using a public transportation search application, the context may be identified as being related to using the public transportation. For example, if the user is using a recipe providing application, the context may be identified as being related to cooking.

[0143] According to one embodiment, in operation (615), the wearable device (103) may identify an alarm object related to the context. The alarm object may be identified from among objects within a recognizable range of the wearable device (103) identified based on the information. The objects within the recognizable range may include objects located around the wearable device (103) (or the user). In other words, the alarm object may represent an object that is a target of notification information provision among the objects. The alarm object may be referenced as an object related to the context, an object of interest, a point of interest (POI), a visual element related to the context, a visual object related to the context, a graphical object related to the context, or a target object.

[0144] The wearable device (103) may identify an object (i.e., an alarm object) related to the context among the objects based on the identified context representing the user's situation. For example, if the context is walking or moving toward a specific destination via public transportation, the alarm object may include the public transportation to be boarded, the destination, obstacles, traffic lights, crosswalks, and external objects with a risk of collision (e.g., bicycles, etc.). Alternatively, if the context is driving a vehicle, the alarm object may include traffic lights, a crosswalk situation nearby along the route, and the vehicle's blind spot. Alternatively, if the context is watching a performance such as a musical or play, the object may include an actor, a stage area, or an actor's lines. Alternatively, if the context is ordering a menu at a restaurant, the object may include the completion of the order or a waiter, etc.

[0145] The wearable device (103) can identify the priority of the alarm object. For example, the wearable device (103) can identify the priority of the alarm object based on the context. For example, the priority can indicate an emergency level (or urgency) that defines the order in which notification information is to be provided among alarm objects related to the context. According to one embodiment, the wearable device (103) can display text describing the alarm object within the FoV of the wearable device (103) if the priority of the alarm object is equal to or higher than a reference level. Specific details related thereto are described below in FIG. 8B.

[0146] The wearable device (103) can identify available notification functions along with notification information for the alarm object. For example, the notification function may be different from the notification information displayed through the display (530) of the wearable device (103) and may include additional actions performed along with the display of the notification information. For example, if the wearable device (103) is connected to a vehicle, the notification function may include vibration of the steering wheel of the vehicle or a change in seat position. However, the embodiments of the present disclosure are not limited thereto. For example, the notification function may include vibration of the wearable device (103) or blinking of a light-emitting module included in the wearable device (103).

[0147] According to one embodiment, in operation (620), the wearable device (103) can identify whether the alarm object is visual information. The visual information may indicate the type of the alarm object. For example, the type may include visual information representing an image and non-visual information including text and voice. For example, if the alarm object is a traffic light, the type of the alarm object may be visual information. Conversely, if the alarm object is text indicating a red light, the alarm object may be non-visual information.

[0148] In operation (620), if the alarm object is the non-visual information, the wearable device (103) may perform operation (625). Conversely, in operation (620), if the alarm object is the visual information, the wearable device (103) may perform operation (630).

[0149] According to one embodiment, in operation (625), the wearable device (103) may provide notification information according to the type of the alarm object. For example, the wearable device (103) may provide notification information according to the type of the alarm object based on identifying that the alarm object is non-visual information. For example, the type may include text or voice. For example, if the non-visual information is text, the wearable device (103) may display a visual object representing the text through the display (530). The visual object representing the text may be displayed within a second area or the first area within the FoV of the wearable device (103). For example, a visual object summarizing the text may be displayed within the first area. Alternatively, the visual object representing the text may be displayed within the second area. Specific details related thereto are described below in FIG. 8B. Alternatively, if the non-visual information is voice, the wearable device (103) can output the voice.

[0150] According to one embodiment, in operation (630), the wearable device (103) may identify whether the location of the alarm object is within the first area. The wearable device (103) may identify the location of the alarm object based on identifying that the alarm object is the visual information. In operation (630), based on identifying that the location is within the first area, the wearable device (103) may perform operation (635). Alternatively, in operation (630), based on identifying that the location is outside the first area, the wearable device (103) may perform operation (640).

[0151] According to one embodiment, in operation (635), the wearable device (103) may provide a first visual effect based on identifying that the location is within the first area. For example, the first visual effect may include displaying a first visual object. For example, the first visual object may include a line corresponding to the periphery of the alarm object and having a specified thickness. For example, the first visual object may be at least partially superimposed on the alarm object. However, the embodiments of the present disclosure are not limited thereto. For example, the first visual object may be displayed at a position that is spaced apart from the periphery of the alarm object by a specified length without being at least partially superimposed on the alarm object. The first visual effect according to the embodiments of the present disclosure is not limited to the first visual object. For example, the first visual effect may include displaying the alarm object by highlighting it using a first brightness. Alternatively, for example, the first visual effect may include displaying an indicator having a first size indicating the alarm object in a surrounding area of ​​the alarm object.

[0152] According to one embodiment, in operation (640), the wearable device (103) may identify whether the alarm object is within a second area. For example, the second area may represent an area identified based on a field of view (FoV) of the wearable device (103). For example, the FoV may include the first area and the second area. For example, the second area may represent at least a portion of the FoV of the wearable device (103) that is distinct from the first area. The second area may be referred to as a display area of ​​the wearable device (103). For example, the wearable device (103) may identify whether the location of the alarm object is outside the first area and within the second area. In operation (640), if it is outside the first area and within the second area, the wearable device (103) may perform operation (645). In contrast, in operation (640), if outside the second area, the wearable device (103) can perform operation (650).

[0153] According to one embodiment, in operation (645), the wearable device (103) may provide a second visual effect. For example, the wearable device (103) may display the second visual effect based on identifying that the location is within the second region and outside the first region. The second visual effect may include blinking of the first visual object. For example, the first visual object of the second visual effect may include a line corresponding to the periphery of the alarm object and having a specified thickness. However, the embodiments of the present disclosure are not limited thereto. For example, the first visual object for the alarm object may be displayed at a position that is spaced apart from the periphery of the alarm object by a specified length without at least partially overlapping the alarm object. For example, the first visual object of the second visual effect may be at least partially superimposed on the alarm object. At this time, the first visual object of the second visual effect may blink a specified number of times. However, the second visual effect according to the embodiment of the present disclosure is not limited to displaying and blinking the first visual object. For example, the second visual effect may include displaying the alarm object by highlighting it with a second brightness that is brighter than the first brightness. Alternatively, for example, the second visual effect may include displaying an indicator having a second size that is larger than the first size indicating the alarm object in a surrounding area of ​​the alarm object.

[0154] According to one embodiment, in operation (650), the wearable device (103) may provide a third visual effect. For example, the wearable device (103) may display the third visual effect based on identifying that the location is outside the first area and the second area (or outside the FoV). The location outside the first area and the second area may be understood to be located within a third area. For example, the third area may represent an area outside the FoV of the wearable device (103) and may represent an area within an identifiable range based on information acquired by the wearable device (103). The third area may be referred to as a recognition area of ​​the wearable device (103) or an identification area of ​​the wearable device (103).

[0155] The third visual effect may include displaying a second visual object. For example, the second visual object may include a rendered image representing the alarm object. For example, the user may not recognize the alarm object outside the FoV, and only the wearable device (103) may recognize the alarm object. In addition, since the alarm object is located outside the FoV, it may not be displayed through the display of the wearable device (103). Accordingly, the wearable device (103) may display the second visual object, which is a rendered image of the alarm object, through a portion of the FoV. For example, the portion may represent an area within the FoV adjacent to an area outside the FoV where the alarm object is located. For example, the portion may be referred to as a corner, an edge, a corresponding area of ​​the FoV, an adjacent area, or an area of ​​interest. However, the third visual effect according to an embodiment of the present disclosure is not limited to the second visual object. For example, the third visual effect may further display an indicator indicating the direction in which the alarm object is located, in addition to displaying the second visual object.

[0156] According to one embodiment, in operation (655), the wearable device (103) can identify whether the location of the alarm object changes within the first area. For example, the wearable device (103) can identify whether the location within the second area or the third area changes within the first area. For example, the wearable device (103) can identify whether the user's gaze changes to the location of the alarm object while the second visual effect or the third visual effect is displayed. This is because the second visual effect and the third visual effect are used to induce the user to look at the alarm object.

[0157] In operation (655), based on identifying that the location of the alarm object has changed within the first area, the wearable device (103) may perform operation (660). Alternatively, in operation (665), based on identifying that the location of the alarm object has not changed within the first area, the wearable device (103) may perform operation (665).

[0158] In one embodiment, in operation (660), the wearable device (103) may stop providing a visual effect. For example, the wearable device (103) may stop displaying the second visual effect (or the third visual effect) based on a change in the location of the alarm object within the first area.

[0159] According to one embodiment, in operation (665), the wearable device (103) may change the brightness of the second region. For example, the wearable device (103) may lower the brightness of the second region based on determining that the location of the alarm object has not changed within the first region. Since the user's attention is not attracted by the display of the second visual effect (or the third visual effect), the wearable device (103) may lower the brightness of the second region. For example, the wearable device (103) may change the brightness of the second region from a first brightness to a second brightness that is lower than the first brightness. By changing the brightness of the second region, the wearable device (103) may cause the user to look at the alarm object.

[0160] Figures 7a and 7b illustrate examples of a method for identifying a first region based on a user's gaze.

[0161] The above method of FIGS. 7A and 7B can be performed by the wearable device (103) of FIG. 5. For example, the user may represent a user of the wearable device (103). For example, the first region (410) may represent an area within a user-recognizable range (401) identified based on the user's gaze.

[0162] Referring to FIG. 7A, the wearable device (103) can perform calibration. For example, the wearable device (103) can perform the calibration to set a first region (410). For example, the wearable device (103) can perform the calibration to set a first region (410) within the FoV (402) of the wearable device (103). For example, the first region (410) can be identified based on a direction (710) indicated by the user's gaze.

[0163] The wearable device (103) may display calibration objects (720) when performing the calibration. For example, the wearable device (103) may display calibration objects (720) in a peripheral area of ​​the direction (710). For convenience of explanation, an example of displaying 10 calibration objects (720) is illustrated in FIG. 7A, but the present disclosure is not limited thereto. The wearable device (103) may obtain user input for the calibration objects (720). For example, the input may include touch input, point input, drag input, grip input, or various gestures. For example, the input may be referred to as user input. The wearable device (103) may obtain data of inputs for the calibration objects (720). For example, based on the above data, the wearable device (103) can identify a range (401) exceeding a specified confidence level. The specified confidence level can represent a level at which the user can identify an object while looking in a direction (710). The wearable device (103) can identify an area within the range (401) identified based on the calibration as a first area (410).

[0164] The size of the first area (410) may be set to a default size. For example, the default size of the first area (410) may be a value set during the production of the wearable device (103). The wearable device (103) may set (or update) the first area (410) to a size for the user based on the calibration from the default size.

[0165] In one or more embodiments, the wearable device (103) can determine a direction (710) gazed by the user. The wearable device (103) can place calibration objects (720) at various distances from the direction (710) and prompt the user by touching the calibration objects (720). The first region (410) can be accurately identified by the user or with a specified confidence level.

[0166] FIG. 7B illustrates examples (730, 740) of how a wearable device (103) adjusts a first region (410) having a size set based on the calibration. Example (730) may represent a first region (410) of a range (735) having a first size identified based on the calibration. Alternatively, example (740) may represent a first region (410) of a range (745) having a second size changed from the range (735) having the first size.

[0167] The wearable device (103) can set a first area (410) of a range (735) having the first size based on the calibration. The first area (410) of the range (735) having the first size can be set for the user. The wearable device (103) can change the first size to the second size based on the information acquired through the sensor (520) and / or the communication circuit (540). For example, when the information is biometric information including a pupil size, the wearable device (103) can set the first size for a pupil having the first pupil size. Thereafter, when the size of the user's pupil changes from the first pupil size to a smaller second pupil size, the wearable device (103) can identify a first area (410) of a range (745) having the second size reduced from the first size. In addition, for example, when the information is information indicating the movement state of the user, the wearable device (103) can set the first size for the user who is stationary. Thereafter, when the user moves, the wearable device (103) can identify a first area (410) of a range (745) having the second size reduced from the first size. In the above example, a case of changing from the stationary state to the moving state is described, but the embodiments of the present disclosure are not limited thereto. For example, based on identifying a movement of the user having a first movement speed and a movement having a second movement speed that is faster than the first movement speed, the wearable device (103) can identify a range (745) having the second size from a range (735) having the first size. In other words, the size of the first region (410) can be set in inverse proportion to the movement speed and acceleration of the user.

[0168] Figures 8a and 8b illustrate examples of visual effects provided depending on the location and type of an object. In example (801), the object (810) may be within a first area (410). In examples (802) and (803), the object (810) may be within a second area (420).

[0169] Figures 8a and 8b illustrate examples (801, 802, 803, 804) of visual effects provided depending on the position and type of the object. The object may represent an alarm object that is the target of notification information. For example, examples (801, 802, 803) illustrate visual effects provided depending on the position of the object when the object is visual information. In addition, for example, example (804) illustrates a visual effect in which a visual object representing the object is displayed when the object is non-visual information (e.g., text).

[0170] Examples (801, 802, 803, 804) each illustrate a first region (410), a second region (420), and a third region (430) of the wearable device (103). The first region (410) can be identified based on a user-perceivable range (401) of the wearable device (103). The second region (420) can be identified based on a FoV (402) of the wearable device (103). The third region (430) can be identified based on a range (403) within which the wearable device (103) is perceivable other than the FoV (402).

[0171] Referring to example (801), the wearable device (103) can identify that the location of the object (810) is within the first region (410). For example, the wearable device (103) can provide a first visual effect (820) based on identifying that the location is within the first region (410). For example, the first visual effect (820) can include displaying a first visual object. For example, the first visual object can include a line corresponding to the periphery of the object (810) and having a specified thickness. However, the embodiments of the present disclosure are not limited thereto. For example, the first visual effect (820) can include displaying an indicator indicating the object (810) at an upper portion of the object (810).

[0172] The wearable device (103) can perform hand tracking. For example, the wearable device (103) can identify the location of the user's hand (or finger) of the wearable device (103). The wearable device (103) can identify that the location of the user's hand (or a virtual object corresponding to the hand) is located within the first region (410). For example, if the user's hand is located within the first region (410), the wearable device (103) can refrain from providing a visual effect to an object located in a partial region within the first region (410) where the hand is located. In example (801), if the hand at least partially overlaps the location of the object (810), the wearable device (103) can refrain from displaying the first visual object. However, the present disclosure is not limited thereto. For example, the wearable device (103) may display a first visual object having a thickness thinner than the specified thickness for the first visual object having the specified thickness when the hand at least partially overlaps the position of the object (810). In other words, the wearable device (103) may provide a visual effect at a lower level than the first visual effect (820) when the position of the hand identified by the hand tracking is located within the first area (410).

[0173] Referring to example (802), the wearable device (103) can identify that the location of the object (810) is within the second area (420). For example, the wearable device (103) can provide a second visual effect (830) based on identifying that the location is within the second area (420). For example, the second visual effect (830) can include displaying the first visual object and blinking the displayed first visual object. For example, the first visual object can include a line corresponding to the periphery of the object (810) and having a specified thickness. For example, the second visual effect (830) can blink a specified number of times (or for a specified time) while the first visual object is displayed. The second visual effect (830) including the above blinking can more effectively attract the user's attention compared to the first visual effect (820). However, the present disclosure is not limited thereto. For example, the second visual effect (830) may include displaying an indicator indicating the object (810) at an upper portion of the object (810) and blinking the displayed indicator. Alternatively, for example, the second visual effect (830) may include displaying the first visual object relative to the object (810) and displaying a dotted line extending from the position of the user's gaze to the object (810).

[0174] Referring to example (803), the wearable device (103) can identify that the location of the object (810) is within the third region (430). For example, the wearable device (103) can provide a third visual effect (840) based on identifying that the location is within the third region (430). For example, the third visual effect (840) can include displaying a second visual object representing the object (810). For example, the second visual object can represent an image in which the object (810) is rendered. The second visual object can be displayed in a portion (850) of the FoV (402). For example, the portion (850) can represent an area adjacent to the location of the third region (430) among the areas of the FoV (402). In the example (803) of FIG. 8A, the object (810) is located at the upper right within the third region (430) with respect to the FoV (402), and the partial region (850) may be the upper right of the FoV (402). In addition, for example, the third visual effect (840) may include blinking of the partial region (850) together with the display of the second visual object. Or, for example, the third visual effect (840) may further display an indicator (e.g., an arrow) to indicate the location of the object (810) together with the display of the second visual object. For example, the user cannot recognize the object (810) other than the FoV (402), and only the wearable device (103) can recognize the object (810). Additionally, since the object (810) is located outside of the FoV (402), it may not be displayed through the display (530) of the wearable device (103). Accordingly, the wearable device (103) may display an image that renders the second visual object corresponding to the object (810). However, the embodiments of the present disclosure are not limited thereto.For example, the third visual effect (840) may include adjusting the resolution of the second visual object and the partial region (850) to correspond to the resolution of the first region (410). For example, in a wearable device (103) that provides VR, the wearable device (103) may render and display an image having a relatively high resolution for the first region (410) where the user's gaze is located. In addition, the wearable device (103) may adjust the resolution to be lower as the image gets farther from the gaze. The method of adjusting the resolution described above may be referred to as foveated rendering. The wearable device (103) may adjust the resolution of the second visual object and the partial region (850) to correspond to the resolution of the first region (410). For example, the third visual effect (840) may output text or voice describing the object (810). For example, the text may be displayed in another part of the FoV (402) or within the first part (410). For example, the text displayed within the first part (410) may be a simplified description of the object (810). For example, if the text displayed within the other part of the FoV (402) is “The traffic light ahead is red,” the text displayed within the first part (410) may be “Stop sign.” As an example of how the text is displayed, reference may be made to example (804) of FIG. 8B below.

[0175] Referring to example (804), the wearable device (103) can identify whether the type of object is the non-visual information. For example, the wearable device (103) can display text describing the object based on identifying the object as the non-visual information. For example, the wearable device (103) can display text (860) that summarizes and describes the object within the first area (410). Alternatively, for example, the wearable device (103) can display text (870) describing the object within the second area (420). In the example of FIG. 8B, if the text (870) is "The traffic light ahead is red," the text (860) can be "Stop sign." Additionally, as described in example (803) of FIG. 8A, the wearable device (103) may display text (860) and text (870) together with the second visual object based on identifying the location of the object within the third area (430). However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may display only text (870) and display an indicator indicating the location of the text (870) instead of the text (860).

[0176] In one or more embodiments, the context may include driving. For example, the context may include an action in which the user engages, and the user may engage in driving.

[0177] Figure 9a illustrates an example of a recognition area identified when a user wearing a wearable device uses a vehicle.

[0178] FIG. 9A illustrates an example (900) of an area recognized by an external electronic device (570) connected to a wearable device (103). For example, the external electronic device (570) may include a vehicle. For example, the external electronic device (570) may be connected to the wearable device (103) via a communication circuit (571). For example, the external electronic device (570) may be a vehicle in which a user wearing the wearable device (103) is riding.

[0179] In example (900), a method for synchronizing an area (930) recognized by an external electronic device (570) and an area recognized by a user wearing a wearable device (103) (e.g., FoV (402) including a first area (410) and a second area (420)) is illustrated. For example, the first area (410) may be identified based on a range (401) recognizable to the user of the wearable device (103). The second area (420) may be identified based on the FoV (402) of the wearable device (103).

[0180] The area (930) may represent an area within a recognizable range (903) based on information acquired by the external electronic device (570). The information may include information about the surroundings of the vehicle acquired through the sensor (573) and / or the communication circuit (571) of the external electronic device (570). For example, the area (930) may be referred to as a recognition area of ​​the external electronic device (570) (or the vehicle). The area (930) may be identified based on the information about the surroundings of the vehicle. For example, the information about the surroundings of the vehicle may include surrounding traffic information, information about pedestrians, and driving-related information of other vehicles (e.g., moving speed, direction, etc.) acquired based on the communication circuit (571) and the sensor (573) of the external electronic device (570).

[0181] The external electronic device (570) can receive FoV information and gaze information from the wearable device (103). For example, the FoV information can include information about the FoV (402) of the wearable device (103). For example, the gaze information can include information indicating the position of the gaze of the user of the wearable device (103) and information about the range (401). For example, the FoV information and the gaze information can include an image. Accordingly, the external electronic device (570) can identify (or detect) the first region (410) and the second region (420).

[0182] The external electronic device (570) can map the identified first region (410), second region (420), and region (930). For example, the external electronic device (570) can synchronize the first region (410), second region (420), and region (930). Referring to FIG. 9A, the first region (410) and second region (420) can be included within the windshield (910) of the external electronic device (570). The example (900) of FIG. 9A illustrates a situation in which the user is driving, in which the first region (410) and second region (420) can be located within the windshield (910) located in front of the external electronic device (570). However, the embodiments of the present disclosure are not limited thereto. Additionally, the windshield (910) may be a narrower area than the recognizable area (930) of the external electronic device (570).

[0183] An external electronic device (570) can identify the context and object of the user of the wearable device (103). The object may be an object related to the context. For example, the object may be referenced as a notification object. The external electronic device (570) can identify the context by obtaining information about the context identified by the wearable device (103). However, the embodiments of the present disclosure are not limited thereto. For example, the external electronic device (570) may identify the context of the user based on information about the surroundings of the vehicle obtained by the external electronic device (570) through a sensor (573) and a communication circuit (571).

[0184] An external electronic device (570) can identify the object by receiving information about the object from a wearable device (103). However, the embodiments of the present disclosure are not limited thereto. For example, the external electronic device (570) can also identify the object based on information about the surroundings of the vehicle acquired by the external electronic device (570) through a sensor (573) and a communication circuit (571).

[0185] An external electronic device (570) can identify the location of the notification object. For example, the external electronic device (570) can identify that the location of the notification object is outside the FoV (402) and within the area (930).

[0186] The external electronic device (570) may provide a notification (or notification function) for the object based on identifying the location within the area (930). For example, the notification may include at least one of a visual object displayed through a display on the windshield (910) of the vehicle, vibration by the steering wheel of the vehicle, a visual object displayed through another display disposed within the vehicle, movement of a seat in which the user is positioned within the vehicle, or an acoustic provided through a speaker of the vehicle. The display on the windshield (910) may include a head-up display (HUD). For example, the other display disposed within the vehicle may include a display including a dashboard of the vehicle or a display for adjusting settings of the vehicle.

[0187] According to one embodiment, the external electronic device (570) may transmit a signal to the wearable device (103) to instruct the display of a visual object for notifying the object based on identifying the location within the area (930). For example, the signal may instruct the display of the visual object for notifying the object located within the area (930) in a portion of the FoV (402) of the wearable device (103). The wearable device (103) may display the object in the portion of the area based on receiving the signal. For example, the wearable device (103) may provide a third visual effect (e.g., the third visual effect (840) of FIG. 8A). For example, the third visual effect may include a rendered image corresponding to the object.

[0188] FIG. 9b illustrates an example of a wearable device that provides notification information based on information provided from a vehicle.

[0189] FIG. 9b illustrates examples (950, 960) of a method for providing notification information based on a signal received from an external electronic device (570) (e.g., the vehicle) of FIG. 9a.

[0190] Referring to example (950), the wearable device (103) may provide a navigation function. For example, the wearable device (103) may provide a visual object (953) indicating a route toward a destination based on the execution of an application for the navigation function. The wearable device (103) may identify a context indicating the user's situation based on identifying that the software application is running. For example, the context may include driving. For example, based on the identified context, the wearable device (103) may identify the visual object (953) indicating the route as an alarm object. In addition, a pedestrian (955) may be identified as an object related to driving (or the alarm object), but the wearable device (103) may not identify a pedestrian (955) outside of the FoV (402). At this time, the external electronic device (570) connected to the wearable device (103) can identify a pedestrian (955) located in an area (930) within a recognizable range (903). The external electronic device (570) can identify a first area (410) based on the FoV (402) of the wearable device (103) and the user's gaze, and transmit a signal to the wearable device (103) to instruct the display of a visual object (965) to notify the pedestrian (955).

[0191] Referring to example (960), in one embodiment, the wearable device (103) may receive a signal for indicating display of a visual object (965) from an external electronic device (570) connected to the wearable device (103). For example, the wearable device (103) may perform display of the visual object (965) based on the signal. At this time, the wearable device (103) may display the visual object (965) within the first region (410). For example, the first region (410) of example (960) may be different from the first region (410) of example (950). This is because the user's gaze may change in real time. Accordingly, the wearable device (103) can display a visual object (965) to notify a pedestrian (955) within the first area (410) where the user's gaze is located. For example, the visual object (965) can include text to notify the pedestrian (955). Referring to the above, the user can recognize information about a pedestrian (955) that becomes an alarm object for the driving while driving an external electronic device (570) while wearing the wearable device (103), and can drive more safely.

[0192] In FIG. 9B, a method for providing information to a user about an object (or alarm object) within an area (e.g., area (930)) that is difficult for the wearable device (103) to recognize based on the signal received from the external electronic device (570) is described. However, the embodiments of the present disclosure are not limited thereto. For example, even when not connected to the external electronic device (570), the wearable device (103) may identify an object within a second area (420) other than the first area (410) of the user, and display a visual object within the first area (410) for providing information about the object. For example, the wearable device (103) may identify a pedestrian located within the second area (420) of the FoV (402) as the alarm object based on the context of the user driving a vehicle. As in example (950), the wearable device (103) can identify a first area (410) on a visual object (953) indicating a path. At this time, the wearable device (103) can identify that information about the pedestrian other than the first area (410) needs to be provided. Thereafter, the wearable device (103) can identify the first area (410) on the changed location of example (960) and display a visual object (965) within the first area (410) to provide information about the pedestrian in the first area (410) on the changed location. Accordingly, the user wearing the wearable device (103) can recognize the pedestrian within the FoV (402) more quickly and drive more safely.

[0193] In one or more embodiments, the external electronic device may include an IoT device.

[0194] FIGS. 10A and 10B illustrate examples of a method for providing notification information based on the space in which an external electronic device is located.

[0195] FIG. 10A illustrates an example (1000) of a space where an external electronic device is located based on information provided from a server (580) connected to a wearable device (103). For example, the external electronic device may include an IoT device connected to the server (580). For example, the server (580) may represent an external electronic device that manages an IoT device. For example, the notification information may be an object (or notification object) related to a context indicating a situation of the user wearing the wearable device (103). For example, the context may include the user's control of the IoT device.

[0196] Referring to example (1000), a wearable device (103) may be connected to a server (580). For example, the wearable device (103) may be connected to the server (580) via a communication circuit (540). For example, the server (580) may be connected to a first IoT device (1003) and a second IoT device (1005). FIG. 10A illustrates an example of a server (580) connected to two IoT devices, but the embodiments of the present disclosure are not limited thereto. For example, the server (580) may be connected to one IoT device or three or more IoT devices. For example, the first IoT device (1003) may be located in the same space (1010) as the wearable device (103). In contrast, the second IoT device (1005) may be located in a space (1020) different from the space (1010) where the wearable device (103) is located.

[0197] The wearable device (103) can obtain information about the first IoT device (1003) and the second IoT device (1005) from the server (580). For example, the information about the first IoT device (1003) may include at least one of an image representing the appearance of the first IoT device (1003), a model name of the first IoT device (1003), a current location of the first IoT device (1003) (e.g., space (1010)), or an event occurring in the first IoT device (1003). For example, the information about the second IoT device (1005) may include at least one of an image representing the appearance of the second IoT device (1005), a model name of the second IoT device (1005), a current location of the second IoT device (1005) (e.g., space (1020)), or an event occurring in the second IoT device (1005).

[0198] The wearable device (103) can obtain a floor plan of the space where the wearable device (103) is located from the server (580). For example, the wearable device (103) can identify that the space (e.g., space (1010)) where the wearable device (103) is located and the space (e.g., space (1010)) where the first IoT device (1003) is located are the same, based on the obtained floor plan and the current location of the first IoT device (1003). In addition, for example, the wearable device (103) can identify that the space (e.g., space (1010)) where the wearable device (103) is located and the space (e.g., space (1020)) where the second IoT device (1005) is located are different based on the acquired floor plan and the current location of the second IoT device (1005). However, the embodiments of the present disclosure are not limited thereto. For example, the server (580) can identify whether the space (e.g., space (1010)) where the wearable device (103) is located and the space where the first IoT device (1003) (or the second IoT device (1005)) is located are the same or different based on the floor plan, location information of the first IoT device (1003), and location information of the second IoT device (1005) stored by the server (580). The server (580) may transmit information about the identity of the identified space to the wearable device (103). For example, the event generated in the first IoT device (1003) or the second IoT device (1005) may include trigger identification for executing, stopping, or terminating a specific function, or executing a new function. However, the embodiments of the present disclosure are not limited thereto.

[0199] FIG. 10B illustrates an example (1050) in which the user recognizes the first IoT device (1003) or the second IoT device (1005) through the wearable device (103). The example (1050) illustrates an example in which the first IoT device (1003) is located within the first area (410) based on the user's line of sight identified by the wearable device (103), but the embodiments of the present disclosure are not limited thereto. For example, the first IoT device (1003) may be located within the second area (420) or the third area (430) even if it is located within the same space (1010) as the wearable device (103). The embodiments of the present disclosure may also be applied to a case in which the first IoT device (1003) is located within the second area (420) or the third area (430) within the same space (1010) as the wearable device (103).

[0200] Referring to example (1050), the wearable device (103) may display a first IoT device (1003) within a first region (410). The wearable device (103) may provide a first visual effect (1060) to the first IoT device (1003), which is an object related to the user's context. For example, the first visual effect (1060) may include displaying a first visual object. For example, the first visual object may include a line corresponding to the periphery of the first IoT device (1003) and having a specified thickness. For example, the first visual object may be at least partially superimposed on the first IoT device (1003). However, the first visual effect (1060) according to an embodiment of the present disclosure is not limited to the first visual object.

[0201] Additionally, referring to example (1050), the wearable device (103) can identify a second IoT device (1005) within a third area (430). For example, the wearable device (103) can identify a second IoT device (1005) located in a space (1020) within the third area (430). The wearable device (103) can provide a third visual effect (1065) based on identifying that the second IoT device (1005), which is an object related to the user's context, is located within the third area (430) outside the FoV (402). For example, the third visual effect (1065) can include a display of a second visual object. For example, the second visual object can include a rendered image representing the second IoT device (1005). The wearable device (103) may display the second visual object, which is a rendered image for the second IoT device (1005), through a portion (1070) of the FoV (402). For example, the portion (1070) may represent an area within the FoV (402) that is adjacent to an area outside the FoV (402) where the second IoT device (1005) is located. In the example (1050) of FIG. 10b, the portion (1070) may include the upper right. In this case, the second IoT device (1005) may be located within a space (1020) located in the upper right direction with respect to the space (1010) where the user is located. The third visual effect (1065) may include displaying the second visual object and displaying a visual object (1075) rendered for the space (1020). For example, the visual object (1075) rendered for space (1020) may be a background image for the second visual object. However, the third visual effect (1065) according to the embodiment of the present disclosure is not limited to the second visual object.For example, the third visual effect (1065) may further include, in addition to the display of the second visual object, the display of an indicator to indicate the direction in which the second IoT device (1005) is located.

[0202] Referring to FIGS. 10A and 10B , the wearable device (103) can identify an IoT device as an object related to the context through a connected server (580). The wearable device (103) can identify the location of the IoT device based on information obtained from the server (580). The wearable device (103) can provide a visual effect according to the location of the IoT device, thereby increasing the user's awareness of the object.

[0203] Figure 11 illustrates an example of an operational flow for a method of providing notification information based on the location of an object.

[0204] At least some of the above methods of FIG. 11 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the above methods may be controlled by the processor (510) of the wearable device (103).

[0205] Referring to FIG. 11, in operation (1110), the wearable device (103) can identify a context representing a user's situation, such as, among other things, an action in which the user participates. The user can represent a user wearing the wearable device (103). For example, the context can define the user's action or situation. For example, the context can be referred to as a routine, situational information, an action in which the user participates, or a behavior pattern. In other words, the context can be defined so that the wearable device (103) can interpret the user's action or situation and provide the user with information or services required. For example, the context can include the user walking, the user using public transportation, the user driving, the user watching a performance, the user ordering a menu at a restaurant, the user cooking, or the user resting. However, the embodiments of the present disclosure are not limited thereto.

[0206] Although not illustrated in FIG. 11, the wearable device (103) can execute an application that provides augmented reality. For convenience of explanation, the following description will focus on augmented reality, but embodiments of the present disclosure can also be applied to virtual reality or mixed reality. For example, the wearable device (103) can identify whether a function for providing notification information based on the location of the object is set while the software application is running. For example, the function for providing notification information based on the location of the object can be activated or deactivated in the global settings of the wearable device (103) or in the settings within the software application. In other words, when the function is activated, the wearable device (103) can provide the notification information based on the location of the object, as described below.

[0207] The wearable device (103) can identify a first region based on the user's gaze. For example, the wearable device (103) can perform calibration to identify the first region. The calibration can include a process for identifying the first region based on the direction of the user's gaze. Within the calibration, the wearable device (103) can display a visual object at an arbitrary location within a peripheral region centered on the direction (or point) of the gaze. The visual object can be referred to as a calibration object. For example, the wearable device (103) can obtain a user's input for the calibration object. For example, based on the input, the wearable device (103) can identify a range recognizable by the user with respect to the direction of the gaze. For example, the range can represent a boundary of the first region. For example, the area within the range can be referred to as the first region. The size of the first region may be changed based on information obtained through the sensor (520) and / or the communication circuit (540). For example, the size of the first region may be changed based on biometric information contained in the information or information indicating whether the user is moving. As specific details related to this, the contents described in FIG. 7b may be practically applied.

[0208] The wearable device (103) can obtain information through the sensor (520) and the communication circuit (540). For example, the wearable device (103) can obtain the information through the sensor (520) and / or the communication circuit (540). For example, the information can be used to identify a context indicating the user's situation or an object within a range recognizable by the wearable device (103).

[0209] The wearable device (103) can obtain an image of a surrounding area of ​​a user wearing the wearable device (103) based on a sensor (520). For example, the wearable device (103) can obtain an image of the surrounding area through a camera included in the sensor (520). For example, the surrounding area can represent an omnidirectional view of the user. The wearable device (103) can identify at least one object within the image based on the image of the surrounding area.

[0210] The wearable device (103) can obtain acoustic information about the surrounding area of ​​the user based on the sensor (520). For example, the wearable device (103) can obtain the acoustic information through the microphone included in the sensor (520). For example, the acoustic information can include the user's voice. Based on the acoustic information, the wearable device (103) can identify the voice spoken by the user or a word (or sentence) required by the user.

[0211] The wearable device (103) can obtain the user's biometric information based on the sensor (520). For example, the wearable device (103) can obtain the biometric information based on the biometric sensor included in the sensor (520). For example, the biometric information may include at least one of the user's heart rate, body fat information, blood pressure, or pupil size. The wearable device (103) can identify the user's context by identifying the user's health status based on the biometric information. The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the biometric information.

[0212] According to one embodiment, the wearable device (103) can identify the user's moving direction, moving speed, or direction toward which the user's head is directed based on the sensor (520). For example, the wearable device (103) can identify the user's moving direction, moving speed, or direction toward which the user's head is directed based on an IMU sensor included in the sensor (520). The wearable device (103) can identify the first area (or user gaze area) based on the user's gaze based on the user's moving direction, moving speed, or direction toward which the user's head is directed.

[0213] The wearable device (103) can identify location information of the wearable device (103) based on the sensor (520). For example, the wearable device (103) can identify the location information of the wearable device (103) based on GPS information. Alternatively, for example, the wearable device (103) can identify current time information, temperature or air pressure information around the wearable device (103) based on the sensor (520).

[0214] The wearable device (103) can obtain communication information from another external electronic device connected through the communication circuit (540). For example, the communication circuit (540) can include a WiFi module or a BT module. For example, the other external electronic device can include a router connected through the WiFi module or a beacon connected through the BT module. For example, the communication information can include information about electronic devices connected to the router or the beacon. For example, the communication information can include the location of the electronic devices, information of the electronic devices (e.g., type, model name, etc.), information used by the electronic devices (e.g., traffic information, location information, etc.), and the location of the wearable device (103).

[0215] In operation (1110), the wearable device (103) can identify a context representing the user's situation. For example, the wearable device (103) can identify the context representing the user's situation based on the information acquired through the sensor (520) and the communication circuit (540). The wearable device (103) can identify the context further based on the software application that the user is using through the wearable device (103). For example, if the user is using a map application, the wearable device (103) can identify the context as being related to movement. Alternatively, for example, if the user is using a public transportation search application, the wearable device (103) can identify the context as being related to using the public transportation. For example, if the user is using a recipe providing application, the context can be identified as being related to cooking.

[0216] In operation (1120), the wearable device (103) can identify the location of an object related to the context. The object related to the context can be identified from among at least one object within a recognizable range of the wearable device (103) identified based on the information. For example, the wearable device (103) can identify the location of the object related to the identified context. The object related to the context can indicate an object that is a target of notification information provision from among the at least one object. The object related to the context can be referred to as an alarm object, an object of interest, a point of interest (POI), a visual element related to the context, a visual object related to the context, a graphical object related to the context, or a target object.

[0217] The wearable device (103) may identify an object (i.e., an alarm object) related to the context among the surrounding objects based on the identified context representing the user's situation. For example, if the context is walking or moving toward a specific destination via public transportation, the alarm object may include the public transportation to be boarded, the destination, obstacles, traffic lights, crosswalks, and external objects with a risk of collision (e.g., bicycles, etc.). Alternatively, if the context is driving a vehicle, the alarm object may include traffic lights, a crosswalk situation nearby along the route, and the vehicle's blind spot. Alternatively, if the context is watching a performance such as a musical or play, the object may include an actor, a stage area, or an actor's lines. Alternatively, if the context is ordering a menu at a restaurant, the object may include the completion of the order or a waiter, etc.

[0218] The wearable device (103) can identify the priority of the alarm object. For example, the wearable device (103) can identify the priority of the alarm object based on the context. For example, the priority can indicate an emergency level (or urgency) that defines the order in which notification information is to be provided among alarm objects related to the context. According to one embodiment, the wearable device (103) can display text describing the alarm object within the FoV of the wearable device (103) if the priority of the alarm object is equal to or higher than a reference level. Specific details related thereto can be substantially identically applied to the contents described in FIG. 8B.

[0219] In one embodiment, the wearable device (103) may identify an available notification function along with notification information for the alarm object. For example, the notification function may be different from the notification information displayed through the display (530) of the wearable device (103) and may include an additional action performed along with the display of the notification information. For example, if the wearable device (103) is connected to a vehicle, the notification function may include vibration of the steering wheel of the vehicle or a change in seat position. However, the embodiments of the present disclosure are not limited thereto. For example, the notification function may include vibration of the wearable device (103) or blinking of a light-emitting module included in the wearable device (103).

[0220] In operation (1130), the wearable device (103) may display a first visual object that at least partially overlaps the object and overlaps the periphery of the object based on verifying that the object is located within the first region corresponding to the gaze direction of the user within the field of view (FoV) region of the wearable device (103). For example, the wearable device (103) may provide a first visual effect based on identifying that the location of the object related to the context is within the first region. The first visual effect may include displaying the first visual object.

[0221] For example, the first visual object may include a line corresponding to the periphery of the alarm object and having a specified thickness. For example, the first visual object may be at least partially superimposed on the alarm object. However, the first visual effect according to the embodiment of the present disclosure is not limited to the first visual object.

[0222] In operation (1140), the wearable device (103) may display the first visual object that at least partially overlaps the object by blinking based on determining that the object is located within the second region excluding the first region within the FoV region of the wearable device (103). For example, the wearable device (103) may provide a second visual effect based on identifying that the location of the object related to the context is within the second region. For example, the FoV region may include the first region and the second region. For example, the second region may represent at least a portion of the FoV region of the wearable device (103) that is distinct from the first region. The second region may be referred to as a display region of the wearable device (103).

[0223] For example, the second visual effect may include blinking of the first visual object. For example, the first visual object of the second visual effect may include a line corresponding to the periphery of the alarm object and having a specified thickness. For example, the first visual object of the second visual effect may be at least partially superimposed on the alarm object. In this case, the first visual object of the second visual effect may blink a specified number of times. However, the second visual effect according to the embodiment of the present disclosure is not limited to displaying and blinking the first visual object.

[0224] In operation (1150), the wearable device (103) may display a second visual object representing the object at a point within the FoV area based on confirming that the object is located within a third area outside the FoV area. For example, the third area may represent an area outside the FoV area of ​​the wearable device (103) and may represent an area within an identifiable range based on information acquired by the wearable device (103). The third area may be referred to as a recognition area of ​​the wearable device (103) or an identification area of ​​the wearable device (103).

[0225] The third visual effect may include displaying the second visual object. For example, the second visual object may include a rendered image representing the alarm object. For example, the wearable device (103) may display the second visual object, which is a rendered image of the alarm object, through a point in the FoV area. For example, the point may represent an area within the FoV that is adjacent to an area outside the FoV where the alarm object is located. For example, the point may be referred to as a corner, an edge, a corresponding area of ​​the FoV, an adjacent area, or an area of ​​interest. However, the third visual effect according to an embodiment of the present disclosure is not limited to the second visual object. For example, the third visual effect may further display an indicator for indicating the direction in which the alarm object is located, in addition to displaying the second visual object.

[0226] Although not shown in FIG. 11, the wearable device (103) can identify whether the location of the object changes within the first area. For example, the wearable device (103) can identify whether the location within the second area or the third area changes within the first area. For example, the wearable device (103) can identify whether the user's gaze changes to the location of the alarm object while the second visual effect or the third visual effect is displayed. This is because the second visual effect and the third visual effect are visual effects used to make the user pay attention to the alarm object. The wearable device (103) can stop displaying the second visual effect (or the third visual effect) based on whether the location of the object changes within the first area. According to one embodiment, the wearable device (103) may lower the brightness of the second region based on determining that the location of the alarm object has not changed within the first region. Since the user's attention is not attracted by the display of the second visual effect (or the third visual effect), the wearable device (103) may lower the brightness of the second region. For example, the wearable device (103) may change the brightness of the second region from a first brightness to a second brightness lower than the first brightness. Changing the brightness of the second region may cause the user to look at the alarm object.

[0227] Figure 12 illustrates an example of an operational flow for providing notification information based on the location of an object and transmitting a signal to an external electronic device.

[0228] At least some of the methods of FIG. 12 may be performed by the external electronic device (570) of FIG. 5. For example, at least some of the methods may be controlled by the processor (577) of the external electronic device (570). For example, the external electronic device (570) may include a vehicle, and the context may include driving by the user.

[0229] Referring to FIG. 12, in operation (1210), an external electronic device (570) may receive FoV information of a wearable device (103) and gaze information of a user. For example, the external electronic device (570) may be connected to the wearable device (103). For example, the user may be wearing the wearable device (103) and riding the external electronic device (570). For example, the FoV information may include information about the FoV (e.g., the FoV (402) of FIG. 9A) (or FoV area) of the wearable device (103). For example, the gaze information may include information indicating the position of the user's gaze of the wearable device (103) and information about a range recognizable by the user (e.g., the range (401) of FIG. 9A). For example, the FoV information and the gaze information may include an image.

[0230] In operation (1220), the external electronic device (570) can identify the FoV of the wearable device (103) with respect to the gaze. For example, the external electronic device (570) can identify the FoV of the wearable device (103) with respect to the gaze based on the FoV information and the gaze information. Accordingly, the external electronic device (570) can identify (or detect) a first region based on the gaze (e.g., the first region (410) of FIG. 9A) and a second region of the FoV (e.g., the second region (420) of FIG. 9A).

[0231] In operation (1230), the external electronic device (570) can identify a recognition area other than the FoV. For example, the external electronic device (570) can obtain information about the surroundings of the external electronic device (570) through the sensor (573) and the communication circuit (571). The information about the surroundings of the external electronic device (570) can be referenced as information about the surroundings of the vehicle. For example, the external electronic device (570) can identify an area (i.e., the recognition area) (e.g., area (930) of FIG. 9A) that the external electronic device (570) can recognize based on the information about the surroundings.

[0232] The external electronic device (570) can map the first area, the second area, and the recognition area. For example, the external electronic device (570) can synchronize the first area, the second area, and the recognition area. For example, the first area and the second area can be included in a windshield of the external electronic device (570) (e.g., the windshield (910) of FIG. 9A). While the user is driving, the first area and the second area can be located within the windshield located in front of the external electronic device (570). However, the embodiments of the present disclosure are not limited thereto.

[0233] In operation (1240), the external electronic device (570) may transmit a signal to the wearable device (103) that provides a notification for an object and instructs the display of a visual object to notify the object. For example, the external electronic device (570) may provide a notification for the object based on identifying an object related to a context representing the user's situation. Additionally, the external electronic device (570) may transmit the signal to the wearable device (103) based on identifying an object related to a context representing the user's situation.

[0234] An external electronic device (570) can identify a context representing the user's situation and an object related to the context. For example, the context may define the user's action or situation. For example, the context may be referred to as a routine, situational information, or behavioral pattern. For example, the context may be defined so that the wearable device (103) can interpret the user's action or situation and provide the user with information or services required by the user.

[0235] An external electronic device (570) can identify the context by obtaining information about the context identified by the wearable device (103). However, the embodiments of the present disclosure are not limited thereto. For example, the external electronic device (570) can also identify the user's context based on information about the surroundings obtained by the external electronic device (570) through a sensor (573) and a communication circuit (571).

[0236] An external electronic device (570) can identify an object by receiving information about the object from a wearable device (103). However, the embodiments of the present disclosure are not limited thereto. For example, the external electronic device (570) can also identify the object based on information about the surroundings acquired by the external electronic device (570) through a sensor (573) and a communication circuit (571).

[0237] An external electronic device (570) can identify the location of the object in relation to the context. For example, the external electronic device (570) can identify that the location of the object is outside the FoV and within the recognition area.

[0238] An external electronic device (570) may provide a notification for the object based on identifying the location within the recognition area. For example, the notification may include at least one of a visual object displayed through a display on the windshield, vibration caused by a handle of the vehicle, a visual object displayed through another display disposed within the vehicle, movement of a seat in which the user is positioned within the vehicle, or an acoustic provided through a speaker of the vehicle. For example, the display on the windshield may include a head-up display (HUD). For example, the other display disposed within the vehicle may include a display including a dashboard of the vehicle or a display for adjusting settings of the vehicle.

[0239] According to one embodiment, the external electronic device (570) may transmit a signal to the wearable device (103) to instruct the display of a visual object for notifying the object based on identifying the location within the recognition area. For example, the signal may instruct the wearable device (103) to display the visual object for notifying the object located within the recognition area in a portion of the FoV. The wearable device (103) may display the object in the portion of the area based on receiving the signal. For example, the wearable device (103) may provide a third visual effect (e.g., the third visual effect (840) of FIG. 8A). For example, the third visual effect may include a rendered image corresponding to the object.

[0240] As described above, the wearable device (103) may include a display (530). The wearable device (103) may include a sensor (520). The wearable device (103) may include a communication circuit (540). The wearable device (103) may include a memory (550) that stores instructions and includes one or more storage media. The wearable device (103) may include at least one processor (510) that includes a processing circuit. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a context representing a situation of a user wearing the wearable device (103) based on information acquired through the sensor (520) and the communication circuit (540). The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a location of an object related to the context based on the information. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display a first visual object that is at least partially superimposed on and overlaps a periphery of the object based on determining that the object is located within a first region corresponding to a gaze direction of the user within a field of view (FoV) region of the wearable device (103).The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display, by blinking, the first visual object at least partially overlapping the object based on determining that the object is located within a second region excluding the first region within the FoV region. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display, at a point within the FoV region, a second visual object representing the object based on determining that the object is located within a third region outside the FoV region.

[0241] According to one embodiment, the information obtained through the sensor (520) and the communication circuit (540) may include at least one of an image of a surrounding area of ​​the wearable device (103), acoustic information, information about an external electronic device connected based on short-range wireless communication, biometric information of the user, or movement information of the user.

[0242] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify the context representing the situation of the user based on a running software application and the information. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify the object associated with the context based on the information.

[0243] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify the gaze direction indicated by the user's gaze. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display, through the display (530), at least one calibration object displayed adjacent to the gaze direction. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify the first area based on the user's input with respect to the at least one calibration object.

[0244] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a movement of the user based on the information. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a size of the first area as a first size based on the movement having a first movement speed. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a size of the first area as a second size smaller than the first size based on the movement having a second movement speed that is faster than the first movement speed.

[0245] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a size of a pupil of the user based on the information. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a size of the first region as a first size based on a pupil having a first pupil size. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a size of the first region as a second size greater than the first size based on a second pupil size greater than the first pupil size.

[0246] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to perform hand tracking on the user based on the information. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a subregion corresponding to the user's hand within the first region based on the hand tracking. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to refrain from displaying the first visual object for the object based on the identification of the location within the subregion.

[0247] In one embodiment, the first visual object may include a line having a specified thickness representing the edge of the object. The second visual object may include an image in which the object is rendered. The point of the FoV may represent a portion of the FoV area adjacent to the third area.

[0248] According to one embodiment, the third area may include an area recognizable by the wearable device (103) based on the information.

[0249] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display text describing the object at another point in the FoV area while displaying the second visual object representing the object based on determining that the object is located within the third area. The other point may include the first area and the second area.

[0250] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to output, through a speaker of the wearable device (103), a voice uttering text describing the object while displaying the second visual object representing the object based on determining that the object is located within the third area.

[0251] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify a priority for the object based on the context. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display text describing the object at another point in the FoV area based on the object having the priority greater than or equal to a reference level. The priority may indicate an urgency of the object. The other point may include the first area.

[0252] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to blink the first visual object for a specified period of time based on determining that the object is located at the location within the second region or within the third region. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the brightness of the second region in response to determining that the object is located within the second region or within the third region after the first visual object has blinked for the specified period of time.

[0253] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to transmit a signal to an external electronic device (570) requesting execution of a function of the external electronic device (570) to adjust the location within the first area based on determining that the object is located within the second area or within the third area.

[0254] In one embodiment, the object may include an Internet of Things (IoT) device. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to connect to a server (580) that manages the IoT device via the communication circuit (540). The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to receive information about the IoT device from the server (580). The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to identify, based on the information about the IoT device, whether the location of the object is within a space in which the wearable device (103) is located. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display the second visual object on the point in the FoV area based on identifying the location within the third area, if the location is within the space.

[0255] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to display the second visual object on the point in the FoV area together with an image representing another space in which the location of the object is contained, if the location is outside the space.

[0256] As described above, the method performed by the wearable device (103) may include an operation of identifying a context representing a situation of a user wearing the wearable device (103) based on information acquired through a sensor (520) and a communication circuit (540) of the wearable device (103). The method may include an operation of identifying a location of an object related to the context based on the information. The method may include an operation of displaying a first visual object that at least partially superimposed on and corresponding to a periphery of the object based on identifying the location within a first region based on a gaze of the user. The method may include an action of blinking the first visual object, which at least partially overlaps the object, based on identifying the location within a second region of the field of view (FoV) of the wearable device (103), which is distinct from the first region of the FoV. The method may include an action of displaying a second visual object representing the object, on a portion of the FoV, based on identifying the location within a third region outside the FoV.

[0257] As described above, the non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by a processor (510) of a wearable device (103) including a display (530), a sensor (520), and a communication circuit (540), cause the wearable device (103) to identify a context representing a situation of a user wearing the wearable device (103) based on information obtained through the sensor (520) and the communication circuit (540). The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (510), cause the wearable device (103) to identify a location of an object related to the context based on the information. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (510), cause a first visual object to be displayed that is at least partially superimposed on the object and corresponds to a periphery of the object based on identifying the location within a first area based on the gaze of the user. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (510), cause the first visual object that is at least partially superimposed on the object to blink based on identifying the location within a second area of ​​the field of view (FoV) of the wearable device (103), distinct from the first area of ​​the FoV.The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (510), cause a second visual object representing the object to be displayed on a portion of the FoV based on identifying the location within a third area outside the FoV.

[0258] As described above, the vehicle (570) may include a sensor (573). The vehicle (570) may include a communication circuit (571). The vehicle (570) may include a memory (579) that stores instructions and includes one or more storage media. The vehicle (570) may include at least one processor (577) that includes a processing circuit. The instructions, when individually or collectively executed by the at least one processor (577), may cause the vehicle (570) to receive FoV information of a wearable device (103) worn by a user of the vehicle (570) connected through the communication circuit (571) and gaze information of the user. The instructions, when individually or collectively executed on the at least one processor (577), may cause the vehicle (570) to identify an FoV of the wearable device (103) with respect to the gaze based on the FoV information and the gaze information. The instructions, when individually or collectively executed on the at least one processor (577), may cause the vehicle (570) to identify a recognition area of ​​the vehicle (570) other than the FoV based on information acquired through the sensor (573) and the communication circuit (571).The above instructions, when individually or collectively executed on the at least one processor (577), may cause the vehicle (570) to identify an object related to a context representing a situation of the user within the recognition area, and to transmit a signal to the wearable device (103) indicating the display of a visual object to provide a notification for the object.

[0259] According to one embodiment, the notification for the object may include at least one of a visual object displayed through a display (575) on a windshield of the vehicle (570), a vibration performed through a handle of the vehicle (570), a visual object displayed through another display (575) disposed within the vehicle (570), a movement of a seat positioned within the vehicle (570), or an acoustic provided through a speaker of the vehicle (570).

[0260] In one embodiment, the signal may instruct the wearable device (103) to display a visual object to indicate the object located within the recognition area, on a portion of the FoV.

[0261] In one embodiment, the instructions, when individually or collectively executed on the at least one processor (577), may cause the vehicle (570) to identify, based on the FoV information and the gaze information, a first region based on the gaze of the user and a second region of the FoV distinct from the first region of the FoV. The instructions, when individually or collectively executed on the at least one processor (577), may cause the vehicle (570) to identify, based on the information, the first region, and the second region, the recognition region outside the FoV.

[0262] As described above, the method performed by the vehicle (570) may include an operation of receiving FoV information of the wearable device (103) and gaze information of the user of the vehicle (570) from a wearable device (103) worn by a user of the vehicle (570) connected through a communication circuit (571). The method may include an operation of identifying an FoV of the wearable device (103) with respect to the gaze based on the FoV information and the gaze information. The method may include an operation of identifying a recognition area of ​​the vehicle (570) other than the FoV based on information acquired through a sensor (573) and a communication circuit (571). The method may include an operation of providing a notification for an object based on identifying an object related to a context representing a situation of the user within the recognition area, and transmitting a signal to the wearable device (103) indicating the display of a visual object for notifying the object.

[0263] As described above, the non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by a processor (577) of a vehicle (570) including a sensor (573) and a communication circuit (571), cause the processor to receive FoV information of the wearable device (103) and gaze information of the user from a wearable device (103) worn by a user of the vehicle (570) connected through the communication circuit (571). The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (577), cause the processor to identify, with respect to the FoV of the wearable device (103), based on the FoV information and the gaze information. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (577), cause the processor to identify a recognition area of ​​the vehicle (570) other than the FoV based on information acquired through the sensor (573) and the communication circuit (571). The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the processor (577), cause the processor to identify an object related to a context representing a situation of the user within the recognition area, provide a notification for the object, and transmit a signal to the wearable device (103) indicating the display of a visual object for notifying the object.

[0264] Electronic devices according to one or more embodiments disclosed herein may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments disclosed herein are not limited to the aforementioned devices.

[0265] It should be understood that one or more embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but rather to encompass various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly dictates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0266] The term "module" as used in one or more embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0267] One or more embodiments may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0268] According to one embodiment, a method according to one or more embodiments disclosed herein may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0269] According to one or more embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one or more embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, according to one or more embodiments, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one or more embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable device (103), display (530); sensor (520); Communication circuit (540); A memory (550) storing instructions and including one or more storage media; and At least one processor (510) comprising a processing circuit, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on the information acquired through the above sensor (520) and the communication circuit (540), a context indicating a situation of a user wearing the wearable device (103) is identified; Identifying the location of an object related to the context based on the above information; Based on confirming that the object is located within a first area corresponding to the gaze direction of the user within the FoV (field of view) area of ​​the wearable device (103), a first visual object that is at least partially superimposed on the object is displayed, Based on the confirmation that the object is located within a second region excluding the first region within the FoV region, the first visual object is displayed by blinking, and Based on determining that the object is located within a third region outside the FoV region, causing a second visual object representing the object to be displayed at a point within the FoV region. Wearable device (103).

2. In claim 1, The information obtained through the above sensor (520) and the above communication circuit (540) is: Containing at least one of an image of the surrounding area of ​​the wearable device (103), acoustic information, information about an external electronic device connected based on short-range wireless communication, biometric information of the user, or movement information of the user. The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Identifying said context representing said situation of said user based on said running software application and said information; and Based on the above information, causing the object to be identified in relation to the above context, Wearable device (103).

3. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Identify the direction of gaze indicated by the user's gaze; Displaying at least one calibration object adjacent to the above-mentioned viewing direction through the display (530); and causing said first region to be identified based on said user input for said at least one calibration object; Wearable device (103).

4. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Identifying the user's movement based on the above information; Based on the movement having the first movement speed, the size of the first area is identified as the first size; and Based on the movement having a second movement speed that is faster than the first movement speed, causing the size of the first area to be identified as a second size smaller than the first size, Wearable device (103).

5. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Identifying the size of the user's pupil based on the above information; Based on the pupil having the first pupil size, the size of the first region is identified as the first size; and Based on the second pupil size being larger than the first pupil size, causing the size of the first area to be identified as a second size being larger than the first size, Wearable device (103).

6. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Perform hand tracking on the user based on the above information; Based on the hand tracking, identifying a sub-region corresponding to the user's hand within the first region; and Based on identifying said location within said sub-region, causing said first visual object to be refrained from displaying said object; Wearable device (103).

7. In claim 1, The first visual object comprises a line having a specified thickness representing an edge of the object, The second visual object comprises an image rendered based on the object, The above point of the above FoV represents a part of the FoV area adjacent to the third area, and The third area includes an area that the wearable device (103) can recognize based on the information. Wearable device (103).

8. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: While displaying the second visual object representing the object based on confirmation that the object is located within the third area, cause text describing the object to be displayed at another point in the FoV area; The above other point includes the first area and the second area, Wearable device (103).

9. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Causing a voice to be output through a speaker of the wearable device (103) that utters text describing the object while displaying the second visual object representing the object based on confirmation that the object is located within the third area. Wearable device (103).

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on the above context, identify the priority for the object; and Based on the object having the above priority level greater than or equal to the reference level, cause text describing the object to be displayed at another point in the FoV area, The above priority indicates the urgency of the object, The above other point includes the first area, Wearable device (103).

11. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on determining that the object is located within the second area or within the third area, blinking the first visual object for a specified period of time; and causing the brightness of the second area to change in response to determining that the object is located within the second area or within the third area after the first visual object has blinked for the specified time; Wearable device (103).

12. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on the determination that the object is located within the second area or within the third area, a signal is transmitted to the external electronic device (570) requesting execution of a function of the external electronic device (570) to adjust the location within the first area. Wearable device (103).

13. In claim 1, The above object includes an IoT (internet of things) device, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Through the above communication circuit (540), connect to a server (580) that manages the IoT device; Receive information about the IoT device from the server (580); Based on the information about the IoT device, identifying whether the location of the object is within the space where the wearable device (103) is located; and If the location is within the space, causing the second visual object to be displayed on the point in the FoV area based on identifying the location within the third area. Wearable device (103).

14. In a method performed by a wearable device (103), the method comprises: An operation of identifying a context representing a situation of a user wearing the wearable device (103) based on information acquired through a sensor (520) and a communication circuit (540) of the wearable device (103); An action of identifying the location of an object related to the context based on the above information; An operation of displaying a first visual object at least partially superimposed on the object based on confirming that the object is located within a first area corresponding to a gaze direction of the user within a field of view (FoV) area of ​​the wearable device (103); An operation of displaying the first visual object by blinking based on confirmation that the object is located within a second area excluding the first area within the FoV area, and Based on determining that the object is located within a third area outside the FoV area, an action is included to display a second visual object representing the object at a point within the FoV area. method.

15. A non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor (510) of a wearable device (103) including a display (530), a sensor (520), and a communication circuit (540): Based on the information acquired through the above sensor (520) and the communication circuit (540), a context indicating a situation of a user wearing the wearable device (103) is identified; Identifying the location of an object related to the context based on the above information; Based on confirming that the object is located within a first area corresponding to the gaze direction of the user within the FoV (field of view) area of ​​the wearable device (103), a first visual object that is at least partially superimposed on the object is displayed, Based on the confirmation that the object is located within a second region excluding the first region within the FoV region, the first visual object is displayed by blinking, and storing one or more programs including instructions that cause a second visual object representing the object to be displayed at a point within the FoV area, based on determining that the object is located within a third area outside the FoV area; A non-transitory computer-readable storage medium.

Images