Electronic device, method, and computer-readable storage medium for controlling external electronic device to output vibrations

Abstract

A wearable device may include a communication circuit, a display, and at least one processor. The wearable device can display a graphic effect, which spreads out into an external space from a first location spaced apart from the wearable device, on the display in response to an event that occurs at the first location and causes the output of vibrations. The wearable device can determine the distance between the first location and a second location of an external electronic device spaced apart from the wearable device. The wearable device, on the basis of determining the distance, can identify the point in time at which at least a portion of the graphic effect intersects with the second location. The present document may relate to a metaverse service for enhancing the interconnectivity between a real object and a virtual object. For example, the metaverse service may be provided over a fifth generation (5G) and / or sixth generation (6G)-based network.

Description

Electronic device, method, and computer-readable storage medium for controlling an external electronic device to output vibration

[0001] The present disclosure relates to an electronic device, a method, and a computer-readable storage medium for controlling an external electronic device to output vibration.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide extended reality (XR) services, which display computer-generated information in conjunction with external objects in the real world. These electronic devices can provide XR services to users by utilizing virtual objects that correspond to the user.

[0003] In one embodiment, a wearable device may include a communication circuit, a display, at least one processor including a processing circuit, and a memory storing instructions and including one or more storage media. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display, on the display, a graphic effect that extends from the first location into external space, in response to an event occurring at a first location spaced apart from the wearable device and causing an output of a vibration. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a distance between the first location and a second location of an external electronic device spaced apart from the wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a point in time at which at least a portion of the graphic effect intersects the second location, based on the identified distance. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to transmit, through the communication circuit, a signal to the external electronic device to control an actuator of the external electronic device to output a vibration having a magnitude determined based on the distance at the point in time.

[0004] In one embodiment, a method of a wearable device may include an operation of displaying, on a display, a graphic effect that spreads from the first location to an external space in response to an event that occurs at a first location spaced apart from the wearable device and causes an output of a vibration. The method may include an operation of identifying a distance between the first location and a second location of an external electronic device spaced apart from the wearable device. The method may include an operation of identifying a point in time at which at least a portion of the graphic effect intersects the second location based on the identified distance. The method may include an operation of transmitting, through a communication circuit, a signal for controlling an actuator of the external electronic device such that a vibration having a magnitude determined based on the distance is output at the point in time.

[0005] A computer-readable storage medium storing instructions according to one embodiment, wherein the instructions, when executed by a processor of a wearable device, cause the wearable device to display, on a display, a graphic effect that spreads from the first location into external space in response to an event occurring at a first location spaced apart from the wearable device and causing an output of a vibration. The instructions, executed by the processor, cause the wearable device to identify a distance between the first location and a second location of an external electronic device spaced apart from the wearable device. The instructions, executed by the processor, cause the wearable device to identify a point in time when at least a portion of the graphic effect intersects the second location based on the identified distance. The instructions executed by the processor may cause the wearable device to transmit, through a communication circuit, a signal to the external electronic device to control an actuator of the external electronic device to output a vibration having a magnitude determined based on the distance at the point in time.

[0006] FIGS. 1A and 1B illustrate an example of a block diagram of a wearable device and an external electronic device according to one embodiment.

[0007] FIG. 2A illustrates an example of a perspective view of a wearable device, according to one embodiment.

[0008] FIG. 2b illustrates an example of one or more hardware elements disposed within a wearable device, according to one embodiment.

[0009] FIGS. 3A and 3B illustrate an example of an appearance of a wearable device according to one embodiment.

[0010] FIG. 4 illustrates an example of an operation of a wearable device according to one embodiment to control an external electronic device based on a distance between a location of an event and a location of the external electronic device.

[0011] FIG. 5 illustrates an example of an operation of a wearable device according to one embodiment to control an external electronic device based on an event.

[0012] FIG. 6 illustrates an example of a graphic effect displayed by a wearable device based on an event according to one embodiment.

[0013] FIG. 7 illustrates an example of a change in vibration intensity output from an external electronic device according to one embodiment.

[0014] FIG. 8 illustrates an example of a virtual space including a plurality of wearable devices according to one embodiment.

[0015] FIG. 9 illustrates an example of a screen for controlling the vibration intensity of an external electronic device according to one embodiment.

[0016] FIG. 10 illustrates an example of a flowchart illustrating the operation of a wearable device according to one embodiment.

[0017] Figure 11 is an example diagram of a network environment in which a metaverse service is provided through a server according to one embodiment.

[0018] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0019] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0020] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0021] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., the processor (120)) of the machine (e.g., the electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0022] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0023] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0024] Figures 1A and 1B illustrate block diagrams of a wearable device and an external electronic device according to one embodiment. In one embodiment, the wearable device (101) may be referred to as a terminal (or user terminal) in terms of being owned by a user. The terminal may include, for example, a personal computer (PC) such as a laptop or desktop. The terminal may include, for example, a smartphone, a smartpad, and / or a tablet PC. The terminal may include a smart accessory such as a smartwatch and / or a head-mounted device (HMD).

[0025] According to one embodiment, a wearable device (101) can log in to an XR service based on the execution of an XR application by linking with the server while establishing a communication link with the server. The wearable device (101) can receive input information for using the XR service from an external electronic device and display a screen representing at least a portion of a virtual space corresponding to the input information.

[0026] For example, the wearable device (101) may include a head-mounted display (HMD) that is wearable on the user's head. According to one embodiment, the wearable device (101) may include a camera positioned toward the front of the user when the device is worn by the user. The front of the user may include a direction toward the user's head and / or the user's gaze. According to one embodiment, the wearable device (101) may include a sensor for identifying the motion of the user's head and / or the wearable device (101) when the device is worn by the user. The wearable device (101) may identify the angle (or posture) of the wearable device (113) based on data from the sensor. In order to provide a user interface (UI) based on virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) to a user wearing a wearable device (101), the wearable device (101) may control a camera and / or a sensor. The UI may be related to a metaverse service and / or a notification service provided by the wearable device (101) and / or a server connected to the wearable device (101).

[0027] According to one embodiment, a wearable device (101) may perform functions related to augmented reality (AR) and / or mixed reality (MR). When a user wears the wearable device (101), the wearable device (101) may include at least one lens positioned adjacent to the user's eyes. Ambient light passing through the lens may be combined (or mixed) with light emitted from a display (120) of the wearable device (101). A display area of ​​the display may be formed within the lens through which the ambient light passes. Since the wearable device (101) combines the ambient light and the light emitted from the display, the user may see an image that is a mixture of a real object recognized by the ambient light and a virtual object formed by the light emitted from the display.

[0028] According to one embodiment, a wearable device (101) may perform functions related to video see-through (VST) and / or virtual reality (VR). When a user wears the wearable device (101), the wearable device (101) may include a housing that covers the user's eyes. The wearable device (101), in the above state, may include a display disposed on a first side facing the eyes (e.g., the first side (310) of FIG. 3A). The wearable device (101) may include a camera (e.g., cameras 260-7, 260-8, 260-9, 260-10, 260-11, 260-12 of FIG. 3B) disposed on a second side opposite the first side (e.g., the second side (320) of FIG. 3A). Using the camera, the wearable device (101) can acquire frame images containing ambient light. The wearable device (101) can output the frame images to a display disposed on the first surface, thereby allowing a user to recognize the ambient light through the display. A display area of ​​the display disposed on the first surface can be formed by one or more pixels included in the display. The wearable device (101) can synthesize a virtual object within the frame images output through the display, so as to allow a user to recognize the virtual object together with a real object recognized by the ambient light.

[0029] According to one embodiment, a wearable device (101) can provide a user experience based on mixed reality (MR) by utilizing a virtual space. The wearable device (101) can recognize an external space (e.g., a real space) in which the wearable device (101) is included, and generate a virtual space mapped to the external space. The spatial recognition performed by the wearable device (101) can include SLAM and / or spatial mapping (e.g., scene understanding).

[0030] According to one embodiment, a wearable device (101) can identify a 6 degrees of freedom pose (6 dof pose) of the wearable device (101) using a sensor (130) of the wearable device (101). The wearable device (101) can identify input information based on the 6 degrees of freedom pose. The input information can include at least one of location information, viewpoint information, or pose information of the wearable device (101) identified by one or more programs included in the wearable device (101).

[0031] According to one embodiment, a wearable device (101) may receive location information from one or more external electronic devices (190) while connected to the one or more external electronic devices (190) through a communication circuit. For example, the wearable device (101) may identify an event that occurs within a virtual space and causes a vibration output. The wearable device (101) may obtain a distance between the location of the event and the location of one or more external electronic devices (190) identified by the location information. The wearable device (101) may transmit a signal for controlling an actuator included in one or more external electronic devices (190) based on the distance.

[0032] Referring to FIG. 1A, a wearable device (101) according to one embodiment may include at least one of a processor (110), a memory (115), a display (120), a camera (125), a sensor (130), or a communication circuit (135). The processor (110), the memory (115), the display (120), the camera (125), the sensor (130), and the communication circuit (135) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (102). The type and / or number of hardware components included in the wearable device (101) is not limited to those illustrated in FIG. 1A. For example, the wearable device (101) may include only some of the hardware components illustrated in FIG. 1A. The elements within the memory described below (e.g., layers and / or modules) may be logically distinct, but are not limited thereto.

[0033] A processor (110) of a wearable device (101) according to one embodiment may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors (110) may be one or more. For example, the processor (110) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.

[0034] The memory (115) of the wearable device (101) according to one embodiment may include a hardware component for storing data and / or instructions input and / or output to the processor (110). The memory (115) may include, for example, a volatile memory such as a random-access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multi media card (eMMC).

[0035] In one embodiment, the display (120) of the wearable device (101) can output visualized information to a user of the wearable device (101). For example, the display (120) can be controlled by a processor (110) including a circuit such as a graphic processing unit (GPU) to output visualized information to the user. The display (120) can include a flat panel display (FPD) and / or electronic paper. The FPD can include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs).

[0036] In one embodiment, the camera (125) of the wearable device (101) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera (125) may be arranged in the form of a two-dimensional array. The camera (125) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously, and generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the camera (125) may mean one (a) two-dimensional frame data acquired from the camera (125). For example, video data captured using the camera (125) may mean a sequence of a plurality of two-dimensional frame data acquired from the camera (125) according to a frame rate. The camera (125) may further include a flash light that is positioned toward the direction in which the camera (125) receives light and outputs light toward the direction.

[0037] According to one embodiment, the wearable device (101) may include a plurality of cameras, for example, cameras (125), arranged in different directions. A first camera among the plurality of cameras may be referred to as a motion recognition camera (e.g., motion recognition cameras 260-2 and 260-3 of FIG. 2B ), and a second camera may be referred to as a gaze tracking camera (e.g., gaze tracking camera 260-1 of FIG. 2B ). The wearable device (101) may identify a position, shape, and / or gesture of a hand using an image acquired using the first camera. The wearable device (101) may identify a direction of a gaze of a user wearing the wearable device (101) using an image acquired using the second camera. For example, the direction in which the first camera faces may be opposite to the direction in which the second camera faces.

[0038] According to one embodiment, a sensor (130) of a wearable device (101) may generate electrical information that may be processed by a processor (110) and / or a memory (115) of the wearable device (101) from non-electronic information related to the wearable device (101). The information may be referred to as sensor data. The sensor (130) may include a global positioning system (GPS) sensor, an image sensor, an ambient light sensor, and / or a time-of-flight (ToF) sensor for detecting a geographic location of the wearable device (101), and an inertial measurement unit (IMU) for detecting a physical motion of the wearable device (101).

[0039] In one embodiment, the communication circuit (135) of the wearable device (101) may include hardware components for supporting transmission and / or reception of electrical signals between the wearable device (101) and an external electronic device. The communication circuit (135) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (135) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G NR (new radio), and / or 6G.

[0040] According to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (110) of the wearable device (101) may be stored in the memory (115) of the wearable device (101). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable device (101) and / or the processor (110) may perform at least one of the operations of FIG. 10 when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. Hereinafter, the fact that an application is installed in a wearable device (101) may mean that one or more instructions provided in the form of an application are stored in a memory (115), and that the one or more applications are stored in a format executable by the processor (110) (e.g., a file having an extension specified by the operating system of the wearable device (101)). For example, an application may include a program and / or a library related to a service provided to a user.

[0041] Referring to FIG. 1A, programs installed in the wearable device (101) may be classified into any one of different layers, including an application layer (140), a framework layer (150), and / or a hardware abstraction layer (HAL) (180), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware of the wearable device (101) (e.g., the display (120), the camera (120), and / or the sensor (130)) may be classified within the hardware abstraction layer (180). The framework layer (150) may be referred to as an XR framework layer in that it includes one or more programs for providing an XR (extended reality) service. For example, FIG. 1A illustrates layers within the memory (115) as being divided, but the layers may be logically divided. However, the present invention is not limited thereto. Depending on the embodiment, the layers may be stored in a designated area within the memory (115).

[0042] For example, within the framework layer (150), programs designed to target at least one of the hardware abstraction layer (180) and / or the application layer (140) (e.g., a position tracker (171), a space recognizer (172), a gesture tracker (173), and / or an eye tracker (174)) may be classified. Programs classified within the framework layer (150) may provide an executable API (application programming interface) based on other programs.

[0043] For example, within the application layer (140), programs designed to target users controlling wearable devices (101) may be classified. As examples of programs classified into the application layer (140), an extended reality (XR) system user interface (UI) and / or an XR application (142) are exemplified, but the embodiment is not limited thereto. For example, programs (e.g., software applications) classified into the application layer (140) may call an application programming interface (API) to cause the execution of functions supported by programs classified into the framework layer (150).

[0044] For example, the wearable device (101) may display one or more visual objects on the display (120) for performing interaction with a user for using a virtual space based on the execution of the XR system UI (141) (user interface). A visual object may refer to an object that can be deployed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (101) may provide a service for controlling functions available within a virtual space to the user based on the execution of the XR system UI (141).

[0045] Referring to FIG. 1A, a lightweight renderer (143) and / or an XR plug-in (144) are illustrated as being included within the XR system UI (141), but are not limited thereto. For example, the XR system UI (141) may cause execution of functions supported by the lightweight renderer (143) and / or the XR plug-in (144) included within the framework layer (150).

[0046] For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (142). The lightweight renderer (143) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (143) may include a renderer built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (101) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (144). The XR plugin (144) may be referred to as an open XR native client from the perspective of defining (or setting up) the entire rendering pipeline.

[0047] For example, the wearable device (101) may display a screen representing at least a portion of a virtual space on the display (120) based on the execution of the XR application (142). The XR plug-in (144-1) included in the XR application (142) may be referenced by the XR plug-in (144) of the XR system UI (141). Descriptions of the XR plug-in (144-1) that overlap with the description of the XR plug-in (144) may be omitted. The wearable device (101) may cause the execution of the virtual space manager (151) based on the execution of the XR application (142).

[0048] According to one embodiment, the wearable device (101) may provide a virtual space service based on the execution of the virtual space manager (151). For example, the virtual space manager (151) may include a platform (e.g., an Android platform) for supporting the virtual space service. The wearable device (101) may display the posture of a virtual object representing the user's posture rendered using data acquired through the sensor (130) based on the execution of the virtual space manager (151) on the display. The virtual space manager (151) may be referred to as a composition presentation manager (CPM).

[0049] For example, the virtual space manager (151) may include a runtime service (152). As an example, the runtime service (152) may be referred to as an OpenXR runtime module. The wearable device (101) may be used to provide at least one of a pose prediction function, a frame timing function, and / or a spatial input function to a user through the wearable device (101) based on the execution of the runtime service (152). As an example, the wearable device (101) may be used to perform rendering for a virtual space service to a user based on the execution of the runtime service (152). For example, an application (e.g., unity or an OpenXR native application) may be implemented based on the execution of the runtime service (152).

[0050] For example, the virtual space manager (151) may include a pass-through manager (153). Based on the execution of the pass-through manager (153), the wearable device (101) may display a screen representing a virtual space on the display (120), while superimposing another screen representing an actual space acquired through the camera (125) on at least a portion of the screen.

[0051] For example, the virtual space manager (151) may include a vibration output control manager (154). The wearable device (101) may identify user perception information (181), actuator information (182), and / or haptic rendering information (183) based on the execution of the vibration output control manager (154). The wearable device (101) may use the identified information to obtain data for controlling the actuators of each of one or more external electronic devices (190) of the wearable device (101).

[0052] For example, the perception abstract layer (160) can be used for data exchange between the virtual space manager (151) and the perception service layer (170). From the perspective of being used for data exchange between the virtual space manager (151) and the perception service layer (170), the perception abstract layer (160) can be referred to as an interface. For example, the perception abstract layer (160) can be referenced as OpenPX. The perception abstract layer (160) can be used for a perception client and a perception service.

[0053] According to one embodiment, the recognition service layer (170) may include one or more programs for processing data acquired from a sensor (130) (or, camera (125)). The one or more programs may include at least one of a position tracker (171), a space recognizer (172), a gesture tracker (173), and / or an eye tracker (174). The type and / or number of the one or more programs included in the recognition service layer (170) are not limited to those illustrated in FIG. 1A.

[0054] For example, the wearable device (101) can identify the pose of the wearable device (101) using the sensor (130) based on the execution of the position tracker (171). The wearable device (101) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (101) using data acquired using the camera (125) and the IMU based on the execution of the position tracker (171). The position tracker (171) can be referred to as a head tracking (HeT) module.

[0055] For example, the wearable device (101) may be used to construct a three-dimensional virtual space around the wearable device (101) (or a user of the wearable device (101)) based on the execution of the space recognizer (172). The wearable device (101) may reconstruct the three-dimensional surroundings of the wearable device (101) using data acquired using the camera (125) based on the execution of the space recognizer (172). The wearable device (101) may identify at least one of a plane, a slope, and stairs based on the three-dimensionally reconstructed surroundings of the wearable device (101) based on the execution of the space recognizer (172). The space recognizer (172) may be referred to as a scene understanding (SU) module.

[0056] For example, the wearable device (101) may be used to identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (101) based on the execution of the gesture tracker (173). As an example, the wearable device (101) may identify a pose and / or gesture of a hand of a user using data acquired from a sensor (130) based on the execution of the gesture tracker (173). As an example, the wearable device (101) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using a camera based on the execution of the gesture tracker (173). The gesture tracker (173) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0057] For example, the wearable device (101) can identify (or track) eye movements of a user of the wearable device (101) based on the execution of the gaze tracker (174). As an example, the wearable device (101) can identify eye movements of the user using data acquired from at least one sensor based on the execution of the gaze tracker (174). As an example, the wearable device (101) can identify eye movements of the user based on data acquired using a camera (e.g., the gaze tracking camera (260-1) of FIGS. 2A and 2B) and / or an infrared light emitting diode (IR LED) based on the execution of the gaze tracker (174). The gaze tracker (174) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0058] For example, the recognition service layer (170) of the wearable device (101) may further include a face tracker for tracking the user's face. For example, the wearable device (103) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker. The wearable device (101) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker. As an example, the wearable device (101) may identify the movement of the user's face and / or the user's expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker.

[0059] For example, the user recognition information (181) may include information indicating the sensitivity of the touch sensation corresponding to each of the user's body parts. For example, the touch sensation may include a kinesthetic sensation that affects the sense of movement and / or a tactile sensation transmitted to the skin. The kinesthetic sensation may be identified based on the form of force-feedback by an external force on the user's body part. The tactile sensation may refer to the sensation of feeling a difference in physical pressure, vibration, or temperature. According to one embodiment, the wearable device (101) or one or more external electronic devices (190) may provide a tactile sensation to the user by using an actuator that outputs vibration. For example, the wearable device (101) or one or more external electronic devices (190) may further include an electronic component for providing a tactile sensation to the user by pressing a body part, such as a pin array, or by outputting an electrical stimulus to the body part.

[0060] For example, user recognition information (181) can be obtained based on the location of each of the body parts, the type of actuator included in the external electronic device worn on the body part, and the vibration intensity (or frequency intensity) output from the actuator. User recognition information (181) can be identified based on a psychophysical methodology. The wearable device (101) can obtain user recognition information (181) corresponding to the body parts by using the intensity of vibration output from each of one or more external electronic devices (190) worn on each of the body parts. The user recognition information (181) can be received from the server (109). For example, the user recognition information (181) can be obtained by using a user interface for the wearable device (101) to change the intensity of vibration output from each of one or more external electronic devices (190). An example of an operation in which a wearable device (101) acquires user recognition information (181) is described below with reference to FIG. 9.

[0061] For example, the actuator information (182) may include information indicating the type of vibration that can be output through the actuator. For example, it may include vibro-tactile type, force feedback type, electro-tactile type, skin deformation type, surface friction type, and / or thermal type. The actuator information (182) may include information indicating the number of actuators included in each of one or more external electronic devices (190). The actuator information (182) may include haptic pattern information that can be output using the actuator included in each of one or more external electronic devices (190), output strength of the actuator, vibration strength of the actuator, voltage strength of the actuator, friction strength of the actuator, and / or a frequency range of the actuator. Actuator information (182) may include time information required for each of one or more external electronic devices (190) to control the actuator. The wearable device (101) may, for example, use the time information to identify a synchronization between an event occurring in a virtual space and a vibration output from the actuator based on the time required for one or more external electronic devices (190) to control the actuator.

[0062] For example, actuator information (182) may include the type of data (e.g., input values) that cause the actuator to operate. For example, in the case of an actuator including an eccentric motor (e.g., a vibro-tactile type), actuator information (182) may include data indicating an output intensity and / or frequency mapped to a voltage intensity, but is not limited thereto.

[0063] For example, haptic rendering information (183) may include information about an event occurring within a virtual space. For example, haptic rendering information (183) may be provided for interaction between a user and a virtual space. The haptic rendering information may include haptic feedback. The haptic rendering information may include information for outputting vibrations in a specified pattern.

[0064] The above event can be identified by an interaction between a virtual object placed in a virtual space and a body part that comes into contact with the virtual object. The event can be provided from a server to a wearable device (101). The wearable device (101) can identify a first location where the event occurred. The first location can be referred to as a haptic interaction point (HIP) from the perspective that it is a location where an interaction that causes vibration is identified.

[0065] For example, when the wearable device (101) identifies an event that occurred at a first location, it can identify first haptic pattern information (183-1) representing vibration related to the event. For example, the first haptic pattern information (183-1) may be a vibration of a pattern corresponding to content when a user interacts with a virtual object on the screen of the wearable device (101). The first haptic pattern information (183-1) may represent information on the occurrence of a specific event, such as contact or collision with the virtual object. The first haptic pattern information (183-1) may identify the type of tactile sensation (e.g., kinesthetic sensation and / or tactile sensation) to be provided to the user based on information on the virtual object, the type of interaction (e.g., contact between the virtual object and the fingertip), and the type of one or more external electronic devices (190) connected to the wearable device (101).

[0066] For example, if the type of actuator included in one or more external electronic devices (190) is a first type (e.g., force feedback type), the first haptic pattern information (183-1) may include haptic output information corresponding to the first type. For example, if the type of information or interaction of the virtual object is related to the first type, the first haptic pattern information (183-1) may include haptic pattern information corresponding to the first type. However, the present invention is not limited thereto.

[0067] For example, if the type of actuator included in one or more external electronic devices (190) is a second type (e.g., tactile feedback-type), the wearable device (101) can obtain first haptic pattern information (183-1) indicating a vibration pattern.

[0068] For example, if the type of actuator included in one or more external electronic devices (190) is a second type (e.g., tactile feedback-type) and the type of information and interaction of the virtual object (e.g., contact between the virtual object and the fingertip) is related to a first type (e.g., force feedback-type), the wearable device (101) can obtain first haptic pattern information (183-1) based on changing haptic pattern information corresponding to the first type into haptic pattern information representing a vibration pattern.

[0069] Since an event according to one embodiment occurred within a virtual space, the wearable device (101) can infer first haptic pattern information (183-1) to be used to output vibration in a real space.

[0070] For example, the wearable device (101) can identify haptic pattern information corresponding to the first type. The wearable device (101) can calculate a penetration depth based on identifying an interaction indicating contact between a virtual object and a body part. The wearable device (101) can identify a specified stiffness coefficient for the virtual object using information about the virtual object. The wearable device (101) can obtain the magnitude of a force to be generated by the interaction based on specified information (e.g., information corresponding to a penalty-based method) using the specified stiffness coefficient and penetration depth. For example, the wearable device (101) can identify an interaction indicating a collision between a moving virtual object and a body part. The wearable device (101) can obtain the magnitude of a force to be generated by the interaction using the difference between the speed before identifying the collision and the speed after identifying the collision. For example, the wearable device (101) can obtain the intensity of the force based on a dynamics engine (or a program representing a physics engine). The wearable device (101) can obtain first haptic pattern information (183-1) indicating the intensity of the force.

[0071] For example, the wearable device (101) can obtain haptic pattern information corresponding to the second type. The wearable device (101) can identify a natural frequency (or frequency) of the virtual object by using information of the virtual object. The wearable device (101) can calculate (or obtain) the intensity of vibration to be generated in real space based on the interaction between the virtual object and the body part and the natural frequency. The wearable device (101) can obtain first haptic pattern information (183-1) representing the vibration based on the shape of the damping function.

[0072] According to one embodiment, a wearable device (101) can obtain second haptic pattern information (183-2) using first haptic pattern information (183-1). The second haptic pattern information (183-2) can be obtained based on the location of one or more external electronic devices (190). For example, the wearable device (101) can identify the second haptic pattern information (183-2) mapped to the first haptic pattern information (183-1). For example, the wearable device (101) can identify the second haptic pattern information (183-2) using a distance between a first location where an event occurred and a second location of one or more external electronic devices (190) and the first haptic pattern information (183-1). The second haptic pattern information (183-2) may be used by one or more external electronic devices (190) positioned at a second location to output vibration. For example, the wearable device (101) may use the first haptic pattern information (183-1) acquired based on one event to acquire second haptic pattern information (183-2) corresponding to each of one or more external electronic devices (190) positioned at a plurality of locations. For example, the wearable device (101) may acquire haptic rendering information (183) based on the execution of the vibration output control manager (154). However, the present invention is not limited thereto.

[0073] For example, the wearable device (101) can obtain second haptic pattern information (183-2) to be transmitted to an external electronic device including an actuator corresponding to the second type by using first haptic pattern information (183-1) based on the second type (e.g., Tactile feedback-type). For example, the wearable device (101) can obtain second haptic pattern information (183-2) by changing the time at which the vibration identified by using the first haptic pattern information (183-1) is output. The wearable device (101) can obtain second haptic pattern information (183-2) by using the first haptic pattern information (183-1) and a designated function (e.g., an amplitude modulation function). For example, the wearable device (101) may obtain second haptic pattern information (183-2) from first haptic pattern information (183-1) using information representing a sine wave having a periodicity or information representing a square wave, but is not limited thereto.

[0074] For example, when the wearable device (101) identifies an event related to the first type, it may transmit second haptic pattern information (183-2) obtained from the first haptic pattern information (183-1) to an external electronic device having an actuator corresponding to the first type. The wearable device (101) may transmit the second haptic pattern information (183-2) to an external electronic device located within a specified range from the first location where the event occurred. However, the present invention is not limited thereto.

[0075] For example, the wearable device (101) may output an audio signal representing a vibration notification. The wearable device (101) may output an audio signal using a speaker (not shown) and provide a vibration notification to the user using an actuator included in an external electronic device.

[0076] Referring to FIG. 1B, the wearable device (101) can establish a connection with one or more external electronic devices (190) and / or a server (109) based on a wired network and / or a wireless network. The wired network may include a network such as the Internet, a local area network (LAN), a wide area network (WAN), Ethernet, or a combination thereof. The wireless network may include a network such as long term evolution (LTE), 5g new radio (NR), wireless fidelity (WiFi), Zigbee, near field communication (NFC), Bluetooth, Bluetooth low-energy (BLE), or a combination thereof. Although the wearable device (101), one or more external electronic devices (190), or the server (109) are illustrated as being directly connected, the wearable device (101), one or more external electronic devices (190), or the server (109) may be indirectly connected via an intermediate node within the network.

[0077] According to one embodiment, one or more external electronic devices (190) may be wearable devices that can be worn by a user. The user may refer to a person wearing the electronic device (or wearable device). At least one of the one or more external electronic devices (190) may be formed in a ring shape. The at least one housing may be formed in a ring shape that can be worn on the user's finger. For example, at least one of the one or more external electronic devices (190) may be formed in a glove shape that can be worn on the user's hand. For example, at least one of the one or more external electronic devices (190) may include a user input device (e.g., a controller). The shape and / or hardware configuration of the one or more external electronic devices (190) are not limited to those illustrated in FIG. 1B.

[0078] Referring to FIG. 1B, among one or more external electronic devices (190) according to one embodiment, a first external electronic device (103) may include at least one of a communication circuit (135-1), a memory (115-1), or an actuator (137). Within the first external electronic device (103), the communication circuit (135-1), the memory (115-1), and the actuator (137) may be electrically and / or operatively coupled via a communication bus (102-1). The communication circuit (135-1), the memory (115-1), and the position tracker (171-1) may include hardware components and / or circuits corresponding to the position tracker (171), the memory (115), and the communication circuit (135) of the wearable device (101). The following description may be omitted.

[0079] In one embodiment, the first external electronic device (103) may include an actuator (137) for generating vibration. The actuator (137) may include an eccentric rotating mass (ERM) and / or a linear resonant actuator (LRA). In one embodiment where the actuator (137) includes the ERM, the actuator (137) may be configured to repeatedly move along a plurality of axes in a coordinate space (e.g., at least two of the x-axis, the y-axis, or the z-axis of FIG. 2A) to generate vibration. In one embodiment where the actuator (137) includes the LRA, the haptic actuator (137) may be configured to repeatedly move along a single axis in a coordinate space to generate vibration. When the haptic actuator (137) vibrates the housing of the electronic device (101), the vibration of the housing can generate a sound (hereinafter, friction sound) based on friction between the housing and an external object. When the actuator (137) vibrates the housing of the first external electronic device (103), a user wearing the first external electronic device (103) can recognize haptic feedback based on the vibration of the housing. The first external electronic device (103) can control the actuator (137) to output vibration using the second haptic pattern information (182-2) received from the wearable device (101). The number of actuators (137) included in the first external electronic device (103) can be one or more. At least one of the one or more external electronic devices (190) may include hardware and / or software included in the first external electronic device (103). Hereinafter, redundant descriptions may be omitted.

[0080] According to one embodiment, the server (109) may include at least one of a processor (110-2), a memory (115-2), and a communication circuit (135-2). Within the server (109), the processor (110-2), the memory (115-2), and the communication circuit (135-2) may be electrically and / or operatively coupled via a communication bus (102-2). The processor (110-2), the memory (115-2), and the communication circuit (135-2) included within the server (109) may include hardware components, circuits, and / or one or more programs corresponding to the processor (110), the memory (115), and the communication circuit (135) of the wearable device (101). Duplicate descriptions may be omitted below. The wearable device (101) may be referred to as a master device in the sense that it controls one or more external electronic devices (190) to output vibrations. One or more external electronic devices (190) may be referred to as slave devices in the sense that they receive commands indicating vibration output from the wearable device (101).

[0081] According to an embodiment as described above, the wearable device (101) may provide haptic feedback corresponding to an event occurring in a virtual space by using one or more external electronic devices (190) while connected to one or more external electronic devices (190). The wearable device (101) may identify a pattern of haptic feedback to be output from one or more external electronic devices (190) based on a location where the event occurred and a location of each of the one or more external electronic devices (190). The wearable device (101) may control the one or more external electronic devices (190) to provide the haptic feedback while displaying a visual object (or graphic effect) on the display. The wearable device (101) may provide spatial haptic feedback by using a distance between an event occurrence location and a location of the one or more external electronic devices (190).

[0082] FIG. 2A illustrates an example of a perspective view of a wearable device according to one embodiment. According to one embodiment, the wearable device (101) may have a form of glasses that can be worn on a body part (e.g., head) of a user. The wearable device (101) of FIGS. 2A and 2B may be an example of the wearable device (101) of FIG. 1A. The wearable device (101) may include a head-mounted display (HMD). For example, the housing of the wearable device (101) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (101) may include one or more straps capable of being twined around the user's head, and / or one or more temples attachable to the ears of the head.

[0083] Referring to FIG. 2A, according to one embodiment, a wearable device (101) may include at least one display (250) and a frame (200) supporting at least one display (250).

[0084] According to one embodiment, a wearable device (101) can be worn on a part of a user's body. The wearable device (101) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (101). For example, the wearable device (101) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.

[0085] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0086] Referring to FIG. 2B, at least one display (250) can provide visual information transmitted from external light to the user through a lens included in the at least one display (250), and other visual information distinct from the visual information. The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of the at least one display (250). When a user wears the wearable device (101), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0087] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (101) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0088] The wearable device (101) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (101) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (101) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (101) can view an image displayed on at least one display (250).

[0089] According to one embodiment, the frame (200) may be formed as a physical structure that allows the wearable device (101) to be worn on the user's body. According to one embodiment, the frame (200) may be configured so that, when the user wears the wearable device (101), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0090] Referring to FIG. 2A, the frame (200) may include a region (220) that is at least partially in contact with a part of the user's body when the user wears the wearable device (101). For example, the region (220) of the frame (200) that is in contact with a part of the user's body may include a region that is in contact with a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (101) makes contact with. According to one embodiment, the frame (200) may include a nose pad (210) that is in contact with a part of the user's body. When the wearable device (101) is worn by the user, the nose pad (210) may be in contact with a part of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that are in contact with another part of the user's body that is distinct from the part of the user's body.

[0091] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (101) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (200).

[0092] According to one embodiment, the wearable device (101) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 4). For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).

[0093] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (101) may be disposed on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (101) is two or more, the wearable device (101) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (200).

[0094] According to one embodiment, at least one optical device (282, 284) may project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) may be a projector. At least one optical device (282, 284) may be disposed adjacent to at least one display (250) or may be included within at least one display (250) as a part of at least one display (250). According to one embodiment, the wearable device (101) may include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0095] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (101). For example, the wearable device (101) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (101) can identify an object (e.g., a real object and / or a virtual object) focused on by the user using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (101) that has identified the focused object can execute a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (101) can express a part corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (101) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality of a first region related to the gaze within the image and the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a second region distinguished from the first region may be different from each other. The wearable device (101) can obtain an image having visual quality of a first area matching the user's gaze and visual quality of a second area using foveated rendering.For example, if the wearable device (101) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1). An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or toward both eyes.

[0096] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera (260-4) can be used to acquire a high-resolution image based on HR (high resolution) or PV (photo video). The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including the image of the specific object acquired using the capturing camera (260-4) and a virtual image provided through at least one optical device (282, 284) are superimposed. The wearable device (101) can compensate for depth information (e.g., the distance between the wearable device (101) and an external object acquired through a depth sensor) using the image acquired through the capturing camera (260-4). The wearable device (101) can perform object recognition through an image acquired using the capturing camera (260-4). The wearable device (101) can perform a function of focusing on an object (or subject) in an image (e.g., auto focus) and / or an optical image stabilization (OIS) function (e.g., anti-shake function) using the capturing camera (260-4). The wearable device (101) can perform a pass-through function to display an image acquired through the capturing camera (260-4) by overlapping at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the capturing camera (260-4) can be disposed on a bridge (203) disposed between the first rim (201) and the second rim (202).

[0097] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing a wearable device (101), thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (101) looks straight ahead, the wearable device (101) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (101) is positioned.

[0098] The motion recognition camera (260-2, 260-3) can recognize the movement of the user's entire body, such as the user's torso, hand, or face, or a part of the body, and thereby provide a specific event on a screen provided on at least one display (250). The motion recognition camera (260-2, 260-3) can recognize the user's motion (gesture recognition), obtain a signal corresponding to the motion, and provide a display corresponding to the signal on at least one display (250). The processor can identify the signal corresponding to the motion, and perform a designated function based on the identification. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 dof pose). The processor can perform a gesture recognition function and / or an object tracking function using the motion recognition camera (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) may be positioned on the first rim (201) and / or the second rim (202).

[0099] The camera (260) included in the wearable device (101) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (101) can identify an external object included in the FoV using a camera positioned toward the user's FoV. The wearable device (101) can identify an external object based on a sensor for identifying the distance between the wearable device (101) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (101) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (101).

[0100] Although not shown, in one embodiment, the wearable device (101) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an LED having an infrared wavelength. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).

[0101] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (101). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0102] The antenna module (275) can transmit signals or power to the outside of the wearable device (101), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0103] The speaker (255) can output an acoustic signal to the outside of the wearable device (101). The acoustic output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (101). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus positioned adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus positioned adjacent to the user's right ear.

[0104] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (101) to the user. For example, when the wearable device (101) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0105] Referring to FIG. 2B, according to one embodiment, a wearable device (101) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (101) (e.g., hardwares illustrated by different blocks in FIG. 1A) may be disposed on the PCB (290). The wearable device (101) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0106] According to one embodiment, a wearable device (101) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (101) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (101). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (101) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (101) based on the IMU.

[0107] Figures 3a and 3b illustrate an example of an exterior appearance of a wearable device according to one embodiment. The wearable device (101) of Figures 3a and 3b may be an example of the wearable device (101) of Figure 1a. An example of an exterior appearance of a first side (310) of a housing of the wearable device (101) according to one embodiment may be illustrated in Figure 3a, and an example of an exterior appearance of a second side (320) opposite to the first side (310) may be illustrated in Figure 3b.

[0108] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (101) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (101) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (101) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0109] According to one embodiment, the wearable device (101) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (101) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (101) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6). For example, the wearable device (101) may change the texture and / or shape of a portion of an avatar (e.g., a portion of an avatar representing a human face) using information obtained by cameras (260-5, 260-6) (e.g., FT cameras) and representing the facial expression of a user wearing the wearable device (101).

[0110] Referring to FIG. 3B, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor (330)) for obtaining information related to the external environment of the wearable device (101) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3A. For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0111] For example, using cameras (260-11, 260-12), the wearable device (101) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (101) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (101) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0112] According to one embodiment, the wearable device (101) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (101) and an external object. Using the depth sensor (330), the wearable device (101) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (101). Although not illustrated, a microphone may be disposed on the second face (320) of the wearable device (101) to obtain a sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0113] FIG. 4 illustrates an example of an operation of a wearable device according to one embodiment of the present invention to control an external electronic device based on the distance between the location of an event and the location of the external electronic device. The wearable device (101) of FIG. 4 may include the wearable device (101) of FIGS. 1A to 3B .

[0114] Referring to FIG. 4, an exemplary state in which a wearable device (101) according to one embodiment establishes a connection with one or more external electronic devices (190) is illustrated. The wearable device (101) can be worn by a user (405). Among the one or more external electronic devices (190), a first external electronic device (103) and a second external electronic device (105) can be worn on at least one of the body parts (e.g., a finger or a wrist) of the user (405). The wearable device (101) can transmit a signal for establishing a communication link to the one or more external electronic devices (190) through a communication circuit. The wearable device (101) can identify the first external electronic device (103) and the second external electronic device (105) that transmitted a response signal to the wearable device (101) in response to the signal. The wearable device (101) can receive location information of a first external electronic device (103) and a second external electronic device (105). The wearable device (101) can identify a first body part (e.g., a finger) of a user (405) wearing the first external electronic device (103) and a second body part (e.g., a wrist) of a user (405) wearing the second external electronic device (105). The wearable device (101) can receive actuator information of the first external electronic device (103) and actuator information of the second external electronic device (105). For example, the wearable device (101) can identify the initial locations of the first external electronic device (103) and the second external electronic device (105) based on the posture of the user (405) identified from the sensor. The wearable device (101) can track the locations of the first external electronic device (103) and the second external electronic device (105) while establishing the communication link. The locations of the first external electronic device (103) and the second external electronic device (105) can be obtained from the first external electronic device (103) and the second external electronic device (105) through a communication circuit.The locations of the first external electronic device (103) and the second external electronic device (105) can be identified through images acquired through the camera of the wearable device (101), but are not limited thereto.

[0115] Referring to FIG. 4, an example of a wearable device (101) identifying an event occurring at a first location (420) according to one embodiment is illustrated. According to one embodiment, the electronic device (101) can identify an event related to the output of vibration using an actuator (137). The event can be generated by a request from an application installed in the wearable device (101) (e.g., an XR application (442) of FIG. 1A). The event can be generated by receiving a notification message transmitted from a server connected to the wearable device (101) (e.g., a server providing XR). The event can be generated based on the identification of an interaction between a body part of a user identified through a camera and a virtual object displayed on a display. The event can be generated by a call to a designated API (application programming interface) provided by an operating system (e.g., Android) of the wearable device (101). For example, the above-mentioned API may be called by at least one of an application request, identification of an interaction, and / or receipt of a notification message.

[0116] For example, the wearable device (101) may display a graphic effect (410) that spreads within a virtual space based on the first location on the display in response to an event that occurs at the first location (420) and causes an output of a vibration. The graphic effect (410) may be identified based on the first haptic pattern information. The graphic effect (410) displayed by the wearable device (101) may increase in size based on the first location (420). The wearable device (101) may display a graphic effect (410) that visually represents a vibration on the display. The graphic effect (410) may be referred to as a visual guidance, a visual object, a visual effect, a UI element, and / or a view element in terms of representing the vibration. The wearable device (101) can display a graphic effect (410) that moves from a first location (420) to the entire virtual space on the display for a specified period of time. The wearable device (101) can adjust the synchronization between the time point at which the graphic effect (410) intersects the location of the external electronic device and the time point at which the external electronic device outputs a vibration. The location of the external electronic device can be referred to as a haptic rendering point (HRP) or a haptic output point (HOP) from the perspective of the location at which the vibration is output.

[0117] For example, the wearable device (101) can obtain first haptic pattern information (e.g., first haptic pattern information (183-1) of FIG. 1A) based on identifying an event that occurred at a first location (420). The event can be generated by an interaction between a body part of a user (405) and a virtual object (415). The wearable device (101) can obtain the first haptic pattern information based on information of the virtual object (415) and the type of the interaction (e.g., contact). The wearable device (101) may obtain second haptic pattern information (e.g., second haptic pattern information (183-2) of FIG. 1A) corresponding to each of the one or more external electronic devices (190) from the first haptic pattern information based on actuator information of one or more external electronic devices (190), user recognition information (e.g., user recognition information (181) of FIG. 1A), and / or the location of each of the one or more external electronic devices (190).

[0118] For example, the wearable device (101) can identify a distance (420-1) between a first location (410) and a second location (421) (e.g., a haptic rendering point, HRP or a haptic output point, HOP)) of the first external electronic device (103). The wearable device (101) can obtain second haptic pattern information (431) corresponding to the first external electronic device (103) from the first haptic pattern information based on the distance (420-1). A graph representing a haptic pattern included in the second haptic pattern information (431) is not limited to that illustrated in FIG. 4. The graph may, for example, have a sinusoidal wave shape and / or a square wave shape. The wearable device (101) can identify a point in time (421-1) at which at least a portion of the graphic effect (410) intersects the second location (421) based on the distance (420-1). The wearable device (101) can identify a pattern, intensity, and / or vibration time of vibration output by the first external electronic device (103) using second haptic pattern information corresponding to the first external electronic device (103). The wearable device (101) can identify the vibration intensity based on the distance (420-1). For example, the wearable device (101) can obtain second haptic pattern information including a lower vibration intensity as the second location (421) becomes farther from the first location (420). For example, the vibration intensity corresponding to the second haptic pattern information may be lower than the vibration intensity corresponding to the first haptic pattern information. However, the present invention is not limited thereto.

[0119] For example, the wearable device (101) may transmit a signal to the first external electronic device (103) through a communication circuit to control an actuator of the first external electronic device (103) to output vibration at a time point (421-1). The signal may include second haptic pattern information (431) corresponding to the first external electronic device (103). The first external electronic device (103) may control the actuator to output vibration corresponding to the second haptic pattern information (431) at the time point (421-1). The first external electronic device (103) may control the actuator to output vibration having a first vibration intensity (431-1) using the second haptic pattern information (431). The first external electronic device (103) may output vibration for a specified period of time. The designated time may include the time at which the moving graphic effect (410) is included in the area corresponding to the location (421). The designated time may be identified by the wearable device (101), but is not limited thereto.

[0120] For example, the wearable device (101) can identify a distance (420-12) between a first location (410) and a third location (422) (e.g., a haptic rendering point, HRP or a haptic output point, HOP) of a second external electronic device (105). The wearable device (101) can obtain second haptic pattern information (432) corresponding to the second external electronic device (105) from the first haptic pattern information based on the distance (420-2). The wearable device (101) can identify a point in time (422-1) at which at least a portion of the graphic effect (410) matches the third location (421) based on the distance (420-2). The wearable device (101) can use the second haptic pattern information corresponding to the second external electronic device (105) to output from the second external electronic device (105). The wearable device (101) can identify the vibration pattern, intensity, and / or vibration time. The wearable device (101) can identify the vibration intensity based on the distance (420-2). For example, the wearable device (101) can obtain second haptic pattern information including a lower vibration intensity as the third location (422) is farther from the first location (420). For example, when the distance (420-2) is longer than the distance (420-1), the vibration intensity corresponding to the second haptic pattern information corresponding to the second external electronic device (105) may be lower than the vibration intensity corresponding to the second haptic pattern information corresponding to the first external electronic device (103). However, the present invention is not limited thereto. For example, the vibration intensity may be changed depending on the type of event.

[0121] For example, the wearable device (101) may transmit a signal to the second external electronic device (105) through a communication circuit to control the actuator of the second external electronic device (105) so that vibration is output at a time point (422-1). The signal may include second haptic pattern information (432) corresponding to the second external electronic device (105). The twelfth external electronic device (105) may control the actuator to output vibration corresponding to the second haptic pattern information (432) at the time point (422-1). The second external electronic device (105) may control the actuator to output vibration having a second vibration intensity (432-1) using the second haptic pattern information (432). The first external electronic device (103) may output vibration for a specified period of time. The wearable device (101) can identify a designated time at which a graphic effect (410) is included in an area corresponding to a location (422), but is not limited thereto.

[0122] According to one embodiment, a wearable device (101) can transmit a signal for controlling an actuator to a first external electronic device (103) and a second external electronic device (105) at substantially similar transmission times. The transmission times may be before times (421-1, 422-1). The first external electronic device (103) receiving the signal can output a vibration at time (421-1), and the second external electronic device (105) can output a vibration at time (422-1). The times (421-1, 422-1) can be identified in proportion to a distance. For example, the wearable device (101) can identify the times (421-1, 422-1) by using a delay time taken by one or more wearable devices (190) to control the actuator.

[0123] According to one embodiment, a wearable device (101) may transmit a signal that causes a vibration output to at least one wearable device among one or more wearable devices (190) based on first haptic pattern information. When the wearable device (101) identifies first haptic pattern information corresponding to a first type (e.g., Force feedback type), the wearable device may transmit the signal to an external electronic device having a type corresponding to the first type. When the wearable device identifies first haptic pattern information corresponding to the first type, the wearable device may temporarily refrain from transmitting the signal to an external electronic device having a type corresponding to a second type (e.g., Tactile feedback type). However, the present invention is not limited thereto. For example, the wearable device (101) may change the first haptic pattern information corresponding to the first type into second haptic pattern information corresponding to the second type, and transmit the signal to an external electronic device having a type corresponding to the second type.

[0124] For example, the wearable device (101) may acoustically indicate vibrations through a speaker (not shown). For example, the wearable device (101) may output an audio signal indicating vibrations corresponding to the second haptic pattern information (431, 432) through the speaker. However, the present invention is not limited thereto.

[0125] As described above, the wearable device (101) according to one embodiment can provide haptic feedback by linking with one or more external electronic devices (190) based on identifying an event. The wearable device (101) can provide spatial haptic feedback to the user by using haptic feedback output from one or more external electronic devices (190) together with a graphic effect (410) displayed on the display. Hereinafter, with reference to FIG. 5, an example of an operation in which the wearable device (101) obtains second haptic pattern information from first haptic pattern information will be described below.

[0126] FIG. 5 illustrates an example of an operation of a wearable device, according to one embodiment, controlling an external electronic device based on an event. The electronic device (101) of FIG. 5 may include the electronic devices (101) of FIGS. 1A to 4 . One or more external electronic devices (190) of FIG. 5 may be referenced to one or more external electronic devices (190) of FIG. 1B .

[0127] According to one embodiment, a wearable device (101) can identify an event that occurs at a first location (501) that is spaced apart from the wearable device (101). The wearable device (101) can identify an event that causes a vibration output. The wearable device (101) can identify an event that indicates an interaction (e.g., contact) between a body part of a user and a virtual object. Based on the identification of the event, the wearable device (101) can obtain first haptic pattern information (515). The first haptic pattern information (515) can be referenced to the first haptic pattern information (183-1) of FIG. 1A. The wearable device (101) can identify virtual object information (510) that indicates characteristics of a virtual object related to the event. The wearable device (101) can identify first haptic pattern information (515) based on virtual object information (510) and the type of interaction (e.g., contact or collision).

[0128] For example, the wearable device (101) may identify one or more external electronic devices (190) within a specified range from the first location (501) based on identifying an event that occurred at the first location (501). The specified range may include a range set to output vibrations due to the event that occurred at the first location (501). A second location (502) and a third location (503) may be included within the specified range.

[0129] For example, the wearable device (101) can identify a first external electronic device (103) and a second external electronic device (105) worn by a user. The wearable device (101) can identify a second location (502) of the first external electronic device (103) and a third location (503) of the second external electronic device (105). Based on the identification of the second location (502) and the third location (503), the wearable device (101) can obtain second haptic pattern information (520) of the first external electronic device (103) and second haptic pattern information (521) of the second external electronic device (105) using the first haptic pattern information (515).

[0130] For example, the wearable device (101) can obtain second haptic pattern information (520) of the first external electronic device (103) using the first haptic pattern information (515) based on the distance between the first location (501) and the second location (502). The wearable device (101) can identify actuator information (182-1) and user recognition information (181-1) corresponding to the first external electronic device (103). The user recognition information (181-1) corresponding to the first external electronic device (103) can include recognition information about a body part of the user on which the first external electronic device (103) is worn. The user recognition information (181-1) corresponding to the first external electronic device (103) can include information indicating the sensitivity of tactile sensation that the user can detect according to a frequency. The user recognition information (181-1) corresponding to the first external electronic device (103) may include information indicating the sensitivity of tactile sensation that the user can perceive according to amplitude. For example, the actuator information (182-1) corresponding to the first external electronic device (103) may include information indicating the type of actuator included in the first external electronic device (103). The actuator information (182-1) corresponding to the first external electronic device (103) may include information indicating the frequency of vibration outputtable by the actuator and / or the intensity of force. However, the present invention is not limited thereto.

[0131] For example, the wearable device (101) may obtain second haptic pattern information (520) using the first haptic pattern information (515), the distance between the first location (501) and the second location (502), the actuator information (182-1) corresponding to the first external electronic device (103), and the user recognition information (181-1). The wearable device (101) may transmit a signal requesting the first external electronic device (103) to output vibration at a specified time using the second haptic pattern information (520). The specified time may include a time when a visual object (e.g., a graphic effect) generated from the first location (501) reaches the second location (502).

[0132] For example, the wearable device (101) can obtain second haptic pattern information (521) of the second external electronic device (105) using the first haptic pattern information (515) based on the distance between the first location (501) and the third location (503). The wearable device (101) can identify actuator information (182-2) and user recognition information (181-2) corresponding to the second external electronic device (105). The user recognition information (181-2) corresponding to the second external electronic device (105) can include recognition information about a body part (e.g., a wrist) of the user on which the second external electronic device (105) is worn. The user recognition information (181-2) corresponding to the second external electronic device (105) can include information indicating a tactile sensitivity that the user can detect according to a frequency and / or amplitude. For example, actuator information (182-2) corresponding to the second external electronic device (105) may include information indicating the type of actuator included in the second external electronic device (105). Actuator information (182-2) corresponding to the second external electronic device (105) may include a range of frequencies of vibrations that the second external electronic device (105) can control and output by the actuator, and / or a range of force intensity, but is not limited thereto.

[0133] For example, the wearable device (101) may obtain second haptic pattern information (521) using the first haptic pattern information (515), the distance between the first location (501) and the third location (503), the actuator information (182-2) corresponding to the second external electronic device (105), and / or the user recognition information (181-2). The wearable device (101) may transmit a signal requesting the second external electronic device (105) to output vibration at a specified time using the second haptic pattern information (521). The specified time may include a time when a visual object (e.g., a graphic effect) generated from the first location (501) is included in an area corresponding to the third location (503).

[0134] For example, the wearable device (101) can identify a first distance between a first location (501) and a second location (502) and a second distance between the first location (501) and a third location (503). The first distance may be shorter than the second distance. The wearable device (101) can identify a first time point for a first external electronic device (103) disposed at a second location (502) to output vibration, and a second time point for a second external electronic device (105) disposed at a third location (503) to output vibration. The first time point may be earlier than the second time point. The wearable device (101)

[0135] According to one embodiment, a wearable device (101) may identify at least one external electronic device (190) for causing a vibration output based on identifying an event that occurred at a first location (501). For example, the wearable device (101) may transmit a signal requesting the output of a vibration to the first external electronic device (103) among the first external electronic device (103) and the second external electronic device (105). The wearable device (101) may at least temporarily refrain from transmitting the signal requesting the output of a vibration to the second external electronic device (105) among the first external electronic device (103) and the second external electronic device (105). The wearable device (101) may select at least one external electronic device (190) for causing a vibration output based on identifying the event. For example, if an event causes an output of a vibration corresponding to a first type (e.g., force feedback type), the wearable device (101) may select at least one external electronic device including an actuator having the first type. For example, if the type of information or interaction of the virtual object is related to the first type, the first haptic pattern information (183-1) may include haptic pattern information corresponding to the first type.

[0136] According to one embodiment, the wearable device (101) can change the second haptic pattern information (520) based on the actuator information (182-1) based on identifying the first haptic pattern information (515). For example, the wearable device (101) can identify a perception intensity state corresponding to the second location (502). The perception intensity state can be set by the user. The perception intensity state can include a first state for outputting vibration based on a certain constant, a second state for sequentially reducing the vibration intensity, and / or a third state for outputting vibration based on a maximum output. The wearable device (101) can obtain the second haptic pattern information (520) corresponding to the second location (502) based on the perception intensity state. However, the present invention is not limited thereto.

[0137] Below, with reference to FIG. 6, an example of an operation in which a wearable device (101) displays a graphic effect related to an event is described.

[0138] FIG. 6 illustrates an example of a graphic effect displayed by a wearable device based on an event according to one embodiment. The wearable device (101) of FIG. 6 may be included in the wearable devices (101) of FIGS. 1A to 5 . The first external electronic device (103) and / or the second external electronic device (105) of FIG. 6 may be included in one or more external electronic devices (190) of FIG. 1B .

[0139] Referring to FIG. 6, in state (600), a wearable device (101) according to one embodiment can identify an interaction between a virtual object (601) and a body part. The wearable device (101) can identify a first location (602) where an interaction with the virtual object (601) was performed. Based on the identification of the first location (602), the wearable device (101) can display a graphic effect (606) generated from the first location (602) on the display. The graphic effect (606) can be an example of a visual object expressing vibration. The graphic effect (606) can represent a visual effect set to spread throughout the virtual space. The wearable device (101) can display a graphic effect (606) that increases in size on the display. Since the wearable device (101) displays a graphic effect (606) that visually represents vibration, the graphic effect (606) can be changed based on haptic rendering information (e.g., haptic rendering information (183) of FIG. 1A). For example, the wearable device (101) can identify second haptic pattern information (e.g., second haptic pattern information (183-2) of FIG. 1A) having a second vibration intensity that is smaller than a first vibration intensity corresponding to first haptic pattern information (e.g., first haptic pattern information (183-1) of FIG. 1A). The wearable device (101) can display a graphic effect of decreasing size on the display based on the second haptic pattern information. However, the present invention is not limited thereto. The wearable device (101) can display a graphic effect (606) having the form of at least one of the visual objects (650) based on an event.

[0140] According to one embodiment, the wearable device (101) can identify a second location (603) of a first external electronic device (103) and a third location (604) of a second external electronic device (105) based on the interaction identified. The wearable device (101) can identify a first distance between the first location (602) and the second location (603). The wearable device (101) can identify a second distance between the first location (602) and the third location (604). Based on the identification of the first distance and / or the second distance, the wearable device (101) can identify a first time point and / or a second time point at which to output vibration by controlling the first external electronic device (103) and / or the second external electronic device (105).

[0141] For example, after identifying an interaction, the wearable device (101) may display a graphic effect (606) moving within a virtual space on the display based on a specified cycle. The wearable device (101) may identify a third point in time and / or a fourth point in time at which the moving graphic effect (606) reaches a second location (603) and / or a third location (604) based on identifying the first distance and / or the second distance.

[0142] For example, the wearable device (101) can map a first time point at which vibration is to be output by controlling the first external electronic device (103) and a third time point at which the graphic effect (606) reaches the second location (603). In a state (610), the wearable device (101) can transmit a signal to the first external electronic device (103) to control an actuator of the first external electronic device (103) so that vibration is output at the first time point (or the mapped third time point). The wearable device (101) can transmit the signal to the first external electronic device (103) to cause the vibration to be output while at least a portion of the graphic effect (606) is included in an area corresponding to the second location (603). The wearable device (101) can, based on transmitting the signal, interact with the first external electronic device (103) and tactually control the first external electronic device (103) to output vibration while visually expressing vibration using a graphic effect (606).

[0143] For example, the wearable device (101) can control the second external electronic device (105) to map a second time point at which vibration is to be output and a fourth time point at which the graphic effect (606) reaches the third location (604). For example, the wearable device (101) can, at state (620), transmit a signal to the second external electronic device (105) to control an actuator of the second external electronic device (105) to output vibration at the second time point (or the mapped fourth time point). The wearable device (101) can transmit the signal to the second external electronic device (105) to cause it to output vibration while at least a portion of the graphic effect (606) is included in an area corresponding to the third location (604).

[0144] As described above, the wearable device (101) according to one embodiment can display a graphic effect (606) representing the vibration and spreading throughout the virtual space on the display based on identifying an event causing a vibration generated from the first location (601). The wearable device (101) can identify the vibration output timing of one or more external electronic devices in conjunction with the graphic effect (606). The wearable device (101) can provide a synesthetic haptic feedback to the user based on visually representing the vibration using the graphic effect (606) and tactilely controlling one or more external electronic devices.

[0145] Fig. 7 illustrates an example of a change in vibration intensity output from an external electronic device according to one embodiment. The wearable device (101) of Fig. 7 may include the wearable device (101) of Figs. 1A to 6 . The second external electronic device (105) of Fig. 7 may be included in one or more external electronic devices (190) of Fig. 1B .

[0146] Referring to FIG. 7, in a state (700), a wearable device (101) according to one embodiment can identify an interaction between a virtual object (701) displayed through a display and a body part of a user (405). The wearable device (101) can identify a first location (702) where the interaction is identified. Based on the identified interaction, the wearable device (101) can generate first haptic pattern information (e.g., first haptic pattern information (183-1) of FIG. 1A) based on information of the virtual object (701). Information of the virtual object (701) can be acquired based on a dynamics engine. For example, the wearable device (101) can acquire the first haptic pattern information while the interaction is maintained between the virtual object (701) and the body part.

[0147] For example, the wearable device (101) can identify a first distance between a first location (702) and a second location (703) of a second external electronic device (105). Based on the identification of the first distance, the wearable device (101) can obtain second haptic pattern information (710). The wearable device (101) can obtain second haptic pattern information (710) using the first haptic pattern information obtained while the interaction is maintained.

[0148] For example, the wearable device (101) may transmit a signal to the second external electronic device (105) requesting that the second external electronic device control the actuator to output vibration based on the second haptic pattern information (710). The second external electronic device (105) may output vibration based on the second haptic pattern information (710).

[0149] According to one embodiment, a wearable device (101) can track the location of a second external electronic device (105). The wearable device (101) can track the second external electronic device (105) moving from a second location (703) to a third location (704). For example, the wearable device (101) can identify the second external electronic device (105) approaching a virtual object (701). The wearable device (101) can identify a second distance between the third location (704) and the first location (702). The wearable device (101) can obtain second haptic pattern information (720) using the first haptic pattern information obtained while the interaction is maintained based on the second distance. The second distance between the first location (702) and the third location (704) may be shorter than the first distance between the first location (702) and the second location (703). The wearable device (101) may obtain second haptic pattern information (720) having a higher frequency than the frequency of the second haptic pattern information (710). The wearable device (101) may transmit a signal representing the second haptic pattern information (720) to the second external electronic device (105). The second external electronic device (105) may control an actuator based on the second haptic pattern information (720) to output vibration.

[0150] For example, when the wearable device (101) identifies the location of a second external electronic device (105) moving away from a virtual object (701), it may obtain second haptic pattern information having a lower frequency than the frequency of the second haptic pattern information (710), but is not limited thereto.

[0151] As described above, the wearable device (101) according to one embodiment can adjust the intensity of vibration to be output from an external electronic device based on the movement of the external electronic device while the interaction between the virtual object and the user is maintained.

[0152] FIG. 8 illustrates an example of a virtual space including a plurality of wearable devices according to one embodiment. The wearable device (101) of FIG. 8 may include the wearable devices (101) of FIGS. 1A to 7 . Referring to FIG. 8 , an example of a server (109) providing a virtual space (800) in which a plurality of users (405, 805) are logged in is illustrated. The server (109) may correspond to the server (109) of FIG. 1B .

[0153] Referring to FIG. 8, a server (109) according to one embodiment can identify the occurrence of an event (810) at a first location (811) within a virtual space (800). Based on the identification of the event (810), the server (109) can identify a plurality of users (405, 805) located within the virtual space (800). The server (109) can transmit a signal to the first wearable device (101) and the second wearable device (801) to display a graphic effect representing the event (810) on the displays of the first wearable device (101) and the second wearable device (801).

[0154] For example, the server (109) can identify electronic devices corresponding to each of the plurality of users (405, 805). The server (109) can identify a first wearable device (101) and a first external electronic device (105) corresponding to a first user (405) among the plurality of users (405, 805). The server (109) can identify a second wearable device (801) and a second external electronic device (803) corresponding to a second user (805) among the plurality of users (405, 805). The second wearable device (801) can be a terminal owned by the second user (805).

[0155] For example, the server (109) can identify first haptic pattern information (e.g., first haptic pattern information (184-1) of FIG. 1B) that causes a vibration output based on an event (810) generated from a first location (811). The server (109) can identify a first distance between the first location (811) and the location of the first wearable device (101). The server (109) can identify a second distance between the first location (811) and the location of the first external electronic device (105). For example, the server (109) can identify a third distance between the first location (811) and the location of the second wearable device (801). The server (109) can identify a fourth distance between the first location (811) and the location of the second external electronic device (803).

[0156] For example, the server (109) may obtain second haptic pattern information (e.g., second haptic pattern information (184-2) of FIG. 1B) from the first haptic pattern information based on the first distance and / or the second distance. The server (109) may obtain the second haptic pattern information using user recognition information (e.g., user recognition information (181-2) of FIG. 1B) corresponding to the first user (405). The server (109) may obtain the second haptic pattern information based on actuator information (e.g., actuator information (182-2) of FIG. 1B) of the first external electronic device (105). However, the present invention is not limited thereto.

[0157] For example, the server (109) can identify a time point for outputting a vibration based on the first distance and / or the second distance. The server (109) can transmit a signal including the time point and second haptic pattern information to the first wearable device (101). Based on receiving the signal, the first wearable device (101) can transmit another signal to the first external electronic device (105) requesting output of a vibration corresponding to the second haptic pattern information at the time point.

[0158] For example, the server (109) may obtain second haptic pattern information (e.g., second haptic pattern information (184-2) of FIG. 1B) from the first haptic pattern information based on the third distance and / or the fourth distance. The server (109) may obtain second haptic pattern information using user recognition information corresponding to the second user (805) (e.g., user recognition information (181-2) of FIG. 1B). The server (109) may obtain second haptic pattern information based on actuator information of the second external electronic device (803) (e.g., actuator information (182-2) of FIG. 1B). However, the present invention is not limited thereto.

[0159] For example, the server (109) can identify a time point for outputting a vibration based on the third distance and / or the fourth distance. The server (109) can transmit a signal including the time point and second haptic pattern information to the second wearable device (801). Based on receiving the signal, the second wearable device (801) can transmit another signal to the second external electronic device (803) requesting output of a vibration corresponding to the second haptic pattern information at the time point. For example, the server (109) can provide haptic rendering information corresponding to an event (810) identified in a virtual space to each of a plurality of electronic devices owned by a plurality of users (405, 805).

[0160] For example, if the first distance between the first location (811) and the first wearable device (101) and the second distance between the first location (811) and the second wearable device (801) are substantially the same, the server (109) may transmit a signal requesting the first wearable device (101) and the second wearable device (801) to output vibration at the same time.

[0161] According to one embodiment, a server (109) may transmit data (e.g., information about a virtual object) related to an event (810) to a first wearable device (101) and / or a second wearable device (801).

[0162] For example, the first wearable device (101) can obtain first haptic pattern information related to the event (810) based on receiving the data. The wearable device (101) can identify the location of the first external electronic device (105) based on receiving the data. The first wearable device (101) can identify the distance between the location of the first external electronic device (101) and the first location (811) where the event (810) occurred. The first wearable device (101) can obtain second haptic pattern information from the first haptic pattern information based on the distance. The first wearable device (101) can control at least a part of the first external electronic device (105) using the second haptic pattern information to output vibration.

[0163] Fig. 9 illustrates an example of a screen for controlling the vibration intensity of an external electronic device according to one embodiment. The wearable device (101) of Fig. 9 may include the wearable devices (101) of Figs. 1A to 8. The first external electronic device (103), the second external electronic device (105), and / or the third external electronic device (107) may be included in one or more external electronic devices (190) of Fig. 1B.

[0164] Referring to FIG. 9, a wearable device (101) according to one embodiment can identify one or more external electronic devices (190) worn by a user (405) in a state (900). The wearable device (101) can identify actuator information (e.g., actuator information (182) of FIG. 1A) of the one or more external electronic devices (190). The wearable device (101) can identify the location of each of the one or more external electronic devices (190) worn by the user (405).

[0165] According to one embodiment, a wearable device (101) may receive a user input for adjusting the intensity of vibration output from one or more external electronic devices (190). In response to the user input, the wearable device (101) may display a user interface (910) on a display. The user interface (910) may be referred to as a calibration screen in that it is used to adjust the intensity of vibration. For example, while displaying the calibration screen, the wearable device (101) may enter a calibration mode (or state).

[0166] For example, the wearable device (101) may transmit a signal requesting vibration output to each of one or more wearable devices (190). For example, the first external electronic device (103) may control an actuator to output vibration in response to the signal. The first external electronic device (103) may sequentially output vibration corresponding to the intensity based on increasing the intensity of the vibration.

[0167] For example, the wearable device (101) may receive a user input for adjusting the intensity of the vibration while the first external electronic device (103) outputs the vibration. The wearable device (101) may display a visual object (915) indicating the intensity of the vibration on the display. The wearable device (101) may adjust the intensity of the vibration output from the first external electronic device (103) based on the user input. However, the present invention is not limited thereto. As an example, the wearable device (101) may obtain user perception information indicating a tactile sensation sensed by the user (405) while the first external electronic device (103) outputs the vibration. The wearable device (101) may obtain the user perception information based on specified information (e.g., a method of adjustment among psychophysical methodologies). A wearable device (101) according to one embodiment can obtain user recognition information corresponding to each of one or more wearable devices (190). Hereinafter, descriptions of the second external electronic device (105) and the third external electronic device (107) that overlap with the operation of obtaining user recognition information corresponding to the first external electronic device (103) may be omitted.

[0168] As described above, the wearable device (101) according to one embodiment can obtain different user recognition information (181) depending on the user. The wearable device (101) can obtain different haptic pattern information based on the identification of the same event by using the different user recognition information (181) depending on the user.

[0169] FIG. 10 illustrates an example of a flowchart illustrating the operation of a wearable device according to one embodiment. The wearable device of FIG. 10 may include the wearable device (101) of FIGS. 1A to 9. At least one of the operations of FIG. 10 may be performed by the wearable device (101) of FIG. 1A. At least one of the operations of FIG. 10 may be controlled by the processor (110) of FIG. 1A. Each of the operations of FIG. 10 may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each of the operations may be changed, and at least two operations may be performed in parallel.

[0170] In operation 1010, a processor according to one embodiment may display a graphic effect on a display in response to an event occurring at a first location spaced apart from the wearable device and causing a vibration output. The graphic effect may expand in size toward the entire virtual space in which the wearable device is logged in. The graphic effect may be displayed to move from the first location to the location of the wearable device, but is not limited thereto. For example, the processor may display a graphic effect that spreads from the first location to an external space. The external space may include a virtual space in which the wearable device is logged in. The external space in which the wearable device is located and the virtual space in which the wearable device is logged in may be mapped. For example, the wearable device may map location information within the external space and location information within the virtual space.

[0171] The processor can obtain first haptic pattern information representing a vibration associated with an event.

[0172] In operation 1020, a processor according to one embodiment may identify a distance between a first location and a second location of an external electronic device spaced apart from the wearable device. The processor may change the first haptic pattern information into second haptic pattern information based on the distance. The processor may obtain the second haptic pattern information based on the distance, the type of the external electronic device, and / or user perception information. The second haptic pattern information may be used by the external electronic device to control an actuator to output a vibration.

[0173] In operation 1030, according to one embodiment, the processor may identify a point in time at which at least a portion of the graphic effect intersects a second location based on the identified distance. The processor may identify the point in time at which at least a portion of the graphic effect intersects the second location while expanding the size of the graphic effect toward the entire virtual space based on the event.

[0174] In operation 1040, a processor according to an embodiment may transmit a signal to the external electronic device via a communication circuit to control an actuator of the external electronic device to output vibration at a point in time. The processor may transmit the signal before the point of intersection. After receiving the signal, the external electronic device may control the actuator at the point of intersection to output vibration (or haptic feedback).

[0175] The Metaverse is a portmanteau of the English word "meta," meaning "virtual" or "transcendence," and "universe," and refers to a three-dimensional virtual world where social, economic, and cultural activities, like those in the real world, take place. The Metaverse is a concept that is one step more advanced than virtual reality (VR, a cutting-edge technology that allows people to have realistic experiences in a computer-generated virtual world), and has the characteristic of allowing people to engage in social and cultural activities similar to those in the real world, rather than simply enjoying games or virtual reality using avatars. The Metaverse service can provide media content to enhance immersion in the virtual world based on augmented reality (AR), virtual reality environment (VR), mixed reality (MR), and / or extended reality (XR).

[0176] For example, media content provided by a metaverse service may include social interaction content, including avatar-based games, concerts, parties, and / or conferences. For example, the media content may include advertisements, user-created content, and / or information for economic activities such as the sale and / or shopping of products. Ownership of the user-created content may be proven by a blockchain-based non-fungible token (NFT). The metaverse service may support economic activities based on real-world currency and / or cryptocurrency. The metaverse service may provide virtual content linked to the real world, such as digital twins or life logging.

[0177] Figure 11 is an example diagram of a network environment (1101) that provides metaverse services through a server (1110).

[0178] Referring to FIG. 11, a network environment (1101) may include a server (1110), a user terminal (1120) (e.g., a first terminal (1120-1) and a second terminal (1120-2)), and a network connecting the server (1110) and the user terminal (1120). Within the network environment (1101), the server (1110) may provide a metaverse service to the user terminal (1120). The network may be formed by at least one intermediate node (1130) including an access point (AP) and / or a base station. The user terminal (1120) may connect to the server (1110) through the network and output a user interface (UI) related to the metaverse service to the user of the user terminal (1120). Based on the above UI, the user terminal (1120) can obtain information to be input into the metaverse service from the user, or output information related to the metaverse service (e.g., multimedia content) to the user.

[0179] At this time, the server (1110) provides a virtual space so that the user terminal (1120) can be active in the virtual space. In addition, the user terminal (1120) installs a software agent for accessing the virtual space provided by the server (1110) to express information provided by the server (1110) to the user or transmit information that the user wishes to express in the virtual space to the server. The software agent may be provided directly through the server (1110), downloaded from a public server, or embedded and provided when the terminal is purchased.

[0180] In one embodiment, the metaverse service may be provided to a user terminal (1120) and / or a user using a server (1110). The embodiment is not limited thereto, and the metaverse service may be provided through individual contact between users. For example, within a network environment (1101), the metaverse service may be provided independently from the server (1110) through a direct connection between a first terminal (1120-1) and a second terminal (1120-2). Referring to FIG. 11, within the network environment (1101), the first terminal (1120-1) and the second terminal (1120-2) may be connected to each other through a network formed by at least one intermediate node (1130). In one embodiment where the first terminal (1120-1) and the second terminal (1120-2) are directly connected, either of the user terminals of the first terminal (1120-1) and the second terminal (1120-2) may perform the role of the server (1110). For example, a metaverse environment may be configured solely through device-to-device connections (e.g., peer-to-peer (P2P) connections).

[0181] In one embodiment, the user terminal (1120) (or the user terminal (1120) including the first terminal (1120-1) and the second terminal (1120-2)) may be made in various form factors and is characterized by including an output device for providing images and / or sounds to the user and an input device for inputting information to the metaverse service. Examples of various form factors of the user terminal (1120) include a smartphone (e.g., the second terminal (1120-2)), an AR device (e.g., the first terminal (1120-1)), a VR device, an MR device, a VST (Video See Through) device, an OST (Optical See Through) device, a smart lens, a smart mirror, an input / output capable TV or projector.

[0182] The network of the present invention (e.g., a network formed by at least one intermediate node (1130)) includes various broadband networks including 3G, 4G, and 5G, and short-range networks including Wifi, BT, etc. (e.g., a wired network or wireless network directly connecting the first terminal (1120-1) and the second terminal (1120-2).

[0183] A wearable device according to one embodiment may be capable of tactilely outputting vibrations by interfacing with multiple external electronic devices while expressing vibrations using visual objects displayed on a display. A method for a wearable device to express vibrations using visual objects and multiple external electronic devices may be required.

[0184] In a wearable device (101) according to one embodiment as described above, the wearable device may include a communication circuit (135), a display (120), at least one processor (110) including a processing circuit, and a memory (115) storing instructions and including one or more storage media. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display, on the display, a graphic effect (410, 606) that spreads from the first location into external space in response to an event that occurs at a first location (420, 501, 602, 702) spaced apart from the wearable device and causes an output of a vibration. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a distance (420-1) between the first location and a second location (420, 502, 603) of an external electronic device spaced from the wearable device. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify, based on the identified distance, a point in time (421-1) at which at least a portion of the graphic effect intersects the second location. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to transmit, through the communication circuit, a signal to the external electronic device to control an actuator of the external electronic device to output a vibration having a magnitude determined based on the distance at the point in time.

[0185] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain first haptic pattern information (183-1) representing the vibration associated with the event based on identifying the event. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the graphic effect on the display based on the first haptic pattern information.

[0186] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to change the first haptic pattern information into second haptic pattern information (183-2) for controlling the actuator of the external electronic device based on at least one of a type of the external electronic device and the distance. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to transmit, to the external electronic device before the time point, the signal requesting the electronic device to output the vibration based on the magnitude corresponding to the second haptic pattern information at the time point.

[0187] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to change the first haptic pattern information to second haptic pattern information having a second vibration intensity that is less than a first vibration intensity corresponding to the first haptic pattern information, based on the distance.

[0188] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify the event indicating contact between at least one body part of a user of the wearable device and a virtual object (415, 701) displayed on the display. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain the first haptic pattern information based on a type of the virtual object.

[0189] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to change from the first haptic pattern information to the second haptic pattern information based on user perception information (181) related to vibration.

[0190] For example, the user recognition information may be obtained based on a user interface (910) for identifying an intensity of vibration generated by the external electronic device on a body part of a user wearing the external electronic device, depending on the type of the external electronic device.

[0191] For example, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to transmit the signal to the external electronic device, causing the electronic device to output a vibration corresponding to the second haptic pattern information while at least a portion of the graphic effect is included in an area corresponding to the second location.

[0192] In a method of a wearable device (101) according to one embodiment as described above, the method may include an operation of displaying, on a display (120), a graphic effect (410; 606) that spreads from the first location to an external space in response to an event that occurs at a first location (420, 501, 602, 702) spaced apart from the wearable device and causes an output of a vibration. The method may include an operation of identifying a distance (420-1) between the first location and a second location (420, 502, 603) of an external electronic device spaced apart from the wearable device. The method may include an operation of identifying a point in time (421-1) at which at least a portion of the graphic effect intersects the second location based on the identified distance. The method may include an operation of transmitting a signal to the external electronic device through a communication circuit (135) to control an actuator of the external electronic device so that vibration having a size determined based on the distance at the point in time is output.

[0193] For example, the action of displaying the graphic effect may include an action of obtaining first haptic pattern information (183-1) representing the vibration associated with the event based on identifying the event. The action of displaying the graphic effect may include an action of displaying the graphic effect on the display based on the first haptic pattern information.

[0194] For example, the operation of transmitting the signal may include an operation of changing the first haptic pattern information into second haptic pattern information (183-2) for controlling the actuator of the external electronic device based on at least one of the type and the distance of the external electronic device. The operation of transmitting the signal may include an operation of transmitting, to the external electronic device before the time point, the signal requesting the external electronic device to output the vibration based on the magnitude corresponding to the second haptic pattern information at the time point.

[0195] For example, the operation of changing to the second haptic pattern information may include an operation of changing the first haptic pattern information to the second haptic pattern information having a second vibration intensity that is smaller than a first vibration intensity corresponding to the first haptic pattern information, based on the distance.

[0196] For example, the operation of obtaining the first haptic pattern information may include an operation of identifying the event indicating contact between at least one body part of the user of the wearable device and a virtual object (415, 701) displayed on the display. The operation of obtaining the first haptic pattern information may include an operation of obtaining the first haptic pattern information based on the type of the virtual object.

[0197] For example, the action of changing to the second haptic pattern information may include an action of changing from the first haptic pattern information to the second haptic pattern information based on user perception information (181) related to vibration.

[0198] For example, the user recognition information may be obtained based on a user interface (910) for identifying an intensity of vibration generated by the external electronic device on a body part of a user wearing the external electronic device, depending on the type of the external electronic device.

[0199] In a computer-readable storage medium storing instructions according to one embodiment as described above, the instructions, when executed by a processor (110) of a wearable device (101), may cause the wearable device to display, on a display (120), a graphic effect (410, 606) that spreads from the first location into external space in response to an event that occurs at a first location (420, 501, 602, 702) spaced apart from the wearable device and causes an output of a vibration. The instructions, when executed by the processor, may cause the wearable device to identify a distance (420-1) between the first location and a second location (420, 502, 603) of an external electronic device spaced apart from the wearable device. The instructions executed by the processor may cause the wearable device to identify a point in time (421-1) at which at least a portion of the graphic effect intersects the second location, based on the identified distance. The instructions executed by the processor may cause the wearable device to transmit, via the communication circuit (135), a signal to the external electronic device to control an actuator of the external electronic device so as to output a vibration having a magnitude determined based on the distance at the point in time.

[0200] For example, the instructions executed by the processor may cause the wearable device to obtain first haptic pattern information (183-1) representing the vibration associated with the event based on identifying the event. The instructions executed by the processor may cause the wearable device to display the graphic effect on the display based on the first haptic pattern information.

[0201] For example, the instructions executed by the processor may cause the wearable device to change the first haptic pattern information to second haptic pattern information (183-2) for controlling the actuator of the external electronic device based on at least one of the type and the distance of the external electronic device. The instructions executed by the processor may cause the wearable device to transmit, prior to the time point, to the external electronic device the signal requesting the device to output the vibration based on the magnitude corresponding to the second haptic pattern information at the time point.

[0202] For example, the instructions executed by the processor may cause the wearable device to change the first haptic pattern information to second haptic pattern information having a second vibration intensity that is less than a first vibration intensity corresponding to the first haptic pattern information, based on the distance.

[0203] For example, the instructions executed by the processor may cause the wearable device to identify the event indicating contact between at least one body part of a user of the wearable device and a virtual object (415, 701) displayed on the display. The instructions executed by the processor may cause the wearable device to obtain the first haptic pattern information based on the type of the virtual object.

[0204] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0205] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0206] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0207] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0208] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In a wearable device (101), Communication circuit (135); display (120); At least one processor (110) comprising a processing circuit; A memory (115) storing instructions and including one or more storage media; The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: In response to an event occurring at a first location (420; 501; 602; 702) spaced from the wearable device and causing an output of vibration, a graphic effect (410; 606) is displayed on the display that spreads from the first location into external space; Identifying a distance (420-1) between the first location and a second location (420; 502; 603) of an external electronic device spaced from the wearable device; Based on identifying the above distance, identifying a point in time (421-1) where at least a portion of the graphic effect intersects the second location; and Causing, through the above communication circuit, to transmit a signal to the external electronic device to control an actuator of the external electronic device so that vibration having a size determined based on the distance at the above point in time is output. Wearable devices.

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on identifying the above event, first haptic pattern information (183-1) representing the vibration related to the event is obtained, and Based on the first haptic pattern information, causing the graphic effect to be displayed on the display. Wearable devices.

3. In either of paragraphs 1 and 2, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on at least one of the type of the external electronic device and the distance, the first haptic pattern information is changed into second haptic pattern information (183-2) for controlling the actuator of the external electronic device, and Causing the external electronic device to transmit, prior to the time, the signal requesting the external electronic device to output the vibration based on the size corresponding to the second haptic pattern information at the time. Wearable devices.

4. In any one of paragraphs 1 to 3, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on the above distance, causing the first haptic pattern information to be changed to the second haptic pattern information having a second vibration intensity smaller than the first vibration intensity corresponding to the first haptic pattern information. Wearable devices.

5. In any one of paragraphs 1 to 4, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: identifying the event indicating contact between at least one body part of the user of the wearable device and a virtual object (415; 701) displayed on the display, and causing the first haptic pattern information to be acquired based on the type of the virtual object; Wearable devices.

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on the user perception information (181) related to vibration, causing a change from the first haptic pattern information to the second haptic pattern information, Wearable devices.

7. In any one of paragraphs 1 to 6, The above user recognition information is: Based on the type of the external electronic device, an intensity of vibration generated by the external electronic device on a body part of a user wearing the external electronic device is obtained based on a user interface (910). Wearable devices.

8. In any one of paragraphs 1 to 7, The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: causing the external electronic device to transmit a signal, causing the external electronic device to output a vibration corresponding to the second haptic pattern information, while at least a portion of the graphic effect is included in an area corresponding to the second location; Wearable devices.

9. In the method of a wearable device (101), the method comprises: An action of displaying, on a display (120), a graphic effect (410; 606) that spreads from the first location to the external space in response to an event that occurs at a first location (420; 501; 602; 702) that is spaced apart from the wearable device and causes an output of vibration, An operation of identifying a distance (420-1) between the first location and a second location (420; 502; 603) of an external electronic device spaced from the wearable device; Based on the identification of the above distance, an operation of identifying a point in time (421-1) at which at least a portion of the above graphic effect intersects the above second location, and An operation of transmitting a signal to the external electronic device through a communication circuit (135) to control an actuator of the external electronic device so that vibration having a size determined based on the distance at the time is output. method.

10. In paragraph 9, the action of displaying the graphic effect is: An operation of obtaining first haptic pattern information (183-1) representing the vibration related to the event based on identifying the event, and An action including displaying the graphic effect on the display based on the first haptic pattern information. method.

11. In any one of paragraphs 9 and 10, the operation of transmitting the signal comprises: An operation of changing the first haptic pattern information into second haptic pattern information (183-2) for controlling the actuator of the external electronic device based on at least one of the type of the external electronic device and the distance, and An operation of transmitting, to the external electronic device, prior to the time point, the signal requesting the external electronic device to output the vibration based on the size corresponding to the second haptic pattern information at the time point. method.

12. In any one of clauses 9 to 11, the operation of changing to the second haptic pattern information is: Based on the distance, the first haptic pattern information includes an operation of changing the first haptic pattern information into the second haptic pattern information having a second vibration intensity smaller than the first vibration intensity corresponding to the first haptic pattern information. method.

13. In any one of clauses 9 to 12, the operation of obtaining the first haptic pattern information comprises: An action for identifying an event indicating contact between at least one body part of a user of said wearable device and a virtual object (415; 701) displayed on said display, and An operation for obtaining the first haptic pattern information based on the type of the virtual object, method.

14. In any one of clauses 9 to 13, the operation of changing to the second haptic pattern information is: An operation including changing from the first haptic pattern information to the second haptic pattern information based on user perception information (181) related to vibration. method.

15. In a computer-readable storage medium storing instructions, when the instructions are executed by a processor (110) of a wearable device (101), In response to an event occurring at a first location (420; 501; 602; 702) spaced from the wearable device and causing an output of vibration, a graphic effect (410; 606) is displayed on the display (120) that spreads from the first location to the external space, Identifying a distance (420-1) between the first location and a second location (420; 502; 603) of an external electronic device spaced from the wearable device, Based on identifying the above distance, identifying a point in time (421-1) where at least a portion of the graphic effect intersects the second location, and Causing the wearable device to transmit a signal to the external electronic device through the communication circuit (135) to control the actuator of the external electronic device so that vibration having a size determined based on the distance at the time is output. Computer readable storage medium.

Images