Wearable device, method, and non-transitory computer-readable storage medium for updating avatar

Abstract

Disclosed is a wearable device. The wearable device may comprise a memory storing instructions. On the basis of comparing first data related to the appearance of a user identified in an image, captured via at least one camera, including a part of the body of the user wearing the wearable device, and second data related to the appearance of an avatar of the user, the wearable device may identify whether an update of at least one of the appearance of the avatar or an avatar animation of the avatar is required. On the basis of identifying that an update of the appearance of the avatar is required, the wearable device may update the appearance attribute of the avatar so that the appearance of the avatar corresponds to the appearance of the user. On the basis of identifying that an update of the animation is required, the wearable device may update a movement attribute of the animation that is related to a joint of the avatar limited due to the attribute of the appearance of the user.

Description

Wearable device, method, and non-transitory computer-readable storage medium for updating an avatar

[0001] The following descriptions relate to a wearable device, a method, and a non-transitory computer readable storage medium for updating an avatar.

[0002] Electronic devices can provide avatars that reflect the user's appearance or behavior in virtual or augmented reality. Electronic devices can convey the user's facial expressions and / or emotional state through the avatars. Facial expressions and emotional expressions can play a crucial role in everyday communication as a medium for nonverbal communication. Furthermore, electronic devices can visually present personalized characters through avatars.

[0003] The electronic device can convey the user's facial expressions and / or emotional state to other users through the facial expressions of an avatar generated based on the user's video and voice input through a camera and microphone.

[0004] A wearable device is disclosed. The wearable device may include at least one camera. The wearable device may include a processor. The wearable device may include a memory storing instructions. The instructions, when executed by the processor, may cause the wearable device to capture an image including a portion of a body of a user wearing the wearable device through the at least one camera. The instructions, when executed by the processor, may cause the wearable device to identify whether an update is required of at least one of the avatar appearance or an avatar animation of the avatar based on comparing first data related to a user appearance of the user identified in the image with second data related to an avatar appearance of an avatar of the user. The instructions, when executed by the processor, may cause the wearable device to update an appearance attribute of the avatar so that the avatar appearance corresponds to the user appearance based on the identification that an update of the avatar appearance is required. The instructions, when executed by the processor, may cause the wearable device to update a movement property of the animation associated with a joint of the avatar whose movement is restricted by a property of the user's appearance, based on the wearable device identifying that an update of the animation is required.

[0005] A method is disclosed. The method may be performed in a wearable device including at least one camera. The method may include an action of capturing an image including a portion of a body of a user wearing the wearable device through the at least one camera. The method may include an action of identifying whether an update is required of at least one of the avatar appearance or an avatar animation of the avatar based on comparing first data related to a user appearance of the user identified in the image with second data related to an avatar appearance of an avatar of the user. The method may include an action of updating an appearance attribute of the avatar so that the avatar appearance corresponds to the user appearance, based on an identification that an update of the avatar appearance is required. The method may include an action of updating a movement attribute of an animation associated with a joint of the avatar whose movement is restricted by an attribute of the user appearance, based on an identification that an update of the animation is required.

[0006] A non-transitory computer-readable storage medium is disclosed. The non-transitory computer-readable storage medium can store a program including instructions. In a wearable device, the instructions, when executed by a processor of the wearable device including at least one camera, can cause the wearable device to capture an image including a portion of a body of a user wearing the wearable device through the at least one camera. The instructions, when executed by the processor, can cause the wearable device to determine whether an update of at least one of the avatar appearance or the avatar animation of the avatar is required based on comparing first data related to a user appearance of the user identified in the image with second data related to an avatar appearance of an avatar of the user. The instructions, when executed by the processor, may cause the wearable device to update an appearance attribute of the avatar so that the avatar appearance corresponds to the user appearance based on the identification that an update of the avatar appearance is required. The instructions, when executed by the processor, may cause the wearable device to update a movement attribute of the animation associated with a joint of the avatar whose movement is restricted by an attribute of the user appearance based on the identification that an update of the animation is required.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0008] FIG. 2A illustrates an example of a perspective view of a wearable device, according to one embodiment.

[0009] FIG. 2b illustrates an example of one or more hardware elements disposed within a wearable device, according to one embodiment.

[0010] FIG. 3A illustrates an example of an appearance of a wearable device according to one embodiment.

[0011] FIG. 3b illustrates an example of an appearance of a wearable device according to one embodiment.

[0012] FIG. 4 illustrates an example block diagram of a wearable device according to one embodiment.

[0013] Figure 5 illustrates an example of a user's appearance and the appearance of the user's avatar.

[0014] Figure 6 illustrates an example of landmarks of a user's user face and an avatar's avatar face.

[0015] Figure 7 illustrates an example of a change in a user's texture.

[0016] Figure 8 illustrates an example of an avatar asset corresponding to a user's belongings.

[0017] Figure 9 shows an example of an animation of an avatar corresponding to a user's belongings.

[0018] Figure 10 is a flowchart illustrating the operation of a wearable device according to one embodiment.

[0019] Figure 11 is a flowchart illustrating the operation of a wearable device according to one embodiment.

[0020] Figure 12 is a flowchart illustrating the operation of a wearable device according to one embodiment.

[0021] Figure 13 is a flowchart illustrating the operation of a wearable device according to one embodiment.

[0022] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0023] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0024] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0025] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0026] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0027] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0030] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0031] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0032] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0036] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0038] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 664 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 6 ms or less for round trip) for URLLC realization.

[0041] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0042] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045] FIG. 2A illustrates an example of a perspective view of a wearable device (200), according to one embodiment. FIG. 2B illustrates an example of one or more hardware elements disposed within a wearable device (200), according to one embodiment. The wearable device (200) of FIGS. 2A and 2B may correspond to the electronic device (101) of FIG. 1. As shown in FIG. 2A, the wearable device (200), according to one embodiment, may include at least one display (250) and a frame supporting the at least one display (250).

[0046] According to one embodiment, the wearable device (200) can be worn on a part of a user's body. The wearable device (200) can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to the user wearing the wearable device (200). For example, the wearable device (200) can output a virtual reality image to the user through at least one display (250) in response to a designated gesture of the user acquired through the motion recognition camera (240-2) of FIG. 2B.

[0047] According to one embodiment, at least one display (250) within a wearable device (200) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0048] Referring to FIG. 2B, at least one display (250) may form a display area on a lens to provide a user wearing the wearable device (200) with visual information contained in external light passing through the lens, along with other visual information distinct from the visual information. The lens may be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. The display area formed by at least one display (250) may be formed on the second surface (232) among the first surface (231) and the second surface (232) of the lens. When the user wears the wearable device (200), external light may be incident on the first surface (231) and transmitted through the second surface (232) to be transmitted to the user. As another example, at least one display (250) may display a virtual reality image to be combined with a real screen transmitted through the external light. The virtual reality image output from at least one display (250) can be transmitted to the user's eyes through one or more hardware (e.g., optical devices (282, 284), and / or at least one waveguide (233, 234)) included in the wearable device (200).

[0049] According to one embodiment, a wearable device (200) may include waveguides (233, 234) that diffract light transmitted from at least one display (250) and relayed by optical devices (282, 284) and transmit the diffracted light to a user. The waveguides (233, 234) may be formed based on at least one of glass, plastic, or polymer. Nanopatterns may be formed on at least a portion of the exterior or interior of the waveguides (233, 234). The nanopatterns may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the waveguides (233, 234) may be propagated to the other end of the waveguides (233, 234) by the nanopatterns. The waveguides (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, the waveguides (233, 234) may be arranged within the wearable device (200) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the waveguides (233, 234).

[0050] According to one embodiment, the wearable device (200) may analyze an object included in a real image collected through a capturing camera (240-1), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object may include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (200) may analyze the object based on a multi-camera, such as a stereo camera. For the object analysis, the wearable device (200) may execute time-of-flight (ToF) and / or simultaneous localization and mapping (SLAM) supported by the multi-camera. A user wearing the wearable device (200) may view an image displayed on at least one display (250).

[0051] According to one embodiment, the frame may be formed as a physical structure that allows the wearable device (200) to be worn on the user's body. According to one embodiment, the frame may be configured so that, when the user wears the wearable device (200), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame may support at least one display (250). For example, the frame may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0052] Referring to FIG. 2A, the frame may include a region (220) that at least partially contacts a part of the user's body when the user wears the wearable device (200). For example, the region (220) of the frame that contacts a part of the user's body may include a region that contacts a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (200) makes contact with. According to one embodiment, the frame may include a nose pad (210) that contacts a part of the user's body. When the wearable device (200) is worn by the user, the nose pad (210) may contact a part of the user's nose. The frame may include a first temple (204) and a second temple (205) that contact a part of the user's body that is distinct from the part of the user's body.

[0053] For example, the frame may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of an ear of the wearer, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of an ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (200) can identify an external object (e.g., a user's fingertip) touching the frame and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame.

[0054] According to one embodiment, the wearable device (200) may include hardwares that perform various functions (e.g., hardwares described above based on the block diagram of FIG. 1). For example, the hardwares may include a battery module (270), an antenna module (275), optical devices (282, 284), speakers (292-1, 292-2), microphones (294-1, 294-2, 294-3), a light-emitting module (not shown), and / or a printed circuit board (290). The various hardwares may be arranged within a frame.

[0055] According to one embodiment, the microphones (294-1, 294-2, 294-3) of the wearable device (200) may be arranged on at least a portion of the frame to acquire a sound signal. A first microphone (294-1) arranged on the nose pad (210), a second microphone (294-2) arranged on the second rim (202), and a third microphone (294-3) arranged on the first rim (201) are illustrated in FIG. 2B, but the number and arrangement of the microphones (294) are not limited to the embodiment of FIG. 2B. When the number of microphones (294) included in the wearable device (200) is two or more, the wearable device (200) may identify the direction of the sound signal by using a plurality of microphones arranged on different portions of the frame.

[0056] According to one embodiment, the optical devices (282, 284) can transmit a virtual object transmitted from at least one display (250) to the waveguides (233, 234). For example, the optical devices (282, 284) can be projectors. The optical devices (282, 284) can be positioned adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). The first optical device (282) can correspond to the first display (250-1), and the second optical device (284) can correspond to the second display (250-2). The first optical device (282) can transmit light output from the first display (250-1) to the first waveguide (233), and the second optical device (284) can transmit light output from the second display (250-2) to the second waveguide (234).

[0057] In one embodiment, the camera (240) may include an eye tracking camera (ET CAM) (240-1), a motion recognition camera (240-2), and / or a recording camera (240-3). The recording camera, the eye tracking camera (240-1), and the motion recognition camera (240-2) may be positioned at different locations on the frame and may perform different functions. The eye tracking camera (240-1) may output data representing the gaze of a user wearing the wearable device (200). For example, the wearable device (200) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (240-1). An example in which the gaze tracking camera (240-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (240-1) may be positioned solely toward the user's left eye, or may be positioned toward both eyes.

[0058] In one embodiment, the capturing camera (240-3) can capture an actual image or background to be aligned with a virtual image to implement augmented reality or mixed reality content. The capturing camera can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about the actual image or background including the image of the specific object acquired using the capturing camera is superimposed on a virtual image provided through optical devices (282, 284). In one embodiment, the capturing camera can be placed on a bridge (203) disposed between the first rim (201) and the second rim (202).

[0059] In one embodiment, the gaze tracking camera (240-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (200) and thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (200) looks straight ahead, the wearable device (200) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (240-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (240-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (240-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (240-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (200) is positioned.

[0060] In one embodiment, the gesture recognition camera (240-2) can provide a specific event on a screen provided on at least one display (250) by recognizing the movement of the user's entire body or a part of the user's body, such as the user's torso, hands, or face. The gesture recognition camera (240-2) can recognize the user's gesture, obtain a signal corresponding to the gesture, and provide a display corresponding to the signal on at least one display (250). The processor can identify the signal corresponding to the gesture and perform a designated function based on the identification. In one embodiment, the gesture recognition camera (240-2) can be disposed on the first rim (201) and / or the second rim (202).

[0061] In one embodiment, the camera (240) included in the wearable device (200) is not limited to the gaze tracking camera (240-1) and the motion recognition camera (240-2) described above. For example, the wearable device (200) can identify an external object included in the FoV using the photographing camera (240-3) positioned toward the user's FoV. The wearable device (200) can identify an external object based on a sensor for identifying the distance between the wearable device (200) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (240) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (200) may include a camera (240) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (200).

[0062] Although not shown, in one embodiment, the wearable device (200) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (240). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame and hinge units (206, 207).

[0063] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (200). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0064] In one embodiment, the antenna module (275) can transmit signals or power to the outside of the wearable device (200), or receive signals or power from the outside. The antenna module (275) can be electrically and / or operatively connected to the communication circuitry. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0065] According to one embodiment, the speakers (292-1, 292-2) may output audio signals to the outside of the wearable device (200). The audio output module may be referred to as a speaker. In one embodiment, the speakers (292-1, 292-2) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ears of a user wearing the wearable device (200). For example, the wearable device (200) may include a second speaker (292-2) positioned within the first temple (204) and thus adjacent to the user's left ear, and a first speaker (292-1) positioned within the second temple (205) and thus adjacent to the user's right ear.

[0066] In one embodiment, the light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state to visually provide information regarding a specific state of the wearable device (200) to the user. For example, when the wearable device (200) requires charging, the wearable device (200) may repeatedly emit red light at a designated time. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0067] Referring to FIG. 2B, according to one embodiment, a wearable device (200) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) and the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardware components included in the wearable device (200) may be disposed on the PCB (290). The wearable device (200) may include a flexible PCB (FPCB) for interconnecting the hardware components.

[0068] According to one embodiment, a wearable device (200) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (200) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (200). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., the x-axis, the y-axis, and the z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (200) may identify a user's motion and / or gesture performed to execute or stop a specific function of the wearable device (200) based on the IMU.

[0069] FIGS. 3A and 3B illustrate an example of an exterior appearance of a wearable device (300) according to one embodiment. The wearable device (300) of FIGS. 3A and 3B may be included in the electronic device (101) of FIG. 1. An example of an exterior appearance of a first side (310) of a housing of the wearable device (300) according to one embodiment is illustrated in FIG. 3A, and an example of an exterior appearance of a second side (320) opposite to the first side (310) may be illustrated in FIG. 3B.

[0070] Referring to FIG. 3A, according to one embodiment, a first surface (310) of a wearable device (300) may have a form attachable to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (300) may further include a strap for fixing to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (300) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0071] According to one embodiment, the wearable device (300) may include cameras (340-1, 340-2) for photographing and / or tracking the user's two eyes adjacent to the first display (250-1) and the second display (250-2), respectively. The cameras (340-1, 340-2) may be referred to as ET cameras. According to one embodiment, the wearable device (300) may include cameras (340-3, 340-4) for photographing and / or recognizing the user's face. The cameras (340-3, 340-4) may be referred to as FT cameras.

[0072] Referring to FIG. 3B, a camera (e.g., cameras (340-5, 340-6, 340-7, 340-8, 340-9, 340-10)) and / or a sensor (e.g., depth sensor (330)) for obtaining information related to the external environment of the wearable device (300) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3A. For example, the cameras (340-5, 340-6, 340-7, 340-8, 340-9, 340-10) may be disposed on the second surface (320) to recognize an external object different from the wearable device (300). For example, using cameras (340-9, 340-10), the wearable device (300) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (340-9) can be placed on the second face (320) of the wearable device (300) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (340-10) can be placed on the second face (320) of the wearable device (300) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes.

[0073] According to one embodiment, the wearable device (300) may include a depth sensor (330) disposed on the second surface (320) to identify a distance between the wearable device (300) and an external object. Using the depth sensor (330), the wearable device (300) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (300).

[0074] Although not shown, a microphone may be placed on the second side (320) of the wearable device (300) to acquire sound output from an external object. The number of microphones may be one or more depending on the embodiment.

[0075] As described above, according to one embodiment, the wearable device (300) may have a form factor for being worn on a user's head. The wearable device (300), while worn on the head, may provide a user experience based on augmented reality, virtual reality, and / or mixed reality. Using cameras (340-5, 340-6, 340-7, 340-8, 340-9, 340-10) for recording video of an external space, the wearable device (300) and a server connected to the wearable device (300) (e.g., server (110) of FIG. 1) may provide an on-demand service that provides video of a location and / or place selected by the user, and / or a metaverse service.

[0076] According to one embodiment, the wearable device (300) can display frames acquired through cameras (340-9, 340-10) on the first display (250-1) and the second display (250-2), respectively. The wearable device (300) can provide a user with a mixed user experience of real objects and virtual objects (e.g., video see-through (VST)) by combining virtual objects within frames displayed through the first display (250-1) and the second display (150-2), which include real objects. The wearable device (300) can change the virtual objects based on information acquired by the cameras (340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8) and / or the depth sensor (330). For example, if a visual object corresponding to a real object and a virtual object at least partially overlap within the frame, the wearable device (300) may stop displaying the virtual object based on detecting a motion for interacting with the real object. By stopping displaying the virtual object, the wearable device (300) may prevent the visibility of the real object from being reduced as the visual object corresponding to the real object is occluded by the virtual object.

[0077] FIG. 4 illustrates an example block diagram of a wearable device (401) according to one embodiment. The wearable device (401) of FIG. 4 may correspond to the electronic device (101) of FIG. 1. The wearable device (401) of FIG. 4 may correspond to the wearable device (200) of FIGS. 2A and 2B. The wearable device (401) of FIG. 4 may correspond to the wearable device (300) of FIGS. 3A and 3B.

[0078] Referring to FIG. 4, a wearable device (401) according to one embodiment may include at least one of a processor (410), a memory (415), a display (420), a camera (425), a sensor (430), or a communication circuit (435). The processor (410) of FIG. 4 may correspond to the processor (120) of FIG. 1. The memory (415) of FIG. 4 may correspond to the memory (130) of FIG. 1. The display (420) of FIG. 4 may correspond to the display module (160) of FIG. 1. The camera (425) of FIG. 4 may correspond to the camera module (180) of FIG. 1. The sensor (430) of FIG. 4 may correspond to the sensor module (176) of FIG. 1. The communication circuit (435) of FIG. 4 may correspond to the communication module (190) of FIG. 1.

[0079] The processor (410), memory (415), display (420), camera (425), sensor (430), and communication circuit (435) may be electrically and / or operably coupled with each other by electronic components such as a communication bus (402). The type and / or number of hardware components included in the wearable device (200) are not limited to those illustrated in FIG. 4. For example, the wearable device (401) may include only some of the hardware components illustrated in FIG. 4. Elements (e.g., layers and / or modules) within the memory described below may be logically separated, but are not limited thereto.

[0080] A processor (410) of a wearable device (401) according to one embodiment may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors (410) may be one or more. For example, the processor (410) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.

[0081] The memory (415) of the wearable device (401) according to one embodiment may include a hardware component for storing data and / or instructions input and / or output to the processor (410). The memory (415) may include, for example, a volatile memory such as a random-access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multi media card (eMMC).

[0082] In one embodiment, the display (420) of the wearable device (401) can output visualized information to a user of the wearable device (401). For example, the display (420) can be controlled by a processor (410) including a circuit such as a graphic processing unit (GPU) to output visualized information to the user. The display (420) can include a flat panel display (FPD) and / or electronic paper. The FPD can include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs).

[0083] In one embodiment, the camera (425) of the wearable device (401) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera (425) may be arranged in the form of a two-dimensional array. The camera (425) may acquire electrical signals of each of the plurality of optical sensors substantially simultaneously to generate two-dimensional frame data corresponding to light reaching the optical sensors of the two-dimensional array. For example, photographic data captured using the camera (425) may mean one (a) two-dimensional frame data acquired from the camera (425). For example, video data captured using the camera (425) may mean a sequence of a plurality of two-dimensional frame data acquired from the camera (425) according to a frame rate. The camera (425) may further include a flash light that is positioned toward the direction in which the camera (425) receives light and outputs light toward the direction.

[0084] According to one embodiment, the wearable device (401) may include a plurality of cameras, for example, cameras (425), arranged in different directions. A first camera among the plurality of cameras may be referred to as a motion recognition camera (e.g., motion recognition cameras 240-2 and 240-3 of FIG. 2B ), and a second camera may be referred to as a gaze tracking camera (e.g., gaze tracking camera 240-1 of FIG. 2B ). The wearable device (401) may identify a position, shape, and / or gesture of a hand using an image acquired using the first camera. The wearable device (401) may identify a direction of a gaze of a user wearing the wearable device (401) using an image acquired using the second camera. For example, the direction in which the first camera faces may be opposite to the direction in which the second camera faces.

[0085] According to one embodiment, a sensor (430) of a wearable device (401) may generate electrical information that may be processed by a processor (410) and / or a memory (415) of the wearable device (401) from non-electronic information related to the wearable device (401). The information may be referred to as sensor data. The sensor (430) may include a global positioning system (GPS) sensor, an image sensor, an ambient light sensor, and / or a time-of-flight (ToF) sensor for detecting a geographic location of the wearable device (401), and an inertial measurement unit (IMU) for detecting a physical motion of the wearable device (401).

[0086] In one embodiment, the communication circuit (435) of the wearable device (401) may include hardware components for supporting transmission and / or reception of electrical signals between the wearable device (401) and an external electronic device. The communication circuit (435) may include, for example, at least one of a modem (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (435) may support transmission and / or reception of electrical signals based on various types of protocols, such as Ethernet, a local area network (LAN), a wide area network (WAN), wireless fidelity (WiFi), Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), 5G NR (new radio), and / or 6G.

[0087] According to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (410) of the wearable device (401) may be stored in the memory (415) of the wearable device (401). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable device (401) and / or the processor (410) may perform at least one of the operations of FIGS. 10 to 13 when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. Hereinafter, the fact that an application is installed in a wearable device (401) may mean that one or more instructions provided in the form of an application are stored in a memory (415), and that the one or more applications are stored in a format executable by the processor (410) (e.g., a file having an extension specified by the operating system of the wearable device (401)). As an example, an application may include a program and / or a library related to a service provided to a user.

[0088] Referring to FIG. 4, programs installed in the wearable device (401) may be classified into any one of different layers, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480), based on the target. For example, programs (e.g., modules or drivers) designed to target the hardware of the wearable device (401) (e.g., the display (420), the camera (420), and / or the sensor (430)) may be classified within the hardware abstraction layer (480). The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing an XR (extended reality) service. For example, FIG. 4 illustrates layers being divided within the memory (415), the layers may be logically divided. However, the present invention is not limited thereto. Depending on the embodiment, the layers may be stored in a designated area within the memory (415).

[0089] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or the application layer (440) (e.g., a position tracker (471), a space recognizer (472), a gesture tracker (473), and / or an eye tracker (474), a face tracker (475)) may be classified. Programs classified within the framework layer (450) may provide an executable API (application programming interface) based on other programs.

[0090] For example, within the application layer (440), programs designed to target users controlling wearable devices (401) may be classified. Examples of programs classified into the application layer (440) include, but are not limited to, an XR (extended reality) system UI (user interface) and / or an XR application (442). For example, programs (e.g., software applications) classified into the application layer (440) may call an API (application programming interface) to cause execution of functions supported by programs classified into the framework layer (450).

[0091] For example, the wearable device (401) may display one or more visual objects on the display (420) for performing interaction with a user for using a virtual space based on the execution of the XR system UI (441). A visual object may refer to an object that can be deployed within a screen for transmitting and / or interacting with information, such as text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (401) may provide a service for controlling functions available within a virtual space to the user based on the execution of the XR system UI (441).

[0092] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plug-in (444) are illustrated as being included within the XR system UI (441), but are not limited thereto. For example, the XR system UI (441) may cause execution of functions supported by the lightweight renderer (443) and / or the XR plug-in (444) included within the framework layer (450).

[0093] For example, the wearable device (401) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline that allows partial changes based on the execution of a lightweight renderer (442). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline that allows partial changes. The lightweight renderer (443) may include a renderer that is built prior to the execution of a software application (e.g., a prebuilt renderer). For example, the wearable device (401) may obtain resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute an entire rendering pipeline based on the execution of an XR plug-in (444). The XR plugin (444) may be referred to as an open XR native client from the perspective of defining (or configuring) the entire rendering pipeline.

[0094] For example, the wearable device (401) may display a screen representing at least a portion of a virtual space on the display (420) based on the execution of the XR application (442). The XR plug-in (444-1) included in the XR application (442) may be referenced by the XR plug-in (444) of the XR system UI (441). Any description of the XR plug-in (444-1) that overlaps with the description of the XR plug-in (444) may be omitted. The wearable device (401) may cause the execution of the virtual space manager (451) based on the execution of the XR application (442).

[0095] According to one embodiment, the wearable device (101) may provide a virtual space service based on the execution of the virtual space manager (451). For example, the virtual space manager (451) may include a platform (e.g., an Android platform) for supporting the virtual space service. The wearable device (401) may display the posture of a virtual object representing the user's posture rendered using data acquired through the sensor (430) on the display based on the execution of the virtual space manager (451). The virtual space manager (451) may be referred to as a composition presentation manager (CPM).

[0096] For example, the virtual space manager (451) may include a runtime service (452). As an example, the runtime service (452) may be referred to as an OpenXR runtime module. The wearable device (401) may be used to provide at least one of a pose prediction function, a frame timing function, and / or a spatial input function to a user through the wearable device (401) based on the execution of the runtime service (452). As an example, the wearable device (401) may be used to perform rendering for a virtual space service to a user based on the execution of the runtime service (452). For example, an application (e.g., unity or an OpenXR native application) may be implemented based on the execution of the runtime service (452).

[0097] For example, the virtual space manager (451) may include a pass-through manager (453). Based on the execution of the pass-through manager (453), the wearable device (401) may display a screen representing a virtual space on the display (420), while superimposing another screen representing an actual space acquired through the camera (425) on at least a portion of the screen.

[0098] For example, the virtual space manager (451) may include an input manager (454). The wearable device (401) may identify data (e.g., sensor data) acquired by executing one or more programs included in the recognition service layer (470) based on the execution of the input manager (454). The wearable device (401) may initiate execution of at least one of the functions of the wearable device (401) using the acquired data.

[0099] For example, the perception abstract layer (460) can be used for data exchange between the virtual space manager (451) and the perception service layer (470). From the perspective of being used for data exchange between the virtual space manager (451) and the perception service layer (470), the perception abstract layer (460) can be referred to as an interface. For example, the perception abstract layer (460) can be referenced as OpenPX. The perception abstract layer (460) can be used for a perception client and a perception service.

[0100] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data acquired from a sensor (430) (or a camera (425)). The one or more programs may include at least one of a position tracker (471), a space recognizer (472), a gesture tracker (473), an eye tracker (474), and / or a face tracker (473). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those illustrated in FIG. 4.

[0101] For example, the wearable device (401) can identify the pose of the wearable device (401) using the sensor (430) based on the execution of the position tracker (471). The wearable device (401) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (401) using data acquired using the camera (425) and the IMU based on the execution of the position tracker (471). The position tracker (471) can be referred to as a head tracking (HeT) module.

[0102] For example, the wearable device (401) may be used to construct a three-dimensional virtual space around the wearable device (401) (or a user of the wearable device (401)) based on the execution of the space recognizer (472). The wearable device (401) may reconstruct the three-dimensional surroundings of the wearable device (401) using data acquired using the camera (425) based on the execution of the space recognizer (472). The wearable device (401) may identify at least one of a plane, a slope, and stairs based on the three-dimensionally reconstructed surroundings of the wearable device (401) based on the execution of the space recognizer (472). The space recognizer (472) may be referred to as a scene understanding (SU) module.

[0103] For example, the wearable device (401) may be used to identify (or recognize) a pose and / or gesture of a hand of a user of the wearable device (401) based on the execution of the gesture tracker (473). As an example, the wearable device (401) may identify a pose and / or gesture of a hand of a user using data acquired from a sensor (430) based on the execution of the gesture tracker (473). As an example, the wearable device (401) may identify a pose and / or gesture of a hand of a user based on data (or images) acquired using a camera based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0104] For example, the wearable device (401) may identify (or track) eye movements of a user of the wearable device (401) based on the execution of the gaze tracker (474). As an example, the wearable device (401) may identify eye movements of the user using data acquired from at least one sensor based on the execution of the gaze tracker (474). As an example, the wearable device (401) may identify eye movements of the user based on data acquired using a camera (e.g., the gaze tracking camera (240-1) of FIG. 2B) and / or an infrared light emitting diode (IR LED) based on the execution of the gaze tracker (474). The gaze tracker (474) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0105] For example, the recognition service layer (470) of the wearable device (401) may further include a face tracker (475) for tracking the user's face. For example, the wearable device (103) may identify (or track) the movement of the user's face and / or the user's expression based on the execution of the face tracker (475). The wearable device (401) may estimate the user's expression based on the movement of the user's face based on the execution of the face tracker (475). As an example, the wearable device (401) may identify the movement of the user's face and / or the user's expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker (475).

[0106] FIG. 5 illustrates an example of the appearance of a user (501) and the appearance of an avatar (505) of the user (501).

[0107] FIG. 5 may be described with reference to FIG. 1, FIG. 2a, FIG. 2b, FIG. 3a, FIG. 3b, or FIG. 4.

[0108] Referring to FIG. 5, the wearable device (401) can capture an image including a part of the body of a user (501) wearing the wearable device (401) through the camera (425). In one embodiment, the wearable device (401) can obtain an image of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501) through the camera (425).

[0109] In one embodiment, the wearable device (401) can acquire an image of the face (511) of the user (501) through a camera (425) (e.g., a face tracking (FT) camera, cameras (340-3, 340-4) of FIG. 3B) arranged toward the user's face (511).

[0110] In one embodiment, the wearable device (401) may acquire images of the user's (501) arms (521, 541), legs (531, 551), and / or body (561) through cameras (425) arranged toward the user's (501) field of view (FoV) for motion recognition (e.g., cameras (240-2, 240-3) of FIG. 2B, cameras (340-5, 340-6, 340-7, 340-8, 340-9, 340-10) of FIG. 3B).

[0111] In one embodiment, the wearable device (401) can generate and / or update an avatar (505) based on an image acquired for the user (501). In one embodiment, the wearable device (401) can generate and / or update an avatar appearance for the face (515), arms (525, 545), legs (535, 555), and / or body (565) of the avatar (505) based on an image acquired for the user (501). In one embodiment, the wearable device (401) can generate and / or update an avatar appearance for the face (515), arms (525, 545), legs (535, 555), and / or body (565) of the avatar (505) based on a user appearance for the user (501). In one embodiment, the user appearance may refer to the body shape (e.g., shape, size, length, area) or texture (e.g., color, wrinkles, scars, eyebrows, hair, beard of the user's skin) of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561). The user appearance may refer to accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) located on the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561). In one embodiment, the avatar appearance may refer to the body shape (e.g., shape, size, length, area) or texture (e.g., color, wrinkles, scars, eyebrows, hair, beard of the user's skin) of the face (515), arms (525, 545), legs (535, 555), and / or body (565) of the avatar (505). The avatar appearance may refer to accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) located on the face (515), arms (525, 545), legs (535, 555), and / or body (565) of the avatar (505).

[0112] In one embodiment, the wearable device (401) may create and / or update an avatar (505) having landmarks corresponding to landmarks of the user (501). Here, the landmarks of the user (501) may include feature regions of the user's (501) body (e.g., lips, face outline, ears, hands, feet, chest). The landmarks of the avatar (505) may include regions on the avatar (505) that correspond to feature regions of the user's (501) body (e.g., lips, face outline, ears, hands, feet, chest).

[0113] In one embodiment, the wearable device (401) can create and / or update an avatar (505) having a body shape corresponding to the body shape (e.g., shape, size, length, area) of the user (501). For example, the wearable device (401) can create and / or update an avatar (505) having proportions corresponding to the body proportions of the user (501). For example, the avatar (505) can have a body shape that is reduced from that of the user (501). For example, the ratio between the height and arm length of the avatar (505) can be substantially the same as the ratio between the height and arm length of the user (501). However, the present invention is not limited thereto.

[0114] In one embodiment, the wearable device (401) can create and / or update an avatar (505) having a texture corresponding to the texture of the user (501) (e.g., the color, wrinkles, scars, eyebrows, hair, and beard of the user's skin). For example, the wearable device (401) can create and / or update an avatar (505) having a skin color corresponding to the skin color of the user (501). For example, the avatar (505) can create and / or update an avatar (505) having a hair style corresponding to the hair style of the user (501). Here, the corresponding style may mean that at least one of color, length, or shape corresponds, but is not limited thereto.

[0115] In one embodiment, the wearable device (401) may create and / or update an avatar (505) having accessories corresponding to the user's (501) accessories (e.g., belongings, jewelry, clothing, body paint, or makeup). For example, the wearable device (401) may create and / or update an avatar (505) having a hat having a style corresponding to the style of a hat worn by the user (501). For example, the avatar (505) may create and / or update an avatar (505) having gloves having a style corresponding to the style of gloves worn by the user (501) on a specific location (e.g., left arm (541)) applied to a specific location (e.g., left arm (545)). However, the present invention is not limited thereto. Here, corresponding styles may mean corresponding at least one of color, length, or shape. However, the present invention is not limited thereto.

[0116] In one embodiment, the wearable device (401) may set appearance properties of the avatar (505) so that the avatar (505) has an avatar appearance corresponding to the user appearance of the user (501). Here, the appearance properties may include a mesh of the avatar (505), a blendshape of the avatar (505), a texture of the avatar (505), and / or an asset of the avatar (505). For example, an appearance property related to a mesh of the avatar (505) may include information about the positions of vertices. For example, an appearance property related to a blendshape of the avatar (505) may be a set of parameters representing the appearance and / or facial expression of the avatar (505). For example, an appearance attribute related to a blendshape of an avatar (505) may include parameters indicating the position and / or movement of each of a plurality of regions of the face of the avatar (505) (mouth region, chin region, nose region, right cheek region, left cheek region, right eye region, and / or left eye region). For example, an appearance attribute related to a texture of the avatar (505) may include information indicating the type of texture (e.g., wrinkle, scar, beard, hair), texture (or color), and / or appearance (e.g., beard style, hair style). For example, an appearance attribute related to an asset of the avatar (505) may include information indicating the type of asset (e.g., cast, glasses, brooch, bracelet, watch, cell phone, bag), texture (or color), strength (or elasticity), and / or appearance. In one embodiment, the asset may be an image object of the avatar (505) corresponding to an accessory of the user (501).

[0117] In one embodiment, the wearable device (401) can update some of the appearance properties of the avatar (505) so that the avatar (505) has an avatar appearance that corresponds to the user appearance of the user (501). For example, the wearable device (401) can update at least one of a mesh of the avatar (505), a blendshape of the avatar (505), a texture of the avatar (505), or an asset of the avatar (505). For example, the wearable device (401) can update at least one of a mesh, a blendshape, a texture, or an asset for at least one body region of the avatar (505) including the face (515), arms (525, 545), legs (535, 555), or body (565). For example, the wearable device (401) can update at least one of a mesh, blendshape, texture, or asset for a portion of the body of the avatar (505) (e.g., a portion of the face (515) (e.g., an eyelid)).

[0118] In one embodiment, the wearable device (401) may determine whether the appearance properties of the avatar (505) are updated. In one embodiment, the wearable device (401) may update some of the appearance properties of the avatar (505) based on the determination that the appearance properties of the avatar (505) are updated.

[0119] Hereinafter, with reference to FIG. 6, FIG. 7, or FIG. 8, an example of a wearable device (401) determining whether to update the appearance properties of an avatar (505) and updating some of the appearance properties of the avatar (505) will be described.

[0120] FIG. 6 illustrates an example of landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640) of a user face (511) of a user (501) and an avatar face (515) of an avatar (505).

[0121] FIG. 6 may be described with reference to FIG. 1, FIG. 2a, FIG. 2b, FIG. 3a, FIG. 3b, FIG. 4, or FIG. 5.

[0122] Referring to FIG. 6, the wearable device (401) can identify landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). In one embodiment, the wearable device (401) can acquire landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) from an image acquired for the user (501). In one embodiment, the wearable device (401) can acquire landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501). For example, the landmarks of the face (511) of the user (501) can include, but are not limited to, the crown of the head (611), the under-eye area (612, 620), and the jawline (613, 614, 615, 616, 617 618, 619). The landmarks of the face (511) of the user (501) can include, but are not limited to, lips, eyes, eyebrows, ears, or nose. For example, landmarks of the arms (521, 541) of the user (501) may include a hand, a finger, a wrist, or a shoulder. For example, landmarks of the legs (531, 551) of the user (501) may include a foot, an ankle, a knee, a thigh, or a calf. For example, landmarks of the body (561) of the user (501) may include, but are not limited to, a neck, a chest, an armpit, or a buttocks.

[0123] In one embodiment, the wearable device (401) may identify only some landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) that are not covered by the accessory (or are exposed to the outside). For example, in one embodiment, the wearable device (401) can identify landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) except for landmarks (615, 616, 617) that are covered by a specific accessory (e.g., mask, clothes).

[0124] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated by comparing the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) and the landmarks (631, 632, 633, 634, 635, 636, 637, 638, 639, 640) of the avatar (505). For example, the wearable device (401) can identify whether the avatar appearance is updated based on the locations of the landmarks of the user (501) and the landmarks of the avatar (505) that correspond to each other.

[0125] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on a first distance and / or a first inclination between two landmarks of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on a second distance and / or a second inclination between two landmarks of the avatar (505). For example, the wearable device (401) can identify whether the avatar appearance is updated based on a first distance and / or a first inclination between two corresponding landmarks of the user (501) and a second distance and / or a second inclination between two landmarks of the avatar (505). For example, the wearable device (401) may identify whether landmarks (615, 617) and / or areas adjacent to landmarks (615, 617) on the avatar (505) are updated based on a first distance and / or a first slope between landmarks (615, 617) of the user (501) and a second distance and / or a second slope between landmarks (635, 637) on the avatar (505). For example, the area adjacent to a landmark may be an area within a specified distance from the landmark. Here, for comparison between the first distance and the second distance, the first distance may be reduced or the second distance may be increased based on a ratio between the user (501) and the avatar (505). Hereinafter, the first distance or the second distance may be distances that are changed based on a ratio between the user (501) and the avatar (505) for comparison between them.

[0126] In one embodiment, the wearable device (401) can identify whether to update at least two landmarks of the avatar (505) and / or an area adjacent to the at least two landmarks based on comparing a difference between a first distance between at least two landmarks of the user (501) and a second distance between at least two landmarks of the avatar (505) with a reference distance difference. For example, the wearable device (401) can identify to update a landmark and / or an area adjacent to the landmark for the second distance on the avatar (505) based on a difference between the first distance and the second distance exceeding the reference distance difference. For example, the wearable device (401) can identify to not update a landmark and / or an area adjacent to the landmark for the second distance on the avatar (505) based on a difference between the first distance and the second distance being less than or equal to the reference distance difference.

[0127] In one embodiment, the wearable device (401) can identify whether to update at least two landmarks of the avatar (505) and / or an area adjacent to the at least two landmarks based on comparing a difference between a first slope between at least two landmarks of the user (501) and a second slope between at least two landmarks of the avatar (505) with a reference slope difference. For example, the wearable device (401) can identify to update a landmark and / or an area adjacent to the landmark for the second slope in the avatar (505) based on a difference between the first slope and the second slope exceeding the reference slope difference. For example, the wearable device (401) can identify to not update a landmark and / or an area adjacent to the landmark for the second slope in the avatar (505) based on a difference between the first slope and the second slope being less than or equal to the reference slope difference.

[0128] In one embodiment, the wearable device (401) can update the avatar appearance such that a second distance and / or a second slope between two landmarks of corresponding avatars (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501). In one embodiment, the wearable device (401) can update the avatar appearance such that a second distance and / or a second slope between two landmarks of corresponding avatars (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501) based on identifying the avatar (505) as being updated. For example, the wearable device (401) may update landmarks (615, 617) of the avatar (505) and / or areas adjacent to landmarks (615, 617) such that a second distance and / or a second slope between landmarks (635, 637) of the avatar (505) corresponds to a first distance and / or a first slope between landmarks (615, 617) of the user (501).

[0129] In one embodiment, the wearable device (401) can update an appearance property associated with a mesh of the avatar (505). In one embodiment, the wearable device (401) can update an appearance property associated with a mesh of the avatar (505) such that a second distance and / or a second slope between two corresponding landmarks of the avatar (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501). In one embodiment, the wearable device (401) can update information about the positions of vertices associated with two landmarks of the avatar (505) such that a second distance and / or a second slope between two corresponding landmarks of the avatar (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501).

[0130] In one embodiment, the wearable device (401) can update the appearance properties of the avatar (505) associated with the blendshape. In one embodiment, the wearable device (401) can update the appearance properties of the avatar (505) associated with the blendshape based on the locations of landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). In one embodiment, the wearable device (401) may update the appearance properties of the avatar (505) associated with the blendshape in response to changes in the positions of the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). For example, when the first distance between the landmarks (615, 617) of the user (501) increases, the wearable device (401) may update the appearance properties of the avatar (505) associated with the blendshape such that the lips of the face of the avatar (505) are replaced with horizontally long lips. For example, the wearable device (401) may update the appearance properties of the avatar (505) associated with the blendshape so that the inclination of the lips of the avatar's face (505) changes according to the first inclination when the first inclination between the landmarks (615, 617) of the user (501) changes. For example, the wearable device (401) may update the position and / or inclination information of the blendshape of the avatar (505) based on the first distance and / or the first inclination of the user (501).

[0131] As described above, the wearable device (401) can update the avatar (505) using fewer resources (e.g., time, power) than creating another avatar to replace the avatar (505) by determining whether the appearance properties of the avatar (505) need to be updated.

[0132] According to an embodiment, the wearable device (401) may obtain body type information (e.g., body profile information) of the user (501) from an external electronic device (e.g., electronic device (102)). The wearable device (401) may update the avatar appearance based on the body type information (e.g., body profile information) of the user (501). The wearable device (401) may update the avatar appearance so that the avatar (505) has a body type corresponding to the body type of the user (501) indicated by the body type information (e.g., body profile information) of the user (501).

[0133] Figure 7 illustrates an example of a change in texture of a user (501).

[0134] FIG. 7 may be described with reference to FIG. 1, FIG. 2a, FIG. 2b, FIG. 3a, FIG. 3b, FIG. 4, FIG. 5, or FIG. 6.

[0135] In one embodiment, the wearable device (401) may identify whether to reflect textures (e.g., color, wrinkles, scars, eyebrows, hair, beard) of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561) in the avatar appearance based on comparing the image (705) of the user (501) with a previously acquired image (701) used to generate the avatar (505).

[0136] In one embodiment, the wearable device (401) can identify whether to update the avatar appearance based on the similarity (e.g., structural similarity index measure (SSIM)) between the image (705) of the user (501) and a previously acquired image (701) used to generate the avatar (505). For example, the wearable device (401) can identify whether to update the avatar appearance (or the texture of the avatar (505)) based on the similarity between the image (705) of the user (501) and a previously acquired image (701) of the user (501) used to generate the avatar (505). For example, the wearable device (401) can identify not to update the avatar appearance (or the texture of the avatar (505)) based on the similarity between the image (705) and the image (701) exceeding a reference similarity. For example, based on the similarity between the image (705) and the image (701) being less than or equal to a reference similarity, the wearable device (401) may identify that the avatar appearance (or the texture of the avatar (505)) needs to be updated. However, this is not limited thereto. For example, the wearable device (401) may obtain the similarity between the images of the face (511) of the user (501) based on another similarity measurement algorithm.

[0137] In one embodiment, the wearable device (401) can identify a user's appearance in an image. In one embodiment, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561) in the image. For example, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) identified in the image. In one embodiment, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around the remaining landmarks (611, 612, 613, 614, 618, 619, 620) when some of the landmarks (615, 616, 617) are covered by an accessory (or not exposed to the outside).

[0138] In one embodiment, the wearable device (401) can identify a texture at a location of the avatar (505) corresponding to the location where the texture of the user (501) is identified. For example, if the user's (501) beard is identified around a landmark (616), the wearable device (401) can identify whether the avatar's (505) beard is present around a landmark (636) of the avatar (505) corresponding to the landmark (616).

[0139] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the texture of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on comparing the texture of the user (501) and the texture of the avatar (505) at corresponding locations. For example, the wearable device (401) can identify whether the avatar appearance is updated by comparing the beard of the user (501) around the landmark (616) and the beard of the avatar (505) around the landmark (636) of the avatar (505). For example, if the beard of the avatar (505) corresponding to the beard of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the beard of the user (501) is different from the type (or appearance) of the beard of the avatar (505), the wearable device (401) can identify that the avatar appearance is updated. For example, if the beard of the avatar (505) corresponding to the beard of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the beard of the user (501) is not different from the type (or appearance) of the beard of the avatar (505), the wearable device (401) can identify that the avatar appearance is not updated. However, the present invention is not limited thereto.

[0140] In one embodiment, the wearable device (401) can change the appearance properties associated with the texture of the avatar (505) based on the texture of the user (501) identified in the image. In one embodiment, if the texture of the user (501) identified in the image is different from the texture of the avatar (505), the wearable device (401) can update the texture of the avatar (505) to correspond to the texture of the user (501). In one embodiment, the wearable device (401) can identify a style of the type of avatar (505) that corresponds to the texture of the user (501) identified in the image. In one embodiment, the wearable device (401) can change the style of the identified avatar (505) to the new style of the avatar (505).

[0141] For example, the wearable device (401) may update the style of the tattoo and / or painting (e.g., face painting) of the face (515) of the avatar (505) based on the style of the tattoo and / or painting (e.g., face painting) of the user (501) identified in the image, if the style of the tattoo and / or painting (e.g., face painting) of the face (515) of the avatar (505) is different from the style of the tattoo and / or painting (e.g., face painting) of the face (515) of the user (501). For example, the wearable device (401) may update the style of the hair of the face (515) of the avatar (505) based on the style of the hair of the face (511) of the user (501) identified in the image, if the style of the hair of the face (515) of the avatar (505) is different from the style of the hair of the face (515) of the user (501). For example, if the hair of the face (511) of the user (501) identified in the image is short and the hair of the face (515) of the avatar (505) is long, the wearable device (401) can update the short hair style of the avatar (505) corresponding to the hair style of the user (501) to the hair style of the face (515) of the avatar (505). For example, if the color of the face (511) of the user (501) identified in the image is dark gray-orange (e.g., #E2CBB4) and the color of the face (515) of the avatar (505) is light gray-orange (e.g., #AD957D), the wearable device (401) can update the color of the face (515) of the avatar (505) to the color of the face (511) of the user (501). For example, the wearable device (401) may apply a double couple to the eyes of the face (515) of the avatar (505) if a double couple exists in the face (511) of the user (501) identified in the image and no double couple exists in the face (515) of the avatar (505).

[0142] In FIG. 7, it is described that the wearable device (401) updates the avatar appearance based on comparing the image (705) of the user (501) with the previously acquired image (701) used to create the avatar (505), but is not limited thereto. In one embodiment, the wearable device (401) can identify whether the avatar appearance (or the texture of the avatar (505)) is updated based on comparing the time when the avatar (505) was created with the current time. For example, the wearable device (401) can identify whether the avatar appearance (or the texture of the avatar (505)) is updated based on the time elapsed since the time when the avatar (505) was created. For example, the wearable device (401) can update the aging degree of the avatar (505) based on the time elapsed since the time when the avatar (505) was created. For example, the wearable device (401) may generate at least one wrinkle on a body part (e.g., face (515)) of the avatar (505) based on the time elapsed from the time the avatar (505) was created. For example, the wearable device (401) may flex a body part (e.g., leg (555)) of the avatar (505) based on the time elapsed from the time the avatar (505) was created. However, the present invention is not limited thereto.

[0143] Figure 8 illustrates an example of an asset of an avatar (505) corresponding to the belongings of a user (501).

[0144] In one embodiment, the wearable device (401) can identify accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) located on the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501). In one embodiment, the wearable device (401) can identify accessories in images of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501) via the camera (425). In one embodiment, the wearable device (401) can identify accessories of the user (501) by their appearance in images of parts of the body of the user (501).

[0145] In one embodiment, the wearable device (401) can identify accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) around landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) identified in the image.

[0146] In one embodiment, the wearable device (401) can identify whether at least one of the avatar appearance or the avatar animation of the avatar is updated based on comparing data related to the user appearance of the user (501) identified in the image with data related to the avatar appearance of the avatar (505).

[0147] In one embodiment, the wearable device (401) can identify an asset corresponding to an accessory at a location of the avatar (505) corresponding to a location where the accessory of the user (501) is identified. For example, if the user's (501) glasses are identified around landmarks (612, 618), the wearable device (401) can identify whether an asset corresponding to the glasses of the avatar (505) exists around landmarks (632, 638) of the avatar (505) corresponding to landmark (616).

[0148] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the accessories of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on comparing the accessories of the user (501) and the assets of the avatar (505) at corresponding locations. For example, the wearable device (401) can identify whether the avatar appearance is updated by comparing the glasses of the user (501) around landmarks (612, 618) and the glasses of the avatar (505) around landmarks (632, 638) of the avatar. For example, if the glasses of the avatar (505) corresponding to the glasses of the user (501) are not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the glasses of the user (501) is different from the type (or appearance) of the glasses of the avatar (505), the wearable device (401) can identify that the avatar appearance is updated. For example, if the glasses of the avatar (505) corresponding to the glasses of the user (501) are not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the glasses of the user (501) is not different from the type (or appearance) of the glasses of the avatar (505), the wearable device (401) can identify that the avatar appearance is not updated. However, the present invention is not limited thereto.

[0149] In one embodiment, the wearable device (401) may change the appearance properties associated with the asset of the avatar (505) based on the accessory of the user (501) identified in the image. In one embodiment, if the accessory of the user (501) identified in the image is different from the asset of the avatar (505), the wearable device (401) may update the asset of the avatar (505) to correspond to the accessory of the user (501).

[0150] In one embodiment, the wearable device (401) can identify an asset of a type of avatar (505) corresponding to an accessory of the user (501) identified in the image. In one embodiment, the wearable device (401) can apply the asset of the identified avatar (505) to the avatar (505).

[0151] For example, if the style of clothes (e.g., a skirt) worn by a user (501) identified in the image is different from the style of clothes (e.g., pants) applied to an avatar (505), the wearable device (401) may update the style of clothes applied to an avatar (505) based on the style of clothes (e.g., a skirt) worn by the user (501). For example, if the style of a hat worn on a face (511) of a user (501) identified in the image is different from the style of a hat applied to a face (515) of an avatar (505), the wearable device (401) may update the style of a hat applied to a face (515) of an avatar (505) based on the style of the hat worn on the face (511). For example, if the right hand of the right arm (521) of the user (501) identified in the image is holding a smartphone, and the right hand of the right arm (525) of the avatar (505) is not holding any object (e.g., a smartphone), the wearable device (401) may update the avatar (505) so that an asset corresponding to the smartphone is applied to the right hand of the right arm (525) of the avatar (505). For example, referring to FIG. 8, if the left arm (541) of the user (501) identified in the image is in a cast, and the left arm (545) of the avatar (505) is not in a cast, the wearable device (401) may update the avatar (505) so that an asset corresponding to the cast is applied to the left arm (545) of the avatar (505).

[0152] Figure 9 illustrates an example of an animation of an avatar (505) corresponding to the belongings of a user (501).

[0153] In one embodiment, the wearable device (401) can identify whether an animation is to be updated based on properties of the accessory and / or asset (802). Here, the properties can indicate the location, intensity, and / or type of the accessory and / or asset (802).

[0154] In one embodiment, the wearable device (401) may identify the accessory as refreshing the animation if it is included in the type that restricts the movement of the user (501). In one embodiment, the wearable device (401) may identify the accessory as refreshing the animation if it restricts the movement of a part of the user's (501) body. In one embodiment, the wearable device (401) may identify the accessory as refreshing the animation based on an attribute associated with the type of the accessory indicating the type that restricts the movement of a part of the user's (501) body. In one embodiment, the wearable device (401) may identify the accessory as refreshing the animation associated with a part of the body of an avatar (505) corresponding to a part of the user's (501) body if it restricts the movement of a part of the user's (501) body. Here, the animation related to a part of the body of the avatar (505) may be an animation that includes the movement of a part of the body of the avatar (505) when the animation is played.

[0155] For example, if the accessory is a cast, the wearable device (401) may identify that the cast is a type that restricts the movement of the user (501). For example, if the accessory is a cast, the wearable device (401) may identify that the accessory is a cast by updating an animation that includes a movement of a body part of the avatar (505) corresponding to the body part of the user (501) on which the cast is positioned (e.g., neck, arm, or leg) (e.g., bending the neck, swinging the arm, sitting on the floor).

[0156] For example, if the accessory is a skirt, the wearable device (401) may identify that the skirt is included in the type that restricts the movement of the user (501). For example, if the accessory is a skirt, the wearable device (401) may identify that the accessory updates an animation that includes the movement of a body part of the avatar (505) corresponding to the body part of the user (501) where the skirt is positioned (e.g., a leg) (e.g., a sitting motion on the floor, a running motion).

[0157] For example, if the accessory is a smartphone, the wearable device (401) may identify that the smartphone is included in the type of movement that restricts the user's (501) movement. For example, if the accessory is a smartphone, the wearable device (401) may identify that the accessory updates an animation that includes a movement of a body part of the avatar (505) corresponding to a body part of the user (501) where the smartphone is positioned (e.g., a hand) (e.g., a hand waving motion, a hand opening motion).

[0158] In one embodiment, the wearable device (401) can change the movement properties of the animation identified as being updated. For example, the wearable device (401) can change the movement properties of a body part of an avatar (505) corresponding to a body part of a user (501) where an accessory is positioned. For example, the wearable device (401) can change the movement properties of a body part of an avatar (505) to which an asset is applied. Here, the movement properties can include at least one of a movement direction, a movement speed, a movement range, or a movement order. For example, the movement direction can mean directions in which a skeletal mesh can move through a joint (e.g., a joint) included in the body of the avatar (505). For example, the direction of movement of a skeletal mesh corresponding to the left upper arm of the avatar (505) through a joint of the avatar (505) corresponding to the left shoulder part of the user (501) may include at least one direction of flexion, extension, abduction, adduction, horizontal flexion, or horizontal extension. For example, the movement speed may mean the movement speed in the movement directions of the skeletal mesh through a joint (e.g., a joint) included in the body of the avatar (505). For example, the movement range may mean an angular range in the movement directions of the skeletal mesh through a joint (e.g., a joint) included in the body of the avatar (505). For example, the angular range of the skeletal mesh corresponding to the left upper arm of the avatar (505) through the joint of the avatar (505) corresponding to the left shoulder part of the user (501) may be 0 to 180 degrees when raised forward (or shoulder flexion).For example, the angular range of motion of the skeletal mesh corresponding to the left upper arm of the avatar (505) through the joint of the avatar (505) corresponding to the left shoulder of the user (501) may be from 0 to 60 degrees when lifting it backward (or shoulder extension). However, the invention is not limited thereto. For example, the movement order may refer to the movement orders between two or more skeletal meshes through two or more joints included in the body of the avatar (505). For example, in the case of a motion of raising the left arm and shaking the left hand, the movement order of the skeletal mesh corresponding to the upper arm for the motion of raising the left arm through the shoulder and the movement order of the skeletal mesh corresponding to the lower arm for the motion of shaking the left arm through the elbow may refer to. However, the invention is not limited thereto. Here, the skeletal mesh may define the connection relationship (or, through the joint) between the skeletal meshes by the skeleton tree. A skeleton tree can define the hierarchical structure of skeletal meshes and their connections. For example, a skeleton tree could connect the clavicle as a top-level tree to the spine, as a subtree to the upper arm, which in turn connects to the lower arm as a subtree, and the lower arm as a subtree to the hand.

[0159] In one embodiment, the wearable device (401) can change the movement properties of the animation associated with the joint of the avatar (505). In one embodiment, the wearable device (401) can change at least one of the movement direction, movement speed, movement range, or movement order of the animation associated with the joint of the avatar (505). For example, the wearable device (401) can change the movement properties of a joint and / or skeletal mesh included in a body part of the avatar (505) corresponding to a body part of the user (501) where the accessory is positioned. For example, the wearable device (401) can change the movement properties of a joint and / or skeletal mesh included in a body part of the avatar (505) based on the properties of the asset (802) corresponding to the accessory. For example, the wearable device (401) can change the movement properties of the skeletal mesh based on the location, strength, and / or type of the asset (802). For example, the wearable device (401) can decrease the movement speed and / or movement range inversely proportional to the strength of the asset (802). For example, the wearable device (401) can decrease the movement speed and / or movement range of the skeletal mesh corresponding to the location of the asset (802) based on the location of the asset (802). For example, as shown in FIG. 8, when an asset (802) corresponding to a cast is attached to the left arm (545) of an avatar (505), the wearable device (401) can change the movement properties of the joints (e.g., elbow, wrist) and the skeletal mesh (e.g., forearm, hand) included in the left arm (545) to which the asset (802) is attached. For example, the wearable device (401) may limit the range of movement of a skeletal mesh corresponding to the lower arm and / or hand. For example, referring to FIG. 9, the wearable device (401) may update an animation (901) including an action of an avatar (505) raising and waving its left hand to an animation (903) including an action of an avatar (505) raising its left hand.For example, the wearable device (401) can generate an animation (903) that does not include a motion of shaking the lower arm among the motions of lifting the upper arm and shaking the lower arm included in the animation (901) by limiting the range of movement of the skeletal mesh corresponding to the lower arm and / or the hand.

[0160] In one embodiment, the wearable device (401) may change the joints and / or skeletal meshes of the avatar (505) to which the animation is applied. For example, the wearable device (401) may change the joints and / or skeletal meshes of the avatar (505) to which the animation is applied based on an identification that an animation update is required. For example, the wearable device (401) may change the joints and / or skeletal meshes of the avatar (505) to which the animation is applied to such that the animation is performed through joints and / or skeletal meshes at opposite positions. For example, the wearable device (401) may identify a first joint of the avatar (505) (e.g., elbow, wrist of the left arm (545)) corresponding to a location where the accessory is attached to the user (501) and may identify a second joint of the avatar (505) (e.g., elbow, wrist of the right arm (525)) corresponding to a location opposite to a location where the accessory is attached to the user (501). For example, the wearable device (401) may update the movement properties of the animation so that the animation related to the first joint and / or the skeletal mesh is performed through the skeletal meshes for the second joint (e.g., elbow, wrist of the right arm (525)) and / or the second joint. For example, referring to FIG. 9, the wearable device (401) can update an animation (901) including a motion of an avatar (505) raising and waving a left hand to an animation (905) including a motion of an avatar (505) raising and waving a right hand. For example, the wearable device (401) can generate an animation (905) including a motion of raising a left upper arm and a motion of shaking a left lower arm, and a motion of raising a right upper arm and a motion of shaking a right lower arm, by limiting the range of movement of a skeletal mesh corresponding to the lower arm and / or the hand.

[0161] In FIG. 9, the wearable device (401) is described as updating the animation of the avatar (505) through the accessory of the user (501), but is not limited thereto. In one embodiment, the wearable device (401) may update the movement properties of the animation of the avatar (505) based on comparing the time at which the avatar (505) was created with the current time. For example, the wearable device (401) may update the aging level of the avatar (505) based on the time elapsed since the time at which the avatar (505) was created. For example, the wearable device (401) may reduce the movement speed of the animation of the avatar (505) based on the time elapsed since the time at which the avatar (505) was created.

[0162] In one embodiment, the wearable device (401) may acquire an image including multiple images of the arms (521, 541), legs (531, 551), and / or body (561) of the user (501) through cameras (425) arranged toward the field of view (FoV) of the user (501) for motion recognition (e.g., cameras (240-2, 240-3) of FIG. 2B and cameras (340-5, 340-6, 340-7, 340-8, 340-9, 340-10) of FIG. 3B). In one embodiment, the wearable device (401) may identify whether an update of the avatar animation of the avatar (505) is required based on comparing the user movements of the user (501) identified in the image with the avatar animation of the avatar (505). For example, the wearable device (401) can identify an update of an animation related to the gait of the avatar (505) by comparing the gait of the user (501) identified in the video with the gait of the avatar (505). For example, the wearable device (401) can update an animation related to the gait of the avatar (505) to correspond to the gait of the user (501) identified in the video. For example, the wearable device (401) can update an animation related to the gait of the avatar (505) to correspond to the gait of the user (501) by changing the movement order and movement properties of joints related to the legs and skeletal meshes related to the joints.

[0163] FIG. 10 is a flowchart showing the operation of a wearable device (401) according to one embodiment.

[0164] FIG. 10 may be described with reference to FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, or FIG. 9.

[0165] Referring to FIG. 10, in operation 1010, the wearable device (401) can acquire an image.

[0166] In one embodiment, the wearable device (401) can capture an image including a part of the body of a user (501) wearing the wearable device (401) through the camera (425). In one embodiment, the wearable device (401) can obtain an image of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501) through the camera (425).

[0167] In one embodiment, the wearable device (401) may obtain the image before executing an application for displaying an avatar (505). For example, the wearable device (401) may obtain the image when a user wears the wearable device (401). For example, the wearable device (401) may obtain the image after the camera (425) is activated while the user (501) is wearing the wearable device (401). Here, the activation of the camera (425) may mean that power is supplied to the camera (425). The activation of the camera (425) may mean that the camera (425) is operated by the framework layer (450). The activation of the camera (425) may mean that the camera (425) obtains the image by an application included in a different framework layer (450) than the application for displaying the avatar (505).

[0168] In operation 1020, the wearable device (401) may identify a user appearance from an image. In one embodiment, the user appearance may refer to a body shape (e.g., shape, size, length, area) or texture (e.g., color, wrinkles, scars, eyebrows, hair, beard of the user's skin) of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561). The user appearance may refer to accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) located on the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561).

[0169] In one embodiment, the wearable device (401) can identify the texture (e.g., color, wrinkles, scars, eyebrows, hair, beard of the user) of the user (501) in the image as the user appearance. In one embodiment, the wearable device (401) can identify the body shape (e.g., shape, size, length, area) of the user (501) in the image as the user appearance. In one embodiment, the wearable device (401) can identify the accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) of the user (501) in the image as the user appearance.

[0170] In operation 1030, the wearable device (401) may determine whether reflection of the user's appearance is necessary. In one embodiment, reflection of the user's appearance may mean reflecting the user's appearance on the avatar's appearance. In one embodiment, reflection of the user's appearance may mean updating the avatar animation of the avatar (505) based on the user's appearance.

[0171] In one embodiment, the wearable device (401) can identify whether at least one of the avatar appearance or the avatar animation of the avatar is updated based on comparing data related to the user appearance of the user (501) identified in the image with data related to the avatar appearance of the avatar (505).

[0172] In one embodiment, the wearable device (401) can identify landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) representing the body shape (e.g., shape, size, length, area) of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501) in the image. In one embodiment, the wearable device (401) can identify only some landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) that are not covered by an accessory (or exposed to the outside). For example, in one embodiment, the wearable device (401) can identify landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) except for landmarks (615, 616, 617) that are covered by a specific accessory (e.g., a mask).

[0173] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated by comparing the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) and the landmarks (631, 632, 633, 634, 635, 636, 637, 638, 639, 640) of the avatar (505). For example, the wearable device (401) can identify whether the avatar appearance is updated based on the locations of the landmarks of the user (501) and the landmarks of the avatar (505) that correspond to each other.

[0174] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on a first distance and / or a first slope between two landmarks. In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on a second distance and / or a second slope between two landmarks of the avatar (505). For example, the wearable device (401) can identify whether the avatar appearance is updated based on a first distance and / or a first slope between two landmarks of the user (501) that correspond to each other and a second distance and / or a second slope between two landmarks of the avatar (505).

[0175] In one embodiment, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) of the user (501) in the image. In one embodiment, the wearable device (401) can identify textures in designated body regions of the user (501) identified in the image. For example, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) identified in the image. In one embodiment, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around the remaining landmarks (611, 612, 613, 614, 618, 619, 620) when some of the landmarks (615, 616, 617) are covered by an accessory (or not exposed to the outside).

[0176] In one embodiment, the wearable device (401) can identify a texture at a location of the avatar (505) corresponding to the location where the texture of the user (501) is identified. For example, if the user's (501) beard is identified around a landmark (616), the wearable device (401) can identify whether the avatar's (505) beard is present around a landmark (636) of the avatar (505) corresponding to the landmark (616).

[0177] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the texture of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on comparing the texture of the user (501) and the texture of the avatar (505) at corresponding locations. For example, the wearable device (401) can identify whether the avatar appearance is updated by comparing the beard of the user (501) around the landmark (616) and the beard of the avatar (505) around the landmark (636) of the avatar (505). For example, if the beard of the avatar (505) corresponding to the beard of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the beard of the user (501) is different from the type (or appearance) of the beard of the avatar (505), the wearable device (401) can identify that the avatar appearance is updated. For example, if the beard of the avatar (505) corresponding to the beard of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the beard of the user (501) is not different from the type (or appearance) of the beard of the avatar (505), the wearable device (401) can identify that the avatar appearance is not updated. However, the present invention is not limited thereto.

[0178] In one embodiment, the wearable device (401) can identify whether to update the avatar appearance based on the similarity (e.g., structural similarity index measure (SSIM)) between the image of the user (501) and a previously acquired image used to generate the avatar (505). For example, the wearable device (401) can identify whether to update the avatar appearance based on the similarity between the image of the user's (501) face (511) and the previously acquired image of the user's (501) face (511) used to update the avatar's (505) face (515). For example, the wearable device (401) can identify not to update the avatar appearance based on the similarity between the image of the user's (501) face (511) and the previously acquired image of the user's (501) face (511) exceeding a reference similarity. For example, the wearable device (401) may determine that the avatar appearance is updated based on the similarity between the image of the face (511) of the user (501) and the previously acquired image of the face (511) of the user (501) being less than or equal to a reference similarity. However, the present invention is not limited thereto. For example, the wearable device (401) may obtain the similarity between the images of the face (511) of the user (501) based on another similarity measurement algorithm.

[0179] In one embodiment, the wearable device (401) can identify an accessory (e.g., a possession, jewelry, clothing, body paint, or makeup) of the user (501) in the image. In one embodiment, the wearable device (401) can identify an accessory in a designated body region of the user (501) identified in the image. For example, the wearable device (401) can identify an accessory (e.g., a possession, jewelry, clothing, body paint, or makeup) around landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) identified in the image. In one embodiment, the wearable device (401) can identify accessories (e.g., belongings, jewelry, clothing, body paint, or makeup) around the remaining landmarks (611, 612, 613, 614, 618, 619, 620) when some landmarks (615, 616, 617) are obscured (or not exposed to the outside) by the accessories.

[0180] In one embodiment, the wearable device (401) can identify an asset corresponding to an accessory at a location of the avatar (505) corresponding to the location where the accessory of the user (501) is identified. For example, if a cast of the user (501) is identified around the left arm (541), the wearable device (401) can identify whether an asset corresponding to the cast of the avatar (505) exists around the left arm (545) of the avatar (505) corresponding to the left arm (541).

[0181] In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on the accessories of the user (501). In one embodiment, the wearable device (401) can identify whether the avatar appearance is updated based on comparing assets corresponding to the accessories of the user (501) and the accessories of the avatar (505) at corresponding locations. For example, the wearable device (401) can identify whether the avatar appearance is updated by comparing assets corresponding to the cast of the user (501) around the left arm (541) and the cast of the avatar (505) around the left arm (545) of the avatar (505). For example, if the asset of the avatar (505) corresponding to the cast of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is updated. For example, if the type (or appearance) of the cast of the user (501) is different from the type (or appearance) of the cast of the avatar (505), the wearable device (401) can identify that the avatar appearance is being updated. For example, if the cast of the avatar (505) corresponding to the cast of the user (501) is not identified, the wearable device (401) can identify that the avatar appearance is being updated. For example, if the type (or appearance) of the cast of the user (501) is not different from the type (or appearance) of the cast of the avatar (505), the wearable device (401) can identify that the avatar appearance is not being updated.

[0182] In one embodiment, the wearable device (401) may identify that reflection of the user's appearance is necessary based on the identification that the avatar's appearance is being updated. In one embodiment, the wearable device (401) may identify that reflection of the user's appearance is not necessary based on the identification that the avatar's appearance is not being updated.

[0183] In operation 1030, based on identification that reflection of the user's appearance is required, the wearable device (401) may perform operation 1040. In operation 1030, based on identification that reflection of the user's appearance is not required, the wearable device (401) may perform operation 1050.

[0184] In operation 1040, the wearable device (401) may update the avatar based on the user's appearance. In one embodiment, the wearable device (401) may change the appearance properties of the avatar (505) so that the user's appearance is reflected in the avatar appearance based on the identification that the avatar appearance is being updated. Here, the appearance properties may include a mesh of the avatar (505), a blendshape of the avatar (505), a texture of the avatar (505), and / or an asset of the avatar (505). For example, an appearance property associated with a mesh of the avatar (505) may include information about the positions of vertices. For example, an appearance property associated with a blendshape of the avatar (505) may be a set of parameters representing the appearance and / or facial expression of the avatar (505). For example, the appearance properties associated with the blendshape of the avatar (505) may include parameters representing the position and / or movement of each of a plurality of regions (mouth region, chin region, nose region, right cheek region, left cheek region, right eye region, and / or left eye region) of the face of the avatar (505). For example, the appearance properties associated with the texture of the avatar (505) may include information representing the type of texture (e.g., wrinkle, scar, beard, hair), texture (or color), and / or appearance (e.g., beard style, hair style). For example, the appearance properties associated with the asset of the avatar (505) may include information representing the type of asset (e.g., cast, glasses, brooch, bracelet, watch, cell phone, bag), texture (or color), strength (or elasticity), and / or appearance.

[0185] In one embodiment, the wearable device (401) may update the avatar appearance such that a second distance and / or a second slope between two landmarks of a corresponding avatar (505) correspond to a first distance and / or a first slope between two landmarks of the user (501).

[0186] In one embodiment, the wearable device (401) can update an appearance property associated with a mesh of the avatar (505). In one embodiment, the wearable device (401) can update an appearance property associated with a mesh of the avatar (505) such that a second distance and / or a second slope between two corresponding landmarks of the avatar (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501). In one embodiment, the wearable device (401) can update information about the positions of vertices associated with two landmarks of the avatar (505) such that a second distance and / or a second slope between two corresponding landmarks of the avatar (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501).

[0187] In one embodiment, the wearable device (401) can update the appearance properties associated with the blendshape of the avatar (505) based on the locations of the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501). In one embodiment, the wearable device (401) can update the appearance properties associated with the blendshape so that multiple areas of the user (501) according to the blendshape are located at the locations of the landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501).

[0188] In one embodiment, the wearable device (401) can change the appearance properties associated with the texture of the avatar (505) based on the texture of the user (501) identified in the image. In one embodiment, if the texture of the user (501) identified in the image is different from the texture of the avatar (505), the wearable device (401) can update the texture of the avatar (505) to correspond to the texture of the user (501). In one embodiment, the wearable device (401) can identify a style of the type of avatar (505) that corresponds to the texture of the user (501) identified in the image. In one embodiment, the wearable device (401) can change the style of the identified avatar (505) to the new style of the avatar (505).

[0189] For example, if the hair style of the face (511) of the user (501) identified in the image is different from the hair style of the face (515) of the avatar (505), the wearable device (401) may update the hair style of the face (515) of the avatar (505) based on the hair style of the user (501).

[0190] In one embodiment, the wearable device (401) may change the appearance properties associated with the asset of the avatar (505) based on the accessory of the user (501) identified in the image. In one embodiment, if the accessory of the user (501) identified in the image is different from the asset of the avatar (505), the wearable device (401) may update the asset of the avatar (505) to correspond to the accessory of the user (501).

[0191] In one embodiment, the wearable device (401) can identify an asset of a type of avatar (505) corresponding to an accessory of the user (501) identified in the image. In one embodiment, the wearable device (401) can apply the asset of the identified avatar (505) to the avatar (505).

[0192] For example, if the style of clothes (e.g., a skirt) worn by the user (501) identified in the image is different from the style of clothes (e.g., pants) worn by the avatar (505), the wearable device (401) may update the style of clothes worn by the avatar (505) based on the style of clothes (e.g., a skirt) worn by the user (501). For example, if the style of a hat worn on the face (511) of the user (501) identified in the image is different from the style of a hat worn on the face (515) of the avatar (505), the wearable device (401) may update the style of a hat worn on the face (515) of the avatar (505) based on the style of the hat worn on the face (511). For example, if the right hand of the right arm (521) of the user (501) identified in the image is holding a smartphone, and the right hand of the right arm (525) of the avatar (505) is not holding any object (e.g., a smartphone), the wearable device (401) may update the avatar (505) so that the right hand of the right arm (525) of the avatar (505) is holding a smartphone. For example, if the left arm (541) of the user (501) identified in the image is in a cast, and the left arm (545) of the avatar (505) is not in a cast, the wearable device (401) may update the avatar (505) so that a cast is attached to the left arm (545) of the avatar (505).

[0193] In one embodiment, the wearable device (401) can update the avatar (505) based on an input for executing an application displaying the avatar (505). In one embodiment, the wearable device (401) can update at least one of the avatar appearance or the avatar animation in response to executing the application displaying the avatar (505) based on an identification that reflection of the user's appearance is necessary. In one embodiment, the wearable device (401) can change the appearance properties of the avatar (505) or change the movement properties of the animation based on an input for executing the application displaying the avatar (505).

[0194] In operation 1050, the wearable device (401) may display an avatar (505).

[0195] However, this is not limited thereto. For example, the wearable device (401) may transmit data for displaying (or rendering) the avatar (505) to an external electronic device (e.g., the electronic device (102)) so that the avatar (505) is displayed on the external electronic device (e.g., the electronic device (102)).

[0196] FIG. 11 is a flowchart showing the operation of a wearable device (401) according to one embodiment.

[0197] FIG. 11 may be described with reference to FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, or FIG. 9.

[0198] Referring to FIG. 11, in operation 1110, the wearable device (401) can identify a texture of a user. In one embodiment, the wearable device (401) can identify a user's appearance from an image. In one embodiment, the wearable device (401) can identify a texture (e.g., color, wrinkles, scars, eyebrows, hair, beard of the user's skin) of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561) from the image. For example, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) identified in the image. In one embodiment, the wearable device (401) can identify textures (e.g., color of the user's skin, wrinkles, scars, eyebrows, hair, beard) around the remaining landmarks (611, 612, 613, 614, 618, 619, 620) when some of the landmarks (615, 616, 617) are covered by an accessory (or not exposed to the outside).

[0199] In one embodiment, the wearable device (401) can identify a texture at a location of the avatar (505) corresponding to the location where the texture of the user (501) is identified. For example, if the user's (501) beard is identified around a landmark (616), the wearable device (401) can identify whether the avatar's (505) beard is present around a landmark (636) of the avatar (505) corresponding to the landmark (616).

[0200] In operation 1120, the wearable device (401) can update the texture of the avatar based on the identified texture. In one embodiment, the wearable device (401) can change the appearance properties associated with the texture of the avatar (505) based on the texture of the user (501) identified in the image. In one embodiment, if the texture of the user (501) identified in the image is different from the texture of the avatar (505), the wearable device (401) can update the texture of the avatar (505) to correspond to the texture of the user (501). In one embodiment, the wearable device (401) can identify a style of the avatar (505) that corresponds to the texture of the user (501) identified in the image. In one embodiment, the wearable device (401) can change the style of the identified avatar (505) to the new style of the avatar (505).

[0201] According to an embodiment, the wearable device (401) may identify whether the avatar appearance is updated. In response to identifying that the avatar appearance (or the texture of the avatar (505)) is updated, the wearable device (401) may update the texture of the avatar based on the identified texture. For example, identifying whether the avatar appearance is updated may be based on the similarity between the image (705) of the user (501) and a previously acquired image (701) of the user (501) used to generate the avatar (505). For example, identifying whether the avatar appearance is updated may be based on comparing the texture of the user (501) and the texture of the avatar (505) at corresponding locations. For example, identifying whether the avatar appearance is updated may be based on comparing the time at which the avatar (505) was generated and the current time.

[0202] FIG. 12 is a flowchart showing the operation of a wearable device (401) according to one embodiment.

[0203] FIG. 12 may be described with reference to FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, or FIG. 9.

[0204] Referring to FIG. 12, in operation 1210, the wearable device (401) can identify landmarks of the user (501). In one embodiment, the wearable device (401) can acquire landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the user (501) from an image acquired for the user (501). In one embodiment, the wearable device (401) can acquire landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) of the face (511), arms (521, 541), legs (531, 551), and / or body (561) of the user (501). For example, the landmarks of the face (511) of the user (501) can include, but are not limited to, the crown of the head (611), the under-eye area (612, 620), and the jawline (613, 614, 615, 616, 617 618, 619). The landmarks of the face (511) of the user (501) can include, but are not limited to, lips, eyes, eyebrows, ears, or nose. For example, landmarks of the arms (521, 541) of the user (501) may include a hand, a finger, a wrist, or a shoulder. For example, landmarks of the legs (531, 551) of the user (501) may include a foot, an ankle, a knee, a thigh, or a calf. For example, landmarks of the body (561) of the user (501) may include, but are not limited to, a neck, a chest, an armpit, or a buttocks.

[0205] For example, the wearable device (401) can identify first locations of a plurality of designated first landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) in the face (511) of the user (501) included in the image, and can identify second locations of a plurality of designated second landmarks in the face (515) of the avatar (505).

[0206] In operation 1220, the wearable device (401) may update the appearance of the avatar based on the identified landmark.

[0207] In one embodiment, the wearable device (401) can update the avatar appearance such that a second distance and / or a second slope between two landmarks of corresponding avatars (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501). In one embodiment, the wearable device (401) can update the avatar appearance such that a second distance and / or a second slope between two landmarks of corresponding avatars (505) corresponds to a first distance and / or a first slope between two landmarks of the user (501) based on identifying the avatar (505) as being updated. For example, the wearable device (401) may update landmarks (615, 617) of the avatar (505) and / or areas adjacent to landmarks (615, 617) such that a second distance and / or a second slope between landmarks (635, 637) of the avatar (505) corresponds to a first distance and / or a first slope between landmarks (615, 617) of the user (501).

[0208] According to an embodiment, the wearable device (401) may identify whether the avatar appearance is updated. In response to identifying that the avatar appearance is updated, the wearable device (401) may update the avatar appearance based on the identified landmark. For example, identifying whether the avatar appearance is updated may be based on a first distance and / or a first slope between two corresponding landmarks of the user (501) and a second distance and / or a second slope between two landmarks of the avatar (505). For example, identifying whether the avatar appearance is updated may be based on comparing a difference between the first distance between at least two landmarks and the second distance between at least two landmarks of the avatar (505) with a reference distance difference. For example, the wearable device (401) may identify that the avatar (505) is updated with respect to a landmark and / or an area adjacent to the landmark for the second distance based on a difference between the first distance and the second distance exceeding the reference distance difference. For example, the wearable device (401) may determine not to update a landmark and / or an area adjacent to a landmark for the second distance in the avatar (505) based on a difference between the first distance and the second distance being less than or equal to a reference distance difference. For example, the determination of whether to update the avatar appearance may be based on comparing a difference between a first slope between at least two landmarks of the user (501) and a second slope between at least two landmarks of the avatar (505) with a reference slope difference. For example, the wearable device (401) may determine to update a landmark and / or an area adjacent to a landmark for the second slope in the avatar (505) based on a difference between the first slope and the second slope being greater than the reference slope difference.For example, the wearable device (401) may identify the avatar (505) as not updating the landmark and / or the area adjacent to the landmark for the second slope based on the difference between the first slope and the second slope being less than or equal to a reference slope difference.

[0209] FIG. 13 is a flowchart illustrating the operation of a wearable device (401) according to one embodiment.

[0210] FIG. 13 may be described with reference to FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, or FIG. 9.

[0211] Referring to FIG. 13, in operation 1310, the wearable device (401) can identify a user's belongings. In one embodiment, the wearable device (401) can identify belongings located on the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561). In one embodiment, the wearable device (401) can identify belongings in images of the user's (501) face (511), arms (521, 541), legs (531, 551), and / or body (561) through the camera (425). In one embodiment, the wearable device (401) can identify belongings of the user (501) by the user's appearance in images of parts of the user's (501) body.

[0212] In operation 1320, the wearable device (401) may determine whether an animation update is required. In one embodiment, the wearable device (401) may determine whether an animation update is required based on attributes of an asset (802) attached to an avatar (505) corresponding to the possession and / or possession. Here, the attributes may indicate the location, strength, and / or type of the possession and / or asset (802).

[0213] In one embodiment, the wearable device (401) may identify that the animation is updated when the possession is included in the type that restricts the movement of the user (501). In one embodiment, the wearable device (401) may identify that the animation is updated when the possession restricts the movement of a part of the user's (501) body. In one embodiment, the wearable device (401) may identify that the animation is updated based on an attribute associated with the type of possession indicating that the possession restricts the movement of a part of the user's (501) body. In one embodiment, the wearable device (401) may identify that the animation is updated when the possession restricts the movement of a part of the user's (501) body. Here, the animation related to a part of the body of the avatar (505) may be an animation that includes the movement of a part of the body of the avatar (505) when the animation is played.

[0214] At operation 1320, based on the identification that the animation needs to be updated, the wearable device (401) may perform operation 1330. At operation 1320, based on the identification that the animation does not need to be updated, the wearable device (401) may terminate the operation according to FIG. 13.

[0215] At action 1330, the wearable device (401) may update the animation based on the belongings.

[0216] In one embodiment, the wearable device (401) can update the animation by changing the movement properties of the animation identified as being updated. For example, the wearable device (401) can update the animation by changing the movement properties of a body part of the avatar (505) corresponding to the body part of the user (501) where the possession is located. For example, the wearable device (401) can update the animation by changing the movement properties of a body part of the avatar (505) where the asset is located. Here, the movement properties may include at least one of a movement direction, a movement speed, a movement range, or a movement order. For example, a movement direction may refer to directions in which a skeletal mesh can move through a joint (e.g., a joint) included in the body of the avatar (505). For example, a movement speed may refer to a movement speed in the movement directions of a skeletal mesh through a joint (e.g., a joint) included in the body of the avatar (505). For example, the range of movement may refer to the angular range in the movement directions of the skeletal mesh through a joint (e.g., a joint) included in the body of the avatar (505). For example, the movement order may refer to the movement orders between two or more skeletal meshes through two or more joints included in the body of the avatar (505).

[0217] In one embodiment, the wearable device (401) can update the animation by changing the movement properties of the animation associated with the joint of the avatar (505). In one embodiment, the wearable device (401) can update the animation by changing at least one of the movement direction, movement speed, movement range, or movement order of the animation associated with the joint of the avatar (505). For example, the wearable device (401) can update the animation by changing the movement properties of a joint and / or skeletal mesh included in a body part of the avatar (505) corresponding to a body part of the user (501) where an item is located.

[0218] As described above, the wearable device (101, 200, 300, 401) may include at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425). In one embodiment, the wearable device (101, 200, 300, 401) may include a processor (120, 410). In one embodiment, the wearable device (101, 200, 300, 401) may include a memory (130, 415) that stores instructions. The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to capture an image including a part of the body of a user (501) wearing the wearable device (101, 200, 300, 401) through the at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425). The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify whether an update of at least one of the avatar appearance or the avatar animation of the avatar (505) is required based on comparing first data related to a user appearance of the user (501) identified in the image with second data related to an avatar appearance of an avatar (505) of the user (501).The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update the appearance properties of the avatar (505) so that the avatar appearance corresponds to the user appearance based on the identification that the avatar appearance requires updating. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update the movement properties of the animation associated with a joint of the avatar (505) whose movement is restricted by the properties of the user appearance based on the identification that the animation requires updating.

[0219] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify an item of the user (501) as part of the user's appearance in the image. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify that a type of the attribute of the item restricts the user's (501) movement, thereby requiring an update of the animation. The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify that no update of the animation is required in response to the type not restricting the user movement.

[0220] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify the joint of the avatar (505) corresponding to a location where the belonging is attached to the user (501) based on the identification that an update of the animation is required. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update a movement range of the joint among the movement properties of the animation to a range according to the type of the belonging.

[0221] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify a first joint of the avatar (505) corresponding to a location where the belonging is attached to the user (501) based on the identification that an update of the animation is required. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify a second joint of the avatar (505) corresponding to a location opposite the location where the belonging is attached to the user (501). The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update the movement properties of the animation so that the animation associated with the first joint is executed via the second joint.

[0222] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify first locations of a plurality of designated first landmarks in the face (511) of the user (501) included in the image. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify second locations of a plurality of designated second landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) in the face (515) of the avatar (505). The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify second locations of a plurality of designated second landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) in the face (515) of the avatar (505). When executed, the wearable device (101, 200, 300, 401) may be caused to identify that an update of the avatar appearance is required based on the identification that a distance difference between a first distance between the first locations and a second distance between the second locations is greater than or equal to a reference distance.

[0223] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update meshes representing the face (515) of the avatar (505) such that the plurality of designated second landmarks are displayed at locations corresponding to the first locations, based on the wearable device identifying that an update of the avatar appearance is required.

[0224] The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update a blendshape for representing the face (515) of the avatar (505) such that the plurality of designated second landmarks are displayed at locations corresponding to the first locations, based on the wearable device identifying that an update of the avatar appearance is required.

[0225] The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify that an update of the avatar appearance is required based on the identification that a similarity between a first facial image representing the user appearance of the user (501) identified in the image and a previously captured second facial image used to generate the avatar (505) is less than a reference similarity.

[0226] The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to acquire an image including a plurality of images including the part of the body of the user (501) wearing the wearable device (101, 200, 300, 401) through the at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425). The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify whether an update of the avatar animation of the avatar (505) is required based on comparing the user movement of the user (501) identified in the image with the avatar animation of the avatar (505).

[0227] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to obtain the image before executing an application displaying the avatar (505). The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify whether an update of at least one of the avatar appearance or the avatar animation of the avatar (505) is required before executing an application displaying the avatar (505).

[0228] The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to obtain input for executing an application that displays the avatar (505). The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update at least one of the avatar appearance or the avatar animation in response to executing the application.

[0229] The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to obtain body shape information of the user (501) from an external electronic device via a communication circuit. The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update the appearance properties of the avatar (505) based on the body shape information.

[0230] The method as described above can be performed in a wearable device (101, 200, 300, 401) comprising at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425). In one embodiment, the method may include an action of capturing an image including a part of a body of a user (501) wearing the wearable device (101, 200, 300, 401) through at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425). In one embodiment, the method may include an action of identifying whether an update is required of at least one of the avatar appearance or an avatar animation of the avatar (505) based on comparing first data related to a user appearance of the user (501) identified in the image with second data related to an avatar appearance of an avatar (505) of the user (501). In one embodiment, the method may include an action of updating an appearance property of the avatar (505) so that the avatar appearance corresponds to the user appearance, based on identifying that an update of the avatar appearance is required. In one embodiment, the method may include an action of updating a movement property of an animation associated with a joint of the avatar (505) whose movement is restricted by a property of the user appearance, based on identifying that an update of the animation is required.

[0231] In one embodiment, the method may include an action of identifying an item of the user (501) as part of the user's appearance in the image. In one embodiment, the method may include an action of identifying that an update of the animation is required in response to a type of the item limiting the user's (501) movement. In one embodiment, the method may include an action of identifying that an update of the animation is not required in response to a type not limiting the user's movement.

[0232] In one embodiment, the method may include an action of identifying the joint of the avatar (505) corresponding to a location where the possession is attached to the user (501) based on an identification that an update of the animation is required. In one embodiment, the method may include an action of setting a range of movement of the joint among the movement properties of the animation to a range according to a type of the possession.

[0233] In one embodiment, the method may include identifying a first joint of the avatar (505) corresponding to a location where the possession is attached to the user (501), based on which an update of the animation is identified as being required. In one embodiment, the method may include identifying a second joint of the avatar (505) corresponding to a location opposite to the location where the possession is attached to the user (501). In one embodiment, the method may include updating a movement property of the animation such that the animation associated with the first joint is executed via the second joint.

[0234] In one embodiment, the method may include an action of identifying first locations of a plurality of designated first landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) in a face (511) of the user (501) included in the image. In one embodiment, the method may include an action of identifying second locations of a plurality of designated second landmarks in a face (515) of the avatar (505). In one embodiment, the method may include an action of identifying that an update of the avatar appearance is required based on a distance difference between the first locations and the second locations being identified as being greater than or equal to a reference distance.

[0235] In one embodiment, the method may include updating meshes representing the face (515) of the avatar (505) such that the plurality of designated second landmarks are displayed at locations corresponding to the first locations, based on an identification that an update of the avatar appearance is required.

[0236] In one embodiment, the method may include updating a blendshape representing the face (515) of the avatar (505) such that the plurality of designated second landmarks are displayed at locations corresponding to the first locations, based on an identification that an update of the avatar appearance is required.

[0237] In one embodiment, the method may include an action of identifying that an update of the avatar appearance is required based on identifying that a similarity between a first facial image representing the user appearance of the user (501) identified in the image and a previously captured second facial image used to generate the avatar appearance is less than a threshold similarity.

[0238] As described above, a non-transitory computer readable storage medium can store a program including instructions. In a wearable device (101, 200, 300, 401), the instructions, when executed by a processor (120, 410) of a wearable device (101, 200, 300, 401) including at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), cause the wearable device (101, 200, 300, 401) to capture at least one camera (180, 240-1, 240-2, 240-3, Through the wearable device (101, 200, 300, 401), an image including a part of the body of a user (501) wearing the wearable device (101, 200, 300, 401) can be captured. The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to identify whether an update of at least one of the avatar appearance or an avatar animation of the avatar (505) is required based on comparing first data related to a user appearance of the user (501) identified in the image with second data related to an avatar appearance of an avatar (505) of the user (501). The instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update an appearance attribute of the avatar (505) so that the avatar appearance corresponds to the user appearance based on identifying that an update of the avatar appearance is required.The above instructions, when executed by the processor (120, 410), may cause the wearable device (101, 200, 300, 401) to update a movement property of the animation associated with a joint of the avatar (505) whose movement is restricted by a property of the user's appearance, based on the wearable device identifying that an update of the animation is required.

[0239] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0240] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0241] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0242] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0243] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0244] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable device (101, 200, 300, 401), At least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), Processor (120, 410), and A memory (130, 415) for storing instructions, wherein when the instructions are executed by the processor (120, 410), the wearable device (101, 200, 300, 401): Taking an image including a part of the body of a user (501) wearing the wearable device (101, 200, 300, 401) through at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), Based on comparing first data related to the user appearance of the user (501) identified in the image and second data related to the avatar appearance of the avatar (505) of the user (501), it is identified whether at least one of the avatar appearance or the avatar animation of the avatar (505) requires updating, Based on the identification that the update of the above avatar appearance is required, the appearance property of the avatar (505) is updated so that the avatar appearance corresponds to the user appearance, Based on the identification that an update of the above animation is required, causing the movement properties of the animation associated with the joints of the above avatar (505) whose movement is restricted by the properties of the above user's appearance to be updated. Wearable devices.

2. In claim 1, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Identifying the belongings of the user (501) as part of the user's appearance in the image above, In response to the above attributes of the above belongings limiting the user's (501) movements, it is identified that an update of the animation is required, In response to the above type not restricting the user movement, causing it to be identified that no update of the animation is required, Wearable devices.

3. In claim 2, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Based on the identification that the above animation requires updating, the joint of the avatar (505) corresponding to the location where the possession is attached to the user (501) is identified, Causing the movement range of the joint among the movement properties of the above animation to be updated to the range according to the type of the possession, Wearable devices.

4. In claim 2, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Based on the identification that the above animation requires updating, the first joint of the avatar (505) corresponding to the location where the possession is attached to the user (501) is identified, Identifying a second joint of the avatar (505) corresponding to a position opposite to the position where the said belonging is attached to the said user (501), Causing the movement properties of the animation to be updated so that the animation associated with the first joint is executed through the second joint. Wearable devices.

5. In any one of claims 1 to 4, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Identifying first locations of a plurality of designated first landmarks in the face (511) of the user (501) included in the image, Identifying second locations of a plurality of designated second landmarks (611, 612, 613, 614, 615, 616, 617, 618, 619, 620) on the face (515) of the above avatar (505), Based on the identification that the difference between the first distance between the first locations and the second distance between the second locations is greater than or equal to the reference distance, causing the identification that the update of the avatar appearance is required, Wearable devices.

6. In claim 5, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Based on the identification that the update of the above avatar appearance is required, causing the meshes for representing the face (515) of the above avatar (505) to be updated so that the plurality of designated second landmarks are displayed at locations corresponding to the first locations. Wearable devices.

7. In claim 5, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Based on the identification that the update of the above avatar appearance is required, causing the blendshape for representing the face (515) of the above avatar (505) to be updated so that the plurality of designated second landmarks are displayed at positions corresponding to the first positions. Wearable devices.

8. In any one of claims 1 to 7, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Based on the identification that the similarity between the first face image representing the user appearance of the user (501) identified in the image and the previously captured second face image used to generate the avatar (505) is less than the reference similarity, causing it to be identified that the update of the avatar appearance is required. Wearable devices.

9. In any one of claims 1 to 8, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Obtaining an image including a plurality of images including the part of the body of the user (501) wearing the wearable device (101, 200, 300, 401) through at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), Based on comparing the user movement of the user (501) identified in the above image with the avatar animation of the avatar (505), causing identification of whether an update of the avatar animation of the avatar (505) is required. Wearable devices.

10. In any one of claims 1 to 9, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Before executing the application displaying the above avatar (505), the image is acquired, Before executing an application displaying the above avatar (505), causing it to identify whether at least one of the avatar appearance or the avatar animation of the avatar (505) requires an update. Wearable devices.

11. In claim 10, The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Obtaining input for executing an application displaying the above avatar (505), In response to executing said application, causing at least one of said avatar appearance or said avatar animation to be updated, Wearable devices.

12. In any one of claims 1 to 11, Further comprising a communication circuit (190, 435), The above instructions, when executed by the processor (120, 410), cause the wearable device (101, 200, 300, 401) to: Through the above communication circuit (190, 435), the body shape information of the user (501) is acquired from an external electronic device, Based on the above body shape information, causing the appearance properties of the avatar (505) to be updated. Wearable devices.

13. A method performed in a wearable device (101, 200, 300, 401) comprising at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), An action of capturing an image including a part of the body of a user (501) wearing the wearable device (101, 200, 300, 401) through at least one camera (180, 240-1, 240-2, 240-3, 340-1, 340-2, 340-3, 340-4, 340-5, 340-6, 340-7, 340-8, 340-9, 340-10, 425), An operation of identifying whether at least one of the avatar appearance or the avatar animation of the avatar (505) requires updating based on comparing first data related to the user appearance of the user (501) identified in the image and second data related to the avatar appearance of the avatar (505) of the user (501). An operation of updating the appearance properties of the avatar (505) so that the avatar appearance corresponds to the user appearance, based on identification that an update of the avatar appearance is required; and Based on the identification that the above animation requires updating, an action is included to update the movement properties of the animation associated with the joints of the avatar (505) whose movement is restricted by the properties of the user's appearance. method.

14. In claim 13, An action of identifying the belongings of the user (501) as part of the user's appearance in the image above, An action to identify that an update of the animation is required in response to a type of the above property of the above belongings restricting the user's (501) movement, and In response to the above type not restricting the user movement, the action includes identifying that no update of the animation is required. method.

15. In claim 14, Based on the identification that the above animation requires updating, an operation of identifying the joint of the avatar (505) corresponding to the location where the possession is attached to the user (501), and An action that includes setting the range of movement of the joint among the movement properties of the above animation to a range according to the type of the possession. method.

Images