Electronic device, method, and non-transitory computer-readable storage medium for generating avatar of user

A head-worn electronic device captures and communicates obscured face images to generate accurate avatars, addressing the challenge of partial obstruction in augmented and virtual reality, thereby improving user experience through comprehensive avatar creation.

WO2026235411A1PCT designated stage Publication Date: 2026-11-12SAMSUNG ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/095820
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-07-15
Filing Date
2026-04-10
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing electronic devices struggle to generate accurate user avatars in augmented and virtual reality environments when the user's face is partially obscured by head-mounted devices, limiting the effectiveness of augmented and virtual reality services.

Method used

The solution involves a head-worn electronic device equipped with a camera and communication circuitry that captures images of the user's face while obscured, transmits these images to an external device, requests additional images, and guides the acquisition of further images to generate a comprehensive avatar based on the received information.

Benefits of technology

This approach enables the generation of accurate user avatars despite partial obstruction, enhancing the user experience in augmented and virtual reality applications by providing a seamless integration of real-world and virtual objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026095820_12112026_PF_FP_ABST
    Figure KR2026095820_12112026_PF_FP_ABST
Patent Text Reader

Abstract

According to one embodiment, an electronic device comprises: a communication circuit; a memory including instructions and comprising at least one storage medium; and at least one processor comprising processing circuitry. The instructions, when executed individually or collectively by the at least one processor, instruct the electronic device to: receive, from a head-mounted electronic device connected to the electronic device, through the communication circuit, information on at least one image representing a user's face obtained while at least a portion of the user's face is covered by the head-mounted electronic device; obtain information on the user's face on the basis of the information on the at least one image and information on the appearance of the head-mounted electronic device; and provide a service related to an avatar of the user generated on the basis of the information on the user's face.
Need to check novelty before this filing date? Find Prior Art

Description

Electronic device, method, and non-transient computer-readable storage medium for generating a user's avatar

[0001] The following descriptions relate to an electronic device, a method, and a non-transitory computer-readable storage medium for generating a user avatar.

[0002] Electronic devices are being developed to provide augmented reality (AR) and / or virtual reality (VR) services that display computer-generated information in conjunction with external objects within the real world in order to provide an enhanced user experience. The electronic device may be a wearable device that can be worn by a user. For example, the electronic device may be a smart glasses device and / or a head-mounted electronic device.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device is provided. The electronic device may include at least one processor comprising a communication circuit and a processing circuit, and a memory comprising one or more storage media for storing instructions. The instructions may cause the electronic device to receive, via the communication circuit, information regarding at least one image representing the face of the user, which is acquired from a head-mounted electronic device connected to the electronic device while at least a portion of the user's face is obscured by the head-mounted electronic device, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to acquire information regarding the face of the user based on the information regarding the at least one image and information regarding the appearance of the head-mounted electronic device when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to provide a service regarding the user's avatar generated based on the information regarding the user's face, when executed individually or collectively by the at least one processor.

[0005] A head-worn electronic device is provided. The head-worn electronic device may include at least one processor comprising a camera positioned toward the outside of the head-worn electronic device, a communication circuit, and a processing circuit, and a memory comprising one or more storage media for storing instructions. When the instructions are executed individually or collectively by the at least one processor, at least one image representing the user's face may be acquired through the camera while at least a portion of the user's face is obscured by the head-worn electronic device. When the instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may cause information regarding the at least one image to be transmitted to an external electronic device through the communication circuit. When the instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may cause a signal to be received through the communication circuit for requesting another image representing a portion of the face from the external electronic device. The above instructions, when executed individually or collectively by the at least one processor, may cause the head-worn electronic device to provide a notification to guide the acquisition of information regarding another image representing the part of the face based on the signal. The above instructions, when executed individually or collectively by the at least one processor, may cause the head-worn electronic device to transmit the information regarding the other image acquired through the camera to the external electronic device via the communication circuit.

[0006] A method performed by an electronic device is provided. The method may include, through a communication circuit, receiving information regarding at least one image representing the face of the user, which is acquired from a head-mounted electronic device connected to the electronic device while at least a portion of the user's face is obscured by the head-mounted electronic device. The method may include, based on the information regarding the at least one image and information regarding the appearance of the head-mounted electronic device, acquiring information regarding the face of the user. The method may include, based on the information regarding the face of the user, providing a service regarding an avatar of the user generated based on the information regarding the face of the user.

[0007] A method is provided to be performed by a head-worn electronic device. The method may include the action of acquiring at least one image representing the user’s face through a camera while at least a portion of the user’s face is obscured by the head-worn electronic device. The method may include the action of transmitting information regarding the at least one image to an external electronic device through a communication circuit. The method may include the action of receiving, through the communication circuit, a signal from the external electronic device to request another image representing a portion of the face. The method may include the action of providing a notification to guide the acquisition of information regarding another image representing a portion of the face based on the signal. The method may include the action of transmitting information regarding the other image acquired through the camera to the external electronic device through the communication circuit.

[0008] A non-transient computer-readable storage medium is provided for storing one or more programs. The one or more programs may include instructions that cause the electronic device to receive, via the communication circuit, information regarding at least one image representing the user’s face, which is acquired from a head-worn electronic device connected to the electronic device while at least a portion of the user’s face is obscured by the head-worn electronic device, when executed by at least one processor of an electronic device having a communication circuit. The one or more programs may include instructions that cause the electronic device to acquire information regarding the user’s face based on the information regarding the at least one image and information regarding the appearance of the head-worn electronic device when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to provide a service regarding the user’s avatar generated based on the information regarding the user’s face when executed by the at least one processor.

[0009] A non-transient computer-readable storage medium is provided for storing one or more programs. The one or more programs may include instructions that cause the head-wearing electronic device to acquire, through the camera, at least one image representing the user's face while at least a portion of the user's face is obscured by the head-wearing electronic device, when executed by at least one processor of the head-wearing electronic device having a camera and a communication circuit. The one or more programs may include instructions that cause the head-wearing electronic device to transmit information about the at least one image to an external electronic device through the communication circuit when executed by the at least one processor. The one or more programs may include instructions that cause the head-wearing electronic device to receive, through the communication circuit, a signal for requesting another image representing a portion of the face from the external electronic device when executed by the at least one processor. The above one or more programs may include instructions that cause the head-worn electronic device to provide a notification to guide the acquisition of information regarding another image representing the part of the face based on the signal when executed by the at least one processor. The above one or more programs may include instructions that cause the head-worn electronic device to transmit the information regarding the other image acquired through the camera to the external electronic device via the communication circuit when executed by the at least one processor.

[0010] Figure 1 is a block diagram of an electronic device in a network environment.

[0011] FIGS. 2a and FIGS. 2b are perspective views of a wearable device.

[0012] FIGS. 3A and FIGS. 3B illustrate an example of the appearance of a wearable device.

[0013] Figure 4 is a block diagram of a wearable device.

[0014] Figure 5a is a schematic diagram of a head-worn electronic device and an electronic device.

[0015] FIG. 5b is a block diagram including a plurality of modules used for avatar creation.

[0016] FIG. 6 illustrates a method for obtaining information about a face based on an image of a user wearing a head-worn electronic device.

[0017] FIG. 7 is a flowchart illustrating the operation of transmitting information about an image from a head-worn electronic device to an electronic device.

[0018] Figure 8 illustrates information about the face.

[0019] FIG. 9a is a flowchart illustrating the operation of creating an avatar based on acquiring face photos from various angles.

[0020] FIG. 9b illustrates a method for identifying a head-worn electronic device using an optical signal.

[0021] FIG. 10 is a flowchart illustrating the operation of acquiring information about a face based on removing a head-worn electronic device from an image.

[0022] FIG. 11 illustrates a method for removing an accessory.

[0023] FIG. 12 illustrates a method for obtaining information about the shape of a face based on the shape of a head-worn electronic device.

[0024] FIG. 13 illustrates a method for correcting a part distorted by a lens.

[0025] FIG. 14 illustrates a method for providing guidance for acquiring images from different angles.

[0026] FIG. 15 is a flowchart illustrating the operation of acquiring information about a face based on a plurality of facial feature points.

[0027] FIG. 16 illustrates a method for obtaining information about a face based on an error value.

[0028] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0029] Figure 1 is a block diagram of an electronic device in a network environment.

[0030] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0031] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0032] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0033] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0034] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0035] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0036] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0037] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0038] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0039] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0040] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0041] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0043] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0045] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0047] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0048] An antenna module (197) can transmit a signal or power to an external source (e.g., an external electronic device) or receive it from an external source. According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0049] According to one embodiment, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0050] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0051] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least a part of the requested function or service, or additional functions or services related to the request, and transmit the result of the execution to the electronic device (101).

[0052] The electronic device (101) may process the above results as they are or additionally and provide them as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199). The electronic device (101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0053] FIGS. 2a and FIGS. 2b are perspective views of a wearable device.

[0054] According to one embodiment, the wearable device (200) may have the form of glasses that are wearable on a part of a user's body (e.g., head). The wearable device (200) of FIGS. 2a and 2b may be an example of the electronic device (101) of FIG. 1. The wearable device (200) may include a head-mounted display (HMD). The wearable device (200) may correspond to a head-worn electronic device. For example, the housing of the wearable device (200) may include a flexible material such as rubber and / or silicone that has a shape that adheres to a part of the user's head (e.g., a part of the face covering both eyes). For example, the housing of the wearable device (200) may include one or more straps that can be twined around the user's head, and / or one or more temples that can be attached to the ears of the head.

[0055] Referring to FIG. 2a, a wearable device (200) according to one embodiment may include at least one display (250) and a frame (295) supporting at least one display (250).

[0056] According to one embodiment, a wearable device (200) may be worn on a part of a user's body. The wearable device (200) may provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality to a user wearing the wearable device (200). For example, the wearable device (200) may display a virtual reality image provided by at least one optical device (282, 284) of FIG. 2b on at least one display (250) in response to a specified gesture of the user obtained through the motion recognition camera (260-2, 263) of FIG. 2b.

[0057] According to one embodiment, at least one display (250) can provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at locations corresponding to the user's left eye and right eye, respectively.

[0058] Referring to FIG. 2b, at least one display (250) may provide visual information transmitted from external light to a user through a lens included in at least one display (250) and other visual information distinct from said visual information. The lens may be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, at least one display (250) may include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area may be formed on the second surface (232) of at least one display (250). When a user wears the wearable device (200), external light may be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided by at least one optical device (282, 284) on a real image transmitted through external light in a display area formed on the second surface (232).

[0059] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits it to a user. At least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano pattern may be formed on the exterior or at least a portion of the interior of at least one waveguide (233, 234). The nano pattern may be formed based on a polygonal and / or curved grating structure. Light incident on one end of at least one waveguide (233, 234) may be propagated to the other end of at least one waveguide (233, 234) by the nano pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., DOE (diffractive optical element), HOE (holographic optical element)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be placed within a wearable device (200) to guide a screen displayed by at least one display (250) to the user's eye. For example, the screen may be transmitted to the user's eye based on total internal reflection (TIR) ​​occurring within at least one waveguide (233, 234).

[0060] A wearable device (200) can analyze an object included in a real-world image collected through a shooting camera (260-4), combine a virtual object corresponding to an object among the analyzed objects that is the target of augmented reality provision, and display it on at least one display (250). The virtual object may include at least one of text and an image regarding various information related to the object included in the real-world image. The wearable device (200) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (200) can perform spatial recognition (e.g., SLAM (simultaneous localization and mapping)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (200) can view the image displayed on at least one display (250).

[0061] According to one embodiment, the frame (295) may be formed as a physical structure that allows the wearable device (200) to be worn on the user's body. According to one embodiment, the frame (295) may be configured so that when the user wears the wearable device (200), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (295) may support at least one display (250). For example, the frame (295) may support the first display (250-1) and the second display (250-2) so that they are positioned corresponding to the user's left and right eyes.

[0062] Referring to FIG. 2a, the frame (295) may include an area (220) in which at least a portion of the frame contacts a part of the user's body when the user wears the wearable device (200). For example, the area (220) of the frame (295) in contact with a part of the user's body may include an area in contact with a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (200) contacts. According to one embodiment, the frame (295) may include a nose pad (210) that contacts a part of the user's body. When the wearable device (200) is worn by the user, the nose pad (210) may contact a part of the user's nose. The frame (295) may include a first temple (204) and a second temple (205) that contact a different part of the user's body distinct from the part of the user's body.

[0063] For example, the frame (295) may include a first rim (201) covering at least a portion of the first display (250-1), a second rim (202) covering at least a portion of the second display (250-2), a bridge (203) positioned between the first rim (201) and the second rim (202), a first pad (211) positioned along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) positioned along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) extending from the first rim (201) and fixed to a portion of the wearer's ear, and a second temple (205) extending from the second rim (202) and fixed to a portion of the ear opposite to the ear. The first pad (211) and the second pad (212) may come into contact with a part of the user's nose, and the first temple (204) and the second temple (205) may come into contact with a part of the user's face and a part of the ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through a first hinge unit (206) positioned between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through a second hinge unit (207) positioned between the second rim (202) and the second temple (205). According to one embodiment, a wearable device (200) can identify an external object (e.g., a user's fingertip) touching the frame (295) and / or a gesture performed by said external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of the surface of the frame (295).

[0064] According to one embodiment, the wearable device (200) may include hardware that performs various functions (e.g., hardware to be described later based on the block diagram of FIG. 4). For example, the hardware may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers (255-1, 255-2)), a microphone (e.g., microphones (265-1, 265-2, 265-3)), a light-emitting module (not shown), and / or a PCB (printed circuit board) (290) (e.g., a printed circuit board). The various hardware may be placed within a frame (295).

[0065] According to one embodiment, a microphone (e.g., microphones (265-1, 265-2, 265-3)) of a wearable device (200) is positioned on at least a portion of a frame (295) to acquire a sound signal. A first microphone (265-1) positioned on a bridge (203), a second microphone (265-2) positioned on a second rim (202), and a third microphone (265-3) positioned on a first rim (201) are shown in FIG. 2b, but the number and position of the microphones (265) are not limited to the embodiment of FIG. 2b. If there are two or more microphones (265) included in the wearable device (200), the wearable device (200) can identify the direction of the sound signal by using a plurality of microphones positioned on different portions of the frame (295).

[0066] According to one embodiment, at least one optical device (282, 284) may project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) may be a projector. At least one optical device (282, 284) may be disposed adjacent to at least one display (250) or included within at least one display (250) as part of at least one display (250). According to one embodiment, a wearable device (200) may include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) positioned at the edge of a first display (250-1) and a second optical device (284) positioned at the edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) positioned on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) positioned on the second display (250-2).

[0067] In one embodiment, the camera (260) may include a shooting camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 206-3). The shooting camera (260-4), the eye tracking camera (260-1), and the motion recognition camera (260-2, 260-3) may be positioned at different locations on the frame (295) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position of the eyes or the gaze of a user wearing the wearable device (200). For example, the wearable device (200) may detect the gaze from an image containing the user's pupils obtained through the eye tracking camera (260-1). A wearable device (200) can identify an object focused by a user (e.g., a real object, and / or a virtual object) by using the user's gaze acquired through an eye-tracking camera (260-1). Upon identifying the focused object, the wearable device (200) can perform a function (e.g., gaze interaction) for interaction between the user and the focused object. The wearable device (200) can represent a portion corresponding to the eyes of an avatar representing the user in a virtual space by using the user's gaze acquired through the eye-tracking camera (260-1). The wearable device (200) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality of a first region associated with the gaze within the image and the visual quality of a second region distinguished from the first region (e.g., resolution, brightness, saturation, grayscale, PPI (pixels per inch)) may differ from each other.The wearable device (200) can acquire an image having a visual quality of a first region and a visual quality of a second region that matches the user's gaze by using foveated rendering. For example, if the wearable device (200) supports an iris recognition function, user authentication can be performed based on iris information acquired using an eye-tracking camera (260-1). An example in which the eye-tracking camera (260-1) is positioned toward the user's right eye is shown in FIG. 2b, but the embodiment is not limited thereto, and the eye-tracking camera (260-1) may be positioned alone toward the user's left eye or toward both eyes.

[0068] In one embodiment, the camera (260-4) can capture a real image or background to be matched with a virtual image in order to implement augmented reality or mixed reality content. The camera (260-4) can be used to acquire high-resolution images based on HR (high resolution) or PV (photo video). The camera (260-4) can capture an image of a specific object located at the position viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information regarding a real image or background including the image of the specific object acquired using the camera (260-4) and a virtual image provided through at least one optical device (282, 284) are superimposed. The wearable device (200) can compensate for depth information (e.g., the distance between the wearable device (200) and an external object acquired through a depth sensor) using the image acquired through the camera (260-4). The wearable device (200) can perform object recognition through an image acquired using a shooting camera (260-4). The wearable device (200) can perform a function of focusing on an object (or subject) in an image (e.g., AF (auto focus)) and / or an OIS (optical image stabilization) function (e.g., anti-shake function) using the shooting camera (260-4). The wearable device (200) can perform a pass-through function to superimpose an image acquired through the shooting camera (260-4) onto at least a portion of a screen representing a virtual space while displaying the screen representing a virtual space on at least one display (250). In one embodiment, the shooting camera (260-4) may be placed on a bridge (203) positioned between a first rim (201) and a second rim (202).

[0069] The eye tracking camera (260-1) can achieve more realistic augmented reality by tracking the gaze of a user wearing the wearable device (200), thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the user looks straight ahead, the wearable device (200) can naturally display environmental information related to the user's front on at least one display (250) at the location where the user is situated. The eye tracking camera (260-1) may be configured to capture an image of the user's pupil to determine the user's gaze. For example, the eye tracking camera (260-1) may receive a gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the eye tracking camera (260-1) may be positioned at locations corresponding to the user's left and right eyes. For example, the eye-tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (200) is located.

[0070] A motion recognition camera (260-2, 260-3) can provide a specific event to a screen provided on at least one display (250) by recognizing the movement of the user's entire body or part thereof, such as the user's torso, hands, or face. A motion recognition camera (260-2, 260-3) can recognize the user's gesture, acquire a signal corresponding to the gesture, and provide a display corresponding to the signal to at least one display (250). A processor can identify the signal corresponding to the gesture and, based on the identification, perform a designated function. A motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM (simultaneous localization and mapping) and / or a depth map for a 6-degrees-of-freedom pose (6 dof pose). A processor can use the motion recognition camera (260-2, 260-3) to perform a gesture recognition function and / or an object tracking function. In one embodiment, a motion recognition camera (260-2, 260-3) may be placed on the first rim (201) and / or the second rim (202).

[0071] The camera (260) included in the wearable device (200) is not limited to the eye-tracking camera (260-1) and motion recognition camera (260-2, 260-3) described above. For example, the wearable device (200) can identify external objects included within the field of view (FoV) by using a camera positioned toward the user's field of view (FoV). The identification of external objects by the wearable device (200) can be performed based on a sensor for identifying the distance between the wearable device (200) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus (AF) function and / or an optical image stabilization (OIS) function. For example, the wearable device (200) may include a camera (260) (e.g., a face tracking camera) positioned toward the face to acquire an image including the face of a user wearing the wearable device (200).

[0072] Although not illustrated, according to one embodiment, the wearable device (200) may further include a light source (e.g., LED) that emits light toward a subject (e.g., user's eye, face, and / or an object outside the FoV) being photographed using a camera (260). The light source may include an LED of infrared wavelength. The light source may be placed in at least one of a frame (295) and hinge units (206, 207).

[0073] According to one embodiment, the battery module (270) can supply power to the electronic components of the wearable device (200). In one embodiment, the battery module (270) may be placed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may each be placed in the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be placed at the end of the first temple (204) and / or the second temple (205).

[0074] The antenna module (275) can transmit a signal or power to the outside of the wearable device (200) or receive a signal or power from the outside. In one embodiment, the antenna module (275) may be placed within the first temple (204) and / or the second temple (205). For example, the antenna module (275) may be placed close to one side of the first temple (204) and / or the second temple (205).

[0075] The speaker (255) can output an acoustic signal to the outside of the wearable device (200). The acoustic output module may be referred to as the speaker. In one embodiment, the speaker (255) may be placed within a first temple (204) and / or a second temple (205) to be positioned adjacent to the ear of a user wearing the wearable device (200). For example, the speaker (255) may include a second speaker (255-2) positioned adjacent to the user's left ear by being placed within the first temple (204), and a first speaker (255-1) positioned adjacent to the user's right ear by being placed within the second temple (205).

[0076] A light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state in order to visually provide information regarding a specific state of the wearable device (200) to the user. For example, when the wearable device (200) requires charging, it may emit red light at a constant frequency. In one embodiment, the light-emitting module may be placed on the first rim (201) and / or the second rim (202).

[0077] Referring to FIG. 2b, a wearable device (200) according to one embodiment may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of a first temple (204) or a second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. On the PCB (290), one or more hardware components included in the wearable device (200) (e.g., hardware components illustrated by different blocks in FIG. 4) may be disposed. The wearable device (200) may include a flexible PCB (FPCB) for interconnecting the hardware components.

[0078] According to one embodiment, a wearable device (200) may include at least one of a gyroscope sensor, a gravity sensor, and / or an acceleration sensor for detecting the posture of the wearable device (200) and / or the posture of a body part (e.g., head) of a user wearing the wearable device (200). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on designated three-dimensional axes (e.g., x-axis, y-axis, and z-axis) that are perpendicular to each other. The gyroscope sensor may measure the angular velocity of each of the designated three-dimensional axes (e.g., x-axis, y-axis, and z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyroscope sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (200) can identify a user's motion and / or gesture performed to execute or stop a specific function of the wearable device (200) based on an IMU.

[0079] FIGS. 3A and FIGS. 3B illustrate an example of the appearance of a wearable device.

[0080] The wearable device (200) of FIG. 3a and FIG. 3b may be an example of the electronic device (101) of FIG. 1 and the wearable device (200) of FIG. 2a and FIG. 2b. According to one embodiment, an example of the appearance of a first surface (310) of the housing of the wearable device (200) is shown in FIG. 3a, and an example of the appearance of a second surface (320) opposite to the first surface (310) may be shown in FIG. 3b.

[0081] Referring to FIG. 3a, according to one embodiment, a first surface (310) of a wearable device (200) may have a shape that is attachable to a part of a user's body (e.g., the user's face). Although not illustrated, the wearable device (200) may further include a strap for fixing to a part of a user's body and / or one or more temples (e.g., a first temple (204) and / or a second temple (205) of FIG. 2a and FIG. 2b). A first display (250-1) for outputting an image to the left eye among the user's two eyes and a second display (250-2) for outputting an image to the right eye among the two eyes may be disposed on the first surface (310). The wearable device (200) may further include rubber or silicone packing formed on the first surface (310) to prevent interference by light different from light emitted from the first display (250-1) and the second display (250-2) (e.g., ambient light).

[0082] According to one embodiment, the wearable device (200) may include cameras (260-1) for photographing and / or tracking both eyes of a user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referenced to the eye-tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (200) may include cameras (260-5, 260-6) for photographing and / or recognizing the user's face. The cameras (260-5, 260-6) may be referenced to FT cameras. The wearable device (200) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6). For example, the wearable device (200) can change the texture and / or shape of a part of an avatar (e.g., a part of an avatar representing a human face) by using information obtained by cameras (260-5, 260-6) (e.g., FT cameras) and representing the facial expression of a user wearing the wearable device (200).

[0083] Referring to FIG. 3b, on a second surface (320) opposite to the first surface (310) of FIG. 3a, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)), and / or a sensor (e.g., a depth sensor (330)) may be disposed to obtain information related to the external environment of the wearable device (200). For example, cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. The cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2b.

[0084] For example, using cameras (260-11, 260-12), the wearable device (200) can acquire images and / or videos to be transmitted to each of the user's two eyes. Camera (260-11) may be placed on the second surface (320) of the wearable device (200) to acquire an image to be displayed through a second display (250-2) corresponding to the right eye among the two eyes. Camera (260-12) may be placed on the second surface (320) of the wearable device (200) to acquire an image to be displayed through a first display (250-1) corresponding to the left eye among the two eyes. Cameras (260-11, 260-12) may be referenced to the shooting camera (260-4) of FIG. 2B.

[0085] According to one embodiment, the wearable device (200) may include a depth sensor (330) disposed on a second surface (320) to identify the distance between the wearable device (200) and an external object. Using the depth sensor (330), the wearable device (200) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (200). Although not illustrated, a microphone may be disposed on the second surface (320) of the wearable device (200) to obtain sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0086] Figure 4 is a block diagram of a wearable device.

[0087] Referring to FIG. 4, a wearable device (200) according to one embodiment may include at least one of a processor (410), memory (415), display (420), camera (425), sensor (430), or communication circuit (435). The processor (410), memory (415), display (420), camera (425), sensor (430), and communication circuit (435) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (402). The type and / or number of hardware components included in the wearable device (200) are not limited to those shown in FIG. 4. For example, the wearable device (200) may include only some of the hardware components shown in FIG. 4. The elements within the memory described below (e.g., layers and / or modules) may be in a logically separated state. However, it is not limited to this.

[0088] A processor (410) of a wearable device (200) according to one embodiment may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). In one embodiment, the wearable device (101) may include one or more processors. According to one embodiment, the structure of the processor (410) is not limited to one embodiment of the present disclosure, and at least one circuit may be formed as a separate processor physically separated from the processor (410) (e.g., an embedded secure element (eSE), a secure processor). The number of processors (410) may be one or more. For example, the processor (410) may have a structure of a multi-core processor such as a dual core, a quad core, or a hexa core. The multi-core processor structure of the processor (410) may include a structure based on multiple core circuits (e.g., a big-little structure) distinguished by power consumption, clock, and / or computational amount per unit time. In one embodiment comprising a processor (410) having a multi-core processor structure, the operations and / or functions of the present disclosure may be performed individually or collectively by one or more cores included in the processor (410).

[0089] A memory (415) of a wearable device (200) according to one embodiment may include a hardware component for storing data and / or instructions that are input and / or output to a processor (410). The memory (415) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, and embedded multi-media card (eMMC). In one embodiment, the memory (415) may be referred to as storage.

[0090] In one embodiment, the display (420) of the wearable device (200) can output visualized information to the user of the wearable device (200). For example, the display (420) may include a wave guide and an optical device.

[0091] For example, the display (420) may be controlled by a processor (410) including circuitry such as a GPU (graphic processing unit) to output visualized information to a user. The display (420), which is positioned in front of the eyes of a user wearing the wearable device (200), may be placed in at least a part of the housing of the wearable device (200). For example, the display (420) may be controlled by a processor (410) including circuitry such as a CPU, a GPU (graphic processing unit), and / or a DPU (display processing unit) to output visualized information to a user. The display (420) may include a flexible display, a flat panel display (FPD), and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LED may include an organic LED (OLED). The embodiments are not limited thereto, and for example, if the wearable device (200) includes a lens for transmitting external light (external light or ambient light), the display (420) may include a projector (or projection assembly) for projecting light onto said lens. In one embodiment, the display (420) may be referred to as a display panel, a display assembly, and / or a display module.

[0092] In one embodiment, the camera (425) of the wearable device (200) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate an electrical signal indicating the color and / or brightness of light. The plurality of light sensors included in the camera (425) may be arranged in the form of a two-dimensional grid (2 dimensional array). The camera (425) may acquire the electrical signals of each of the plurality of light sensors substantially simultaneously to generate two-dimensional frame data corresponding to the light reaching the light sensors of the two-dimensional grid. For example, photo data captured using the camera (425) may refer to one (a) two-dimensional frame data acquired from the camera (425). For example, video data captured using the camera (425) may refer to a sequence of multiple two-dimensional frame data acquired from the camera (425) along a frame rate. The camera (425) may further include a flash light for outputting light in the direction in which the camera (425) receives light.

[0093] According to one embodiment, the wearable device (200) may include a plurality of cameras positioned toward different directions as an example of a camera (425). For example, the camera (425) may include a first camera, a second camera, and at least one third camera. For example, the first camera may be configured to acquire at least one image of a first environment identified by the user's first eye (e.g., left eye). The second camera may be configured to acquire at least one image of a second environment identified by the user's second eye (e.g., right eye). For example, at least one third camera may be configured to identify gaze according to the user's first eye and second eye. At least one of the third cameras may be configured to identify the movement of the first eye. At least one of the third cameras may be configured to identify the movement of the second eye. For example, at least one third camera may be configured to identify gaze according to the movement of the first eye and the movement of the second eye. The wearable device (200) can identify the direction of the user's gaze by using at least one third camera. For example, the direction in which the first camera and the second camera are facing may be opposite to the direction in which at least one third camera is facing.

[0094] According to one embodiment, a sensor (430) of a wearable device (200) can generate electrical information that can be processed by a processor (410) and / or memory (415) of the wearable device (200) from non-electronic information related to the wearable device (200). The information may be referred to as sensor data. The sensor (430) may include a global positioning system (GPS) sensor for detecting the geographic location of the wearable device (200), an image sensor, and / or a time-of-flight (ToF) sensor, and an inertial measurement unit (IMU) for detecting physical motion of the wearable device (200). In addition to the above GPS method, the sensor (430) can generate information indicating the geographical location of the wearable device (200) based on a GNSS (global navigation satellite system) such as Galileo or Beidou (compass). This information may be stored in memory (415), processed by a processor (410), and / or transmitted to another electronic device distinct from the wearable device (200) via a communication circuit.

[0095] Referring to FIG. 4, an image sensor (432) and / or a motion sensor (433) are illustrated as examples of sensors (430) included in a wearable device (200). The sensor (430) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate electrical signals indicating the color and / or brightness of light. The image sensor (432) may be referred to as a camera. A plurality of light sensors included in the image sensor (432) may be arranged in the form of a two-dimensional grid (2 dimensional array). The image sensor (432) may acquire the electrical signals of each of the plurality of light sensors substantially simultaneously to generate two-dimensional frame data corresponding to the light reaching the light sensors of the two-dimensional grid. For example, photo data captured using the image sensor (432) may refer to one (a) two-dimensional frame data acquired from the image sensor (432). For example, video data captured using an image sensor (432) may refer to a sequence of multiple two-dimensional frame data obtained from the image sensor (432) along a frame rate. The image sensor (432) may further include a flash light for outputting light in the direction in which the image sensor (432) receives light.

[0096] According to one embodiment, the wearable device (200) may include a plurality of image sensors arranged toward different directions, as an example of an image sensor (432). As described above with reference to FIGS. 2a, 2b, 3a, and 3b, the plurality of image sensors may include an eye-tracking camera (e.g., the eye-tracking camera (260-1) of FIGS. 2b and 3a) configured to be arranged toward the eyes of a user wearing the wearable device (200). The plurality of image sensors may include an outward camera. A processor (410) may identify the direction of the user's gaze using an image and / or video obtained from the eye-tracking camera. The eye-tracking camera may include an infrared (IR) sensor. The eye-tracking camera may be referred to as an eye sensor and / or an eye tracker.

[0097] An external camera may be positioned facing the front of the user wearing the wearable device (200) (e.g., a direction in which both eyes may face). The wearable device (200) may include a plurality of external cameras. The embodiments are not limited thereto, and the external camera may be positioned facing the external space. Using images and / or videos obtained from the external camera, the processor (410) may identify external objects. For example, the processor (410) may identify the position, shape, and / or gesture (e.g., hand gesture) of the user wearing the wearable device (200) based on images and / or videos obtained from the external camera. Using images and / or videos of the external environment obtained from the external camera, the processor (410) may recognize or track one or more objects within the external environment.

[0098] According to one embodiment, the motion sensor (433) may output an electrical signal representing gravitational accelerations and / or angular velocities of a plurality of axes (e.g., x-axis, y-axis, and z-axis) that are perpendicular to each other and are based on a designated origin within the wearable device (200) and / or the motion sensor (433). For example, the processor (410) may repeatedly receive or acquire sensor data from the motion sensor (433), including accelerations, angular velocities, and / or magnitudes of the magnetic field of a number of the plurality of axes, based on a designated period (e.g., 1 millisecond). In one embodiment, the motion sensor (433) may be referred to as an inertial measurement unit (IMU). The sensor (430) included in the wearable device (200) is not limited to the above and may include a grip sensor, a proximity sensor, a heart rate sensor, a fingerprint sensor, an ambient light sensor, and / or a ToF sensor. By using a motion sensor (433), the processor (410) can detect motion of the wearable device (200) (e.g., motion of the wearable device (200) caused by a user wearing the wearable device (200)).

[0099] In one embodiment, the communication circuit (435) of the wearable device (200) may include hardware components to support the transmission and / or reception of electrical signals between the wearable device (200) and an external electronic device (e.g., electronic device (102), electronic device (104) of FIG. 1). The communication circuit (435) may include, for example, at least one of a modem, an antenna, and an optic / electronic converter. The communication circuit (435) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio) and / or 6G.

[0100] In one embodiment, within the memory (415) of the wearable device (200), one or more instructions (or commands) representing operations and / or operations to be performed on data by the processor (410) of the wearable device (200) may be stored. A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable device (200) and / or the processor (410) may perform at least one of the operations according to the embodiments described below when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed. In the following, the statement that an application is installed in the wearable device (200) means that one or more instructions provided in the form of an application are stored in memory (415), and that the one or more applications are stored in an executable format (e.g., a file having an extension specified by the operating system of the wearable device (200)) by the processor (410). For example, the application may include a program and / or library related to a service provided to the user.

[0101] Referring to FIG. 4, programs installed on a wearable device (200) may be classified into any one of different layers based on the target, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480) (e.g., an Android system HAL, and / or an XR HAL). For example, within the hardware abstraction layer (480), programs (e.g., modules, or drivers) designed to target the hardware of the wearable device (200) (e.g., a display (420), a camera (420), and / or a sensor (430)) may be classified. The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing XR (extended reality) services. For example, FIG. 4 illustrates layers separated within memory (415), but the layers are logically separated and may not imply that the address space of memory (415) is separated by said layers. However, it is not limited thereto. According to an embodiment, the layers may be stored in a designated area within memory (415).

[0102] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or application layer (440) (e.g., location tracker (471), spatial recognizer (472), gesture tracker (473), eye tracker (474), face tracker (475), and / or renderer (490)) may be classified. Programs classified into the framework layer (450) may provide an application programming interface (API) that is executable based on other programs.

[0103] For example, within the application layer (440), programs designed to target a user controlling a wearable device (200) may be classified. Examples of programs classified into the application layer (440) include an XR (extended reality) system UI (user interface) and / or an XR application (442), but embodiments are not limited thereto. For example, programs classified into the application layer (440) (e.g., software applications) may call an API (application programming interface) to cause the execution of functions supported by programs classified into the framework layer (450).

[0104] For example, the wearable device (200) may display one or more visual objects on the display (420) to perform interaction with a user for using a virtual space based on the execution of the XR system UI (441). A visual object may mean an object that can be deployed within the screen for the transmission of information and / or interaction, such as text, images, icons, videos, buttons, checkboxes, radio buttons, text boxes, sliders, and / or tables. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. The wearable device (200) may provide the user with a service to control functions available within the virtual space based on the execution of the XR system UI (441).

[0105] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plugin (444) are depicted within the XR system UI (441), but are not limited thereto. For example, the XR system UI (441) may cause the execution of supported functions in the lightweight renderer (443) and / or the XR plugin (444) included within the framework layer (450).

[0106] For example, the wearable device (200) may acquire resources (e.g., APIs, system processes and / or libraries) used to define, create, and / or execute a rendering pipeline, which is permitted to be partially modified, based on the execution of a lightweight renderer (442). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline, which is permitted to be partially modified. The lightweight renderer (443) may include a renderer (e.g., a pre-built renderer) built prior to the execution of a software application. For example, the wearable device (200) may acquire resources (e.g., APIs, system processes and / or libraries) used to define, create, and / or execute the entire rendering pipeline based on the execution of an XR plugin (444). The XR plugin (444) may be referred to as an open XR native client in terms of defining (or setting) the entire rendering pipeline.

[0107] For example, the wearable device (200) may display a screen representing at least a portion of a virtual space on the display (420) based on the execution of the XR application (442). The XR plugin (444-1) included in the XR application (442) may be referenced by the XR plugin (444) of the XR system UI (441). Descriptions of the XR plugin (444-1) that overlap with descriptions of the XR plugin (444) may be omitted. The wearable device (200) may trigger the execution of the virtual space manager (451) based on the execution of the XR application (442).

[0108] According to one embodiment, the wearable device (200) may provide a virtual space service based on the execution of a virtual space manager (451). For example, the virtual space manager (451) may include a platform (e.g., an Android platform) for supporting the virtual space service. Based on the execution of the virtual space manager (451), the wearable device (200) may display on the display (420) the posture of a virtual object representing the user's posture rendered using data acquired through the sensor (430). The virtual space manager (451) may be referred to as a composition presentation manager (CPM).

[0109] For example, the virtual space manager (451) may include a runtime service (452). In one example, the runtime service (452) may be referred to as an OpenXR runtime module. A wearable device (200) may be used to provide at least one of a user’s pose prediction function, frame timing function, and / or spatial input function through the wearable device (200) based on the execution of the runtime service (452). In one example, the wearable device (200) may be used to perform rendering for the virtual space service for the user based on the execution of the runtime service (452). For example, an application (e.g., Unity or OpenXR native application) may be implemented based on the execution of the runtime service (452).

[0110] For example, the virtual space manager (451) may include a pass-through manager (453). The wearable device (200) may, based on the execution of the pass-through manager (453), display another screen representing the real space obtained through the camera (425) on at least a portion of the screen while displaying a screen representing the virtual space on the display (420).

[0111] For example, the virtual space manager (451) may include an input manager (454). The wearable device (200) may identify acquired data (e.g., sensor data) by executing one or more programs included within the recognition service layer (470) based on the execution of the input manager (454). The wearable device (200) may initiate the execution of at least one of the functions of the wearable device (200) using the acquired data. For example, the wearable device (200) may identify user input related to the wearable device (200) using the acquired data. The user input may be related to user motion (e.g., hand gesture), gaze, and / or speech identified by a sensor (430) (e.g., an image sensor (432) such as an external camera). The user input may be identified based on an external electronic device connected (or paired) via a communication circuit.

[0112] For example, the perception abstract layer (460) can be used for data exchange between the virtual space manager (451) and the perception service layer (470). In terms of being used for data exchange between the virtual space manager (451) and the perception service layer (470), the perception abstract layer (460) can be referred to as an interface. As an example, the perception abstract layer (460) can be referred to as OpenPX. The perception abstract layer (460) can be used for a perception client and a perception service.

[0113] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data obtained from a sensor (430) (or a camera (425)). The one or more programs may include at least one of a position tracker (471), a spatial recognizer (472), a gesture tracker (473), an eye tracker (474), a face tracker (473), and / or a renderer (490). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those shown in FIG. 4.

[0114] For example, the wearable device (200) can identify the posture of the wearable device (200) using the sensor (430) based on the execution of the position tracker (471). The wearable device (200) can identify the 6 degrees of freedom pose (6 dof pose) of the wearable device (200) using data acquired using the camera (425) and the IMU based on the execution of the position tracker (471). The position tracker (471) may be referred to as a head tracking (HeT) module (or head tracker, head tracking program).

[0115] For example, the wearable device (200) may be used to construct the surrounding environment of the wearable device (200) (or the user of the wearable device (200)) into a three-dimensional virtual space based on the execution of the space recognizer (472). The wearable device (200) may reconstruct the surrounding environment of the wearable device (200) in three dimensions using data acquired through the camera (425) based on the execution of the space recognizer (472). The wearable device (200) may identify at least one of a plane, an incline, and a staircase based on the surrounding environment of the wearable device (200) reconstructed in three dimensions based on the execution of the space recognizer (472). The space recognizer (472) may be referred to as a scene understanding (SU) module.

[0116] For example, the wearable device (200) may be used to identify (or recognize) the pose and / or gesture of the user's hand of the wearable device (200) based on the execution of the gesture tracker (473). For example, the wearable device (200) may identify the pose and / or gesture of the user's hand using data acquired from the sensor (430) based on the execution of the gesture tracker (473). For example, the wearable device (200) may identify the pose and / or gesture of the user's hand based on data (or images) acquired using the camera (425) based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0117] For example, the wearable device (200) can identify (or track) the movement of the user's eyes of the wearable device (200) based on the execution of the eye tracker (474). For example, the wearable device (200) can identify the movement of the user's eyes using data obtained from at least one sensor based on the execution of the eye tracker (474). For example, the wearable device (200) can identify the movement of the user's eyes based on data obtained using a camera (e.g., at least one third camera (425-3)) and / or an IR LED (infrared light emitting diode) based on the execution of the eye tracker (474). The eye tracker (474) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0118] For example, the recognition service layer (470) of the wearable device (200) may further include a face tracker (475) for tracking the user's face. For example, the wearable device (200) can identify (or track) the movement of the user's face and / or the user's facial expression based on the execution of the face tracker (475). The wearable device (200) can estimate the user's facial expression based on the movement of the user's face identified using a camera (e.g., a face tracking (FT) camera, a camera facing at least a part of the user's face, an image sensor (432)) based on the execution of the face tracker (475). For example, the wearable device (200) can identify the movement of the user's face and / or the user's facial expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker (475). The face tracker (475) may be referred to as face tracking (FT) (or a face tracking program) and / or a face tracking module.

[0119] For example, the renderer (490) may include instructions for rendering images in a three-dimensional virtual space. The processor (410) that executes the renderer (490) may obtain at least one image to be displayed at least partially in a display area of ​​the display (250) in a software application. For example, the processor (410) that executes the renderer (490) may determine the location of the area where the application (e.g., XR application (442)) is to be rendered. The processor (410) that executes the renderer (490) may generate an image of said application to be displayed on the display (250). The renderer (490) may synthesize images to generate a composite image to be displayed on the display (250).

[0120] FIG. 5a is a schematic diagram of an electronic device and a head-worn electronic device.

[0121] Referring to FIG. 5a, the electronic device (510) may include at least one processor (511), a communication circuit (512), and a memory (513). The electronic device (510) may correspond to the electronic device (101) described with reference to FIG. 1. Some of the components of the electronic device (510) shown in FIG. 5a (e.g., at least one processor (511), a communication circuit (512), and a memory (513)) may be omitted from the electronic device (510). The electronic device (510) may be able to communicate with a head-worn electronic device (520) through the communication circuit (512).

[0122] For example, at least one processor (511) may be operatively coupled with or connected with the communication circuit (512) and / or memory (513). That at least one processor (511) is operatively coupled with or connected with the communication circuit (512) and / or memory (513) may mean that at least one processor (511) can control the communication circuit (512) and / or memory (513). For example, the communication circuit (512) and / or memory (513) may be controlled by at least one processor (511).

[0123] For example, at least one processor (511) may include at least a part of the processor (120) of FIG. 1 or correspond to at least a part of the processor (120). For example, at least one processor (511) may include one or more processors including an application processor (AP) and / or a communication processor (CP). For example, at least one processor (511) may be implemented as a single chip, such as a system on chip (SoC), or as multiple chips. For example, at least one processor (511) may be implemented as a single integrated circuit. For example, at least one processor (511) may include a first integrated circuit and a second integrated circuit. For example, at least one processor (511) may be distributed within an electronic device (510). For example, a part of at least one processor (511) may be physically separated from another part of at least one processor (511). As a non-limiting example, the other part of at least one processor (511) is spaced apart from the part of at least one processor (511), but the other part of at least one processor (511) may be connected to the part of at least one processor (511) through an interface circuit.

[0124] At least one processor (511) can execute instructions stored in memory (513) to cause an electronic device (510) to perform operations to be described with reference to FIGS. 6 through 16.

[0125] The communication circuit (512) may include at least a part of the communication module (190) of FIG. 1 or correspond to at least a part of the communication module (190). For example, the communication circuit (512) of the electronic device (510) may include hardware components to support the transmission and / or reception of electrical signals between the electronic device (510) and an electronic device distinct from the electronic device (510) (e.g., a head-worn electronic device (520)). The communication circuit (512) may include, for example, at least one of a modem, an antenna, and an optic / electronic converter. The communication circuit (512) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio) and / or 6G. At least one processor (511) of the electronic device (510) can execute instructions in memory (513) within the electronic device (510) to perform functions and / or operations indicated by said instructions. For example, if the electronic device (510) includes at least one processor, said at least one processor (511) may be configured to execute said instructions collectively or individually.

[0126] The memory (513) may include at least a portion of the memory (130) of FIG. 1 or correspond to at least a portion of the memory (130). For example, the memory (513) may be implemented as a single chip or as a plurality of chips. For example, the memory (513) may be implemented as a single integrated circuit or as a plurality of integrated circuits (e.g., including a first integrated circuit and a second integrated circuit). For example, the memory (513) may be distributedly arranged within an electronic device (510).

[0127] The memory (513) may store one or more programs configured to be executed individually or collectively by at least one processor (511). The one or more programs may include at least some of instructions that cause an electronic device (510) to perform operations described with reference to FIGS. 6 through 16 when executed individually or collectively by at least one processor (511).

[0128] Referring to FIG. 5a, the head-worn electronic device (520) may include at least one processor (521), a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526), ​​and a display (527). The head-worn electronic device (520) may correspond to the wearable device (200) described with reference to FIG. 2a through FIG. 4. Some of the components of the head-worn electronic device (520) shown in FIG. 5a (e.g., at least one processor (521), a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526), ​​and a display (527)) may be omitted from the head-worn electronic device (520). The head-worn electronic device (520) may be able to communicate with the electronic device (510) through the communication circuit (524).

[0129] For example, at least one processor (521) may be operatively coupled with or connected with a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526) and / or a display (527). That at least one processor (521) is operatively coupled with or connected with a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526) and / or a display (527) may mean that at least one processor (521) can control a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526) and / or a display (527). For example, a light-emitting element (522), a camera (523), a communication circuit (524), a speaker (525), a memory (526) and / or a display (527) can be controlled by at least one processor (521).

[0130] For example, at least one processor (521) may include at least a part of the processor (410) of FIG. 4 or correspond to at least a part of the processor (410). For example, at least one processor (521) may include one or more processors including an application processor (AP) and / or a communication processor (CP). For example, at least one processor (521) may be implemented as a single chip, such as a system on chip (SoC), or as multiple chips. For example, at least one processor (521) may be implemented as a single integrated circuit. For example, at least one processor (521) may include a first integrated circuit and a second integrated circuit. For example, at least one processor (521) may be distributed within a head-worn electronic device (520). For example, a part of at least one processor (521) may be physically separated from another part of at least one processor (521). As a non-limiting example, the other part of at least one processor (521) is spaced apart from the part of at least one processor (521), but the other part of at least one processor (521) may be connected to the part of at least one processor (521) through an interface circuit.

[0131] At least one processor (521) can execute instructions stored in memory (526) to cause a head-worn electronic device (520) to perform operations described with reference to FIGS. 6 through 16.

[0132] For example, the light-emitting element (522) may include at least a part of the light-emitting module (not shown) described with reference to FIG. 2a, or correspond to at least a part of the light-emitting module. The light-emitting element (522) may include an LED (light emitting diode), an OLED (organic light emitting diode), an LD (laser diode), and / or an EL (electroluminescent device). For example, the light-emitting element (522) may be described as a device that converts electricity into light.

[0133] For example, the camera (523) may correspond to the camera (425) of FIG. 4. For example, the camera (523) may be positioned on the rear of the head-worn electronic device (520) to recognize an external object different from the head-worn electronic device (520). For example, using the camera (523), the head-worn electronic device (520) may acquire images and / or videos.

[0134] For example, the communication circuit (524) may include at least a part of the communication circuit (435) of FIG. 4 or correspond to at least a part of the communication circuit (435). For example, the communication circuit (524) of the head-worn electronic device (520) may include hardware components to support the transmission and / or reception of electrical signals between the head-worn electronic device (520) and an electronic device (e.g., electronic device (510)). The communication circuit (524) may include, for example, at least one of a modem, an antenna, and an optic / electronic converter. The communication circuit (524) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio) and / or 6G. At least one processor (521) of the head-worn electronic device (520) can execute instructions in memory (526) within the head-worn electronic device (520) to perform functions and / or operations indicated by said instructions. For example, if the head-worn electronic device (520) includes at least one processor, said at least one processor (521) may be configured to execute said instructions collectively or individually.

[0135] For example, the speaker (525) may include at least a portion of the speakers of FIG. 2a (e.g., speaker (255-1) and speaker (255-2)) or correspond to at least a portion of the speakers (e.g., speaker (255-1) and speaker (255-2)). For example, the speaker may convert an electrical signal into an acoustic signal (or sound wave). For example, the output of the acoustic signal of the speaker (525) may be controlled by at least one processor (521).

[0136] For example, memory (526) may include circuits and / or a storage medium for storing data and / or instructions that are input and / or output to at least one processor (521). Memory (526) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Non-volatile memory may be referred to as storage. Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, solid state drive (SSD), and embedded multi-media-card (eMMC).

[0137] The memory (526) may include at least a portion of the memory (415) of FIG. 4 or correspond to at least a portion of the memory (415). For example, the memory (526) may be implemented as a single chip or as a plurality of chips. For example, the memory (526) may be implemented as a single integrated circuit or as a plurality of integrated circuits (e.g., including a first integrated circuit and a second integrated circuit). For example, the memory (526) may be distributedly arranged within a head-worn electronic device (520).

[0138] The memory (526) may store one or more programs configured to be executed individually or collectively by at least one processor (521). The one or more programs may include at least some of instructions that cause a head-worn electronic device (520) to perform operations described with reference to FIGS. 6 through 16 when executed individually or collectively by at least one processor (521).

[0139] For example, the display (527) may include at least a portion of the display (420) of FIG. 4 or correspond to at least a portion of the display (420). The display (527) of the head-worn electronic device (520) may output visualized information (e.g., a screen) to the user. For example, the display (527) may output visualized information under the control of a controller such as a GPU (graphic processing unit) (e.g., included within at least one processor (521)). The display (527) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or a light emitting diode (LED) display. The LED display may include an organic LED display (OLED). The display (527) may be at least partially bendable or deformable. The deformable display (527) may include a flexible display.

[0140] FIG. 5b is a block diagram including a plurality of modules used for avatar creation.

[0141] Referring to FIG. 5b, at least one of the electronic device (510) and / or head-worn electronic device (520) for performing the creation of an avatar may include at least one of a command control unit (535), a modulation module (536), a head movement tracking module (537), a device identification module (538), a guide creation module (539), a face recognition module (540), a face feature point identification module (541), a size measurement module (542), a 3D Reconstruction module (543), a device shape removal module (544), an accessory identification module (545), a texture creation module (546), a white balancing module (547), and / or an avatar creation module (548).

[0142] For example, the command control unit (535) may provide a signal to the modulation module (536) to request the output of an optical signal based on receiving an input (531) to activate avatar generation. For example, the command control unit (535) may activate the device identification module (538) based on receiving an input (531) to activate avatar generation. For example, the input (531) may correspond to a voice input received through an input device or a microphone. For example, after the operation of the command control unit (535) is performed, the operation of the modulation module (536) and / or the device identification module (538) may be performed.

[0143] For example, the modulation module (536) can output an optical signal (534) corresponding to the modulated signal based on generating a temporally modulated signal. For example, after the operation of the modulation module (536) is performed, the operation of the device identification module (538) can be performed. A method for outputting an optical signal corresponding to the modulated signal will be described later with reference to FIG. 9b. For example, the device identification module (538) can identify an optical signal in which the output optical signal (534) is reflected by a reflector (e.g., the reflector (620) of FIG. 6) within an image acquired through the camera (523). For example, the device identification module (538) can identify the shape of a head-worn electronic device (520) within an image acquired through the camera (523). For example, a method for identifying a reflected light signal and identifying the shape of a head-worn electronic device (520) will be described later with reference to FIGS. 9a and 9b. For example, after the operation of the device identification module (538) is performed, the operation of the face recognition module (540) may be performed.

[0144] For example, the head movement tracking module (637) can identify the orientation of the head-worn electronic device (520) based on motion data (e.g., an inertial measurement unit (IMU) signal) (532) and an image (533). For example, the motion data (532) can be acquired by at least one sensor (e.g., an inertial measurement unit (IMU) sensor). For example, the image (533) can be acquired through a camera (523). For example, the head movement tracking module (637) can acquire information about at least one of the direction, angle, or position of the head-worn electronic device (520). For example, a method for acquiring information about the orientation of the head-worn electronic device (520) will be described later with reference to FIG. 14. For example, the operation of the size measurement module (542) can be performed after the operation of the head movement tracking module (537) is performed.

[0145] For example, the face recognition module (540) can identify whether a visual object representing a user's face exists within an image acquired through the camera (523). For example, the face recognition module (540) can identify whether a visual object representing a face exists based on the location of a visual object representing a head-worn electronic device (520) within the image. A method for identifying whether a visual object representing a user's face exists within an image will be described later with reference to FIG. 9a. For example, after the operation of the face recognition module (540) is performed, the operation of the guide generation module (539) and / or the face feature point identification module (541) may be performed.

[0146] For example, the guide generation module (539) may provide a notification to the user to guide a change in the orientation of the head-worn electronic device (520) when an image of a specified angle is required. For example, a method for providing a notification to guide a change in the orientation of the head-worn electronic device (520) will be described later with reference to FIG. 14. For example, the face feature point identification module (541) may identify multiple face feature points within an image representing the user's face. For example, a method for identifying multiple face feature points within an image representing the user's face will be described later with reference to FIG. 12. For example, the operation of the size measurement module (542) may be performed after the operation of the face feature point identification module (541) is performed.

[0147] For example, the size measurement module (542) can identify the size of the user's face based on information regarding the size of the head-worn electronic device (520) and information regarding a plurality of facial feature points. For example, a method for identifying the size (or shape) of the user's face will be described later with reference to FIG. 8. For example, after the operation of the size measurement module (542) is performed, the operation of the 3D Reconstruction module (543) may be performed. For example, the 3D Reconstruction module (543) can obtain three-dimensional mesh data based on information regarding the size of the face obtained through the size measurement module. For example, the 3D Reconstruction module (543) can obtain three-dimensional mesh data based on information regarding a plurality of facial feature points. For example, a method for obtaining three-dimensional mesh data will be described later with reference to FIG. 15 and FIG. 16.

[0148] For example, the device shape removal module (544) can remove the shape of the head-worn electronic device (520) from the image based on identifying a user wearing the head-worn electronic device (520) in the image obtained through the camera (523). For example, a method for removing the shape of the head-worn electronic device (520) from the image will be described later with reference to FIG. 13. For example, the operation of the avatar generation module (548) may be performed after the operation of the device shape removal module (544) is performed.

[0149] For example, the accessory identification module (545) can remove the shape of the accessory from an image representing a user of a head-worn electronic device (520) wearing the accessory. For example, a method for removing the shape of the accessory from an image representing the user's face will be described later with reference to FIG. 11. For example, the operation of the device shape removal module (544) may be performed after the operation of the accessory identification module (545) is performed. For example, the texture generation module (546) can generate a texture based on mesh data representing the user's face. For example, the texture may be described as the user's skin texture. A method for generating the texture will be described later with reference to FIG. 8. For example, the operation of the accessory identification module (545) and / or the white balancing module (547) may be performed after the operation of the texture generation module (546) is performed. For example, the white balancing module (547) can perform white balancing based on the generated texture. For example, the white balancing module (547) can perform white balancing based on information regarding the color of the head-worn electronic device (520). For example, a method for performing white balancing will be described later with reference to FIG. 8. For example, the operation of the avatar generation module (548) can be performed after the operation of the white balancing module (547) is performed.

[0150] For example, the avatar generation module (548) can generate an avatar representing a user based on information about the user's face. The information about the face may include at least one of information about the external appearance of the face, information about the shape of the face, information about mesh data representing the face, and / or information about a plurality of facial feature points. For example, a method for generating an avatar representing a user will be described later with reference to FIG. 16.

[0151] FIG. 6 illustrates a method for obtaining information about a face based on an image of a user wearing a head-worn electronic device.

[0152] Referring to FIG. 6, the electronic device (510) can obtain information about the face (660) by using an image (640) obtained through a camera (523) of the head-worn electronic device (520). For example, the electronic device (510) may correspond to the electronic device (101) described with reference to FIG. 1. For example, the head-worn electronic device (520) may correspond to the wearable device (200) described with reference to FIG. 2a through FIG. 4. For example, the head-worn electronic device (520) may obtain an image (640) through the camera (523) showing a user (610) wearing the head-worn electronic device (520) positioned in front of a reflector (620) (e.g., a mirror).

[0153] According to an embodiment, a head-worn electronic device (520) can identify a device worn by a user (610) within an image (640) representing a user (610) standing in front of a reflector (620). For example, the head-worn electronic device (520) can identify whether a device worn by a user (610) within the image (640) is a head-worn electronic device (520) using an optical signal. A method for identifying whether a device worn by a user (610) within the image (640) is a head-worn electronic device (520) using an optical signal will be described later with reference to FIGS. 9a and 9b.

[0154] For example, a head-worn electronic device (520) may be able to communicate with an electronic device (510). For example, the head-worn electronic device (520) may transmit information about an image (640) to the electronic device (510) via a communication circuit (524). For example, the electronic device (510) may receive information about an image (640) from the head-worn electronic device (520) via a communication circuit (512).

[0155] For example, the electronic device (510) can remove the head-worn electronic device (520) from the image (640). For example, the electronic device (510) can obtain an image (650) from which the head-worn electronic device (520) has been removed. A method for removing the head-worn electronic device (520) from the image (640) will be described later with reference to FIGS. 12 and 13. For example, the electronic device (510) can obtain information (660) about the face of the user (610) based on the image (650). Information about the face (660) will be described later with reference to FIG. 8.

[0156] For example, multiple images (e.g., images (640)) may be required to obtain information about the face (660). For example, an electronic device (510) may transmit a signal to a head-worn electronic device (520) via a communication circuit (512) to request information about the images for obtaining information about the face (660). A method for transmitting a signal to request information about the images will be described later with reference to FIG. 14.

[0157] For example, an electronic device (510) can generate a user's avatar based on information about the face (660). Information about the face (660) can be described as information about a plurality of facial feature points. A method for obtaining information about the face based on information about a plurality of facial feature points will be described later with reference to FIG. 16. For example, the electronic device (510) can provide a service regarding the generated avatar. For example, a head-worn electronic device (520) can provide a service regarding the generated avatar.

[0158] According to an embodiment, a service regarding a generated avatar may include a service that provides information regarding the user's health based on information regarding the user's avatar. For example, an electronic device (510) may obtain information regarding the user's (610) health by analyzing the user's (610) face based on information regarding the generated avatar. For example, the electronic device (510) may provide a notification to the user (610) inquiring about the user's (610) sleep satisfaction (or health, pattern, quality) based on information regarding the user's (610) health.

[0159] For example, a service regarding a generated avatar may include a service that provides information about clothing suitable for the user based on information about the avatar. For example, a generated avatar may be described as a character within a virtual environment. For example, a service regarding an avatar may include a service that provides a game in a virtual environment using the avatar.

[0160] FIG. 7 is a flowchart illustrating the operation of transmitting information about an image from a head-worn electronic device to an electronic device. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0161] Referring to FIG. 7, according to an embodiment, operations 701 to 703 may be performed by a head-worn electronic device (520), and operations 704 to 705 may be performed by an electronic device (510), but the entity performing the operations is not limited to the above-mentioned example.

[0162] In operation 701, the head-worn electronic device (520) can identify the head-worn electronic device (520) using an optical signal based on an input requesting the creation of an avatar of the user (610). A method of identifying whether a device worn by the user within an image (e.g., image (540) of FIG. 6) is the head-worn electronic device (520) using an optical signal will be described later with reference to FIG. 9a and FIG. 9b. For example, the avatar may be described as a visual object representing the user (610). For example, the avatar may be described as a visual object representing the appearance of the user (610). For example, the avatar may be represented as a three-dimensional visual object.

[0163] In operation 702, the head-worn electronic device (520) can acquire an image representing the face of the user (610) (e.g., image (640) of FIG. 6) through the camera (523). For example, the head-worn electronic device (520) can store information about the image representing the face of the user (610). For example, the head-worn electronic device (520) can store information about the image representing the face of the user (610) in conjunction with information about the direction in which the camera (523) is facing. A method of storing information about the image representing the face of the user (610) in conjunction with information about the direction in which the camera (523) is facing will be described later with reference to FIG. 14.

[0164] In operation 703, the head-worn electronic device (520) can transmit information about an image representing the user (610) to the electronic device (510) via the communication circuit (524). For example, the electronic device (510) can receive information about an image representing the user (610) from the head-worn electronic device (520) via the communication circuit (512). For example, the information about the image representing the user (610) may include information about the direction in which the camera (523) is facing. For example, the information about the image representing the user (610) may include information about the time and / or place in which the image was acquired.

[0165] In operation 704, the electronic device (510) may obtain information about the face of the user (610) based on information about an image representing the user (610). For example, information about the face may be described as information about the appearance of the face. For example, information about the face may include information about the shape of the face, information about the size of the face, information about the color of the face, and / or information about the texture of the face. Information about the shape of the face, information about the size of the face, information about the color of the face, and information about the texture of the face will be described later with reference to FIG. 8.

[0166] In operation 705, the electronic device (510) may provide a service regarding an avatar created based on information about the user (610)'s face. For example, the avatar may be represented as a visual object that realistically depicts the user (610). For example, the avatar may be represented in an animation style. For example, the avatar may be represented in a cartoon style.

[0167] For example, the electronic device (510) can provide services regarding the avatar by displaying a visual object corresponding to the avatar through a display. For example, the electronic device (510) can transmit information regarding the avatar to another electronic device that is distinct from the electronic device (510). For example, another electronic device that receives information regarding the avatar can display a visual object corresponding to the avatar through a display.

[0168] According to an embodiment, the electronic device (510) can transmit information about the avatar to the head-worn electronic device (520) via the communication circuit (512). For example, the head-worn electronic device (520) can provide services related to the avatar based on the information about the avatar received via the communication circuit (524). For example, the head-worn electronic device (520) can display a visual object corresponding to the avatar via the display (527) based on the information about the avatar.

[0169] According to an embodiment, operations 701 to 702 and operations 704 to 705 may be performed by a head-worn electronic device. For example, the head-worn electronic device (520) may identify the head-worn electronic device (520) using an optical signal based on an input requesting the creation of an avatar of a user (610). For example, the head-worn electronic device (520) may acquire an image (640) representing the face of the user (610) through a camera (523). For example, the head-worn electronic device (520) may acquire information about the face of the user (610) based on the image (640). For example, the head-worn electronic device (520) may create an avatar of the user (610) based on the information about the face. For example, the head-worn electronic device (520) may provide a service regarding the created avatar.

[0170] As described above, according to the operation described in FIG. 7, the electronic device (510) can generate an avatar of the user (610) based on the head-worn electronic device (520) transmitting information about an image representing a user (610) standing in front of a reflector to the electronic device (510). For example, the head-worn electronic device (520) and the electronic device (510) can generate an avatar of the user (610) while the user (610) is wearing the head-worn electronic device (520).

[0171] Figure 8 illustrates information about the face.

[0172] Referring to FIG. 8, information about the face may correspond to information about the external shape of the face. For example, information about the external shape of the face may include information about the shape of the face (820), information about the size of the face (821), and / or information about the skin of the face (822). As an example, information about the skin of the face (822) may include information about the texture of the face and information about the color of the face.

[0173] For example, information regarding texture can be expressed as the degree of roughness and / or smoothness of the skin. For example, information regarding color can be expressed by RGB (red, green, and blue) data. For example, information regarding color may include information regarding hue, saturation, and lightness.

[0174] According to an embodiment, the electronic device (510) can identify information regarding the external appearance of the head-worn electronic device (520). For example, the information regarding the external appearance of the head-worn electronic device (520) may include information regarding the shape (810) of the head-worn electronic device (520), information regarding the size (811) of the head-worn electronic device (520), and / or information regarding the color (812) of the head-worn electronic device (520). For example, the electronic device (510) may obtain information regarding the face based on the information regarding the external appearance of the head-worn electronic device (520). For example, the information regarding the face may be determined based on the information regarding the external appearance of the head-worn electronic device (520).

[0175] For example, the electronic device (510) can obtain information (820) about the shape of the face based on information (810) about the shape of the head-worn electronic device (520). A method for obtaining information (820) about the shape of the face based on information (810) about the shape of the head-worn electronic device (520) will be described later with reference to FIG. 12.

[0176] For example, the electronic device (510) can obtain information (821) about the size of the face based on information (811) about the size of the head-worn electronic device (520). For example, the electronic device (510) can identify information about the size of the head-worn electronic device (520). For example, information about the size of the head-worn electronic device (520) may be stored in memory (513). Information (811) about the size of the head-worn electronic device (520) may include information (811-1) about the length of the temples of the glasses and / or information (811-2) about the width of the glasses when viewed from the front. For example, the temples may correspond to the first temple (204) or the second temple (205) described with reference to FIG. 2a. For example, information (821) regarding the size of the face may include information (821-1) regarding the width of the eyes or information (821-2) regarding the distance between the nose and the chin. For example, the electronic device (510) may obtain information regarding the ratio between the size of the head-worn electronic device (520) and the size of the face from an image (640) representing the user (610). For example, the electronic device (510) may obtain information (821) regarding the size of the face based on the information regarding the ratio and the information regarding the size of the head-worn electronic device (520). The information (811) regarding the size of the head-worn electronic device (520) is not limited to the examples described above. According to an embodiment, the information (811) regarding the size of the head-worn electronic device (520) may include information regarding the size of the lens, information regarding the shape of the rim of the glasses, and information regarding the height of the glasses.

[0177] For example, the electronic device (510) can obtain information (822) about the skin of the face based on information about the color of the head-worn electronic device (520). The information about the skin of the face (822) may include information about the color of the face and information about the texture of the face. The electronic device (510) can identify information about the color of the head-worn electronic device (520) stored in memory (513). The electronic device (510) can identify information (812) about the color of the head-worn electronic device (520) within an image (640). The electronic device (510) can identify the difference between the information (812) about the color of the head-worn electronic device (520) within the image (640) and the information about the color stored in memory (513). The head-worn electronic device (520) can change the color of the skin on the face of the user (610) in the image (640) based on the difference. Changing the color of the skin on the face can be described as white balancing. For example, the electronic device (510) can determine a reference color based on information about the color stored in memory (513) through white balancing. For example, the electronic device (510) can change the color of the skin on the face of the user (610) in the image (640) by adjusting the color in the image (640) based on the reference color through white balancing.

[0178] For example, the electronic device (510) can obtain information about the texture of the user's (610) face within the image (640). For example, the electronic device (510) can obtain information about the texture based on identifying an area corresponding to the user's skin within the image (640). For example, the electronic device (510) can obtain information about the texture by identifying a pattern of pixels constituting the image (640) within the area corresponding to the skin.

[0179] FIG. 9a is a flowchart illustrating the operation of creating an avatar based on acquiring face photos from various angles.

[0180] Referring to FIG. 9a, in operation 901, the head-worn electronic device (520) may output an optical signal through a light-emitting element (e.g., the light-emitting element (522) of FIG. 5) based on a modulated signal. For example, the output optical signal may be reflected by a reflector (e.g., the reflector (620) of FIG. 6). For example, the head-worn electronic device (520) may generate a temporally modulated signal. For example, the output optical signal may correspond to the generated signal. For example, a method of outputting an optical signal based on a modulated signal will be described later with reference to FIG. 9b.

[0181] In operation 902, the head-worn electronic device (520) can store information about an image acquired through a camera (523) and an angle of the head-worn electronic device (520). For example, the head-worn electronic device (520) can store information about the image and angle in an image buffer. For example, information about the angle of the head-worn electronic device (520) can be described as information about the angle of the head-worn electronic device (520) at the moment the image is captured. For example, the head-worn electronic device (520) can acquire information about the angle using motion data acquired based on at least one sensor (e.g., an inertial measurement unit (IMU) sensor or a gyroscope).

[0182] In operation 903, the head-worn electronic device (520) can identify a light signal reflected from a reflector based on an acquired image. For example, the head-worn electronic device (520) can transmit information about the image to the electronic device (510) through a communication circuit (524). According to an embodiment, the electronic device (510) can identify a light signal reflected from a reflector based on information about the received image.

[0183] In operation 904, the head-worn electronic device (520) can identify whether the identified optical signal corresponds to the modulated signal in operation 901. For example, the head-worn electronic device (520) can identify whether the optical signal corresponds to the modulated signal based on the on / off pattern of the optical signal. For example, a method for identifying whether the optical signal corresponds to the modulated signal will be described later with reference to FIG. 9b.

[0184] In operation 905, the head-worn electronic device (520) can identify whether the shape of the head-worn electronic device (520) is identified within an image based on identifying that the identified optical signal corresponds to a modulated signal. For example, the head-worn electronic device (520) can enhance the authentication (or security) required to perform avatar creation based on identifying the optical signal corresponding to the modulated signal and identifying the shape of the head-worn electronic device (520) within the image.

[0185] In operation 906, the head-worn electronic device (520) can identify whether the time elapsed since the start of the output of the optical signal exceeds a threshold time based on identifying that the identified optical signal does not correspond to a modulated signal. For example, the head-worn electronic device (520) can identify whether the time elapsed since the start of the output of the optical signal exceeds a threshold time based on identifying that the shape of the head-worn electronic device (520) is not identified within the image.

[0186] In operation 907, the head-worn electronic device (520) may provide a notification to guide the user's movement based on identifying that the time elapsed since the start of the output of the optical signal has not exceeded a threshold time. For example, the head-worn electronic device (520) may provide the notification by outputting a voice signal to guide the user's movement through a speaker (e.g., speaker (525) in FIG. 5a). For example, the head-worn electronic device (520) may provide the notification by displaying a visual object to guide the user's movement through a display (e.g., display (527) in FIG. 5a). For example, after providing the notification to guide the user's movement, the head-worn electronic device (520) may acquire an image representing the user's face through a camera (523). For example, after providing the notification to guide the user's movement, the head-worn electronic device (520) may perform operation 902.

[0187] In operation 908, the head-worn electronic device (520) may terminate the operation to create an avatar representing a user based on identifying that the time elapsed since the start of the output of the optical signal exceeds a threshold time. For example, the head-worn electronic device (520) may determine that the authentication required for creating an avatar is not completed based on identifying that the time elapsed since the start of the output of the optical signal exceeds a threshold time. For example, the head-worn electronic device (520) may refrain from the operation to create an avatar based on the determination that the authentication required for creating an avatar is not completed.

[0188] In operation 909, the head-worn electronic device (520) can identify whether a user's face is present in an image. For example, the head-worn electronic device (520) can identify the presence of a face by identifying a visual object representing a user's face in an image. For example, the head-worn electronic device (520) can identify a visual object representing a user's face in an image based on information regarding the shape of the head-worn electronic device (520). For example, the head-worn electronic device (520) can identify a visual object representing a user's face in an image based on a plurality of feature points regarding the shape of the head-worn electronic device (520). For example, the head-worn electronic device (520) can provide a notification to guide the user's movement based on identifying that a user's face is not present in an image. The above example is based on the operation 909 being performed by a head-worn electronic device (520), but according to the embodiment, the operation 909 may be performed by an electronic device (510) that receives information about an image.

[0189] In operation 910, the head-worn electronic device (520) can identify whether all photos of a designated angle have been taken. For example, the head-worn electronic device (520) can designate (or set) the angles of multiple photos required for creating an avatar representing the user. For example, the angle of the photos may be related to the orientation of the head-worn electronic device (520). For example, the head-worn electronic device (520) can identify at least one of the front, left side, and / or right side of the user's face from the photos taken, based on identifying that all photos of a designated angle have been taken. For example, the head-worn electronic device (520) can provide a notification to guide the user's movement based on identifying that all photos of a designated angle have not been taken. For example, the head-worn electronic device (520) can provide a notification to obtain photos of untaken angles based on identifying that all photos of a designated angle have not been taken.

[0190] In operation 911, the head-worn electronic device (520) can obtain information about a face based on photographs of a specified angle. For example, the head-worn electronic device (520) can obtain information about a face based on a plurality of images of a specified angle. For example, the head-worn electronic device (520) can identify a plurality of facial feature points representing a face based on a plurality of photographs of a specified angle. For example, a method for obtaining information about a face will be described later with reference to FIGS. 12, FIGS. 15, and FIGS. 16. In the example described above, operation 911 is written based on being performed by the head-worn electronic device (520), but operation 911 can be performed by the electronic device (510).

[0191] In operation 912, the head-worn electronic device (520) can generate an avatar representing a user based on information about the face. For example, the electronic device (510) can generate an avatar representing a user based on information about the face. For example, a method for generating an avatar representing a user will be described later with reference to FIG. 16.

[0192] As described above, according to the operation described in FIG. 9a, the head-worn electronic device (520) can perform dual operations related to authentication required for avatar creation. For example, the head-worn electronic device (520) can guide the user to obtain images of angles required for avatar creation. For example, the head-worn electronic device (520) can determine whether to create an avatar based on a threshold time.

[0193] FIG. 9b illustrates a method for identifying a head-worn electronic device using an optical signal.

[0194] Referring to FIG. 9b, the head-worn electronic device (520) can identify whether the head-worn electronic device in the image (960) corresponds to the head-worn electronic device (520) by using a light signal output from a light-emitting element (e.g., 522 in FIG. 5). For example, the light-emitting element (522) may correspond to a light-emitting module (not shown) described with reference to FIG. 2a and FIG. 2b. For example, the light-emitting element (522) may be placed on the rim of the head-worn electronic device (520) (e.g., rim (201) and rim (202) in FIG. 2a). As an example, the light-emitting element (522) may be placed at the left end point (902-1) or the right end point (902-2) of the rim of the head-worn electronic device (520). For example, the light-emitting element (522) may be placed on the bridge of the head-worn electronic device (520) (e.g., the bridge (203) in FIG. 2a). As an example, the light-emitting element (522) may be placed at the midpoint (901) of the bridge of the head-worn electronic device (520). Although the location of the light-emitting element (522) has been described, the present disclosure is not limited thereto.

[0195] For example, a head-worn electronic device (520) can generate a temporally modulated signal. For example, the temporally modulated signal can be described as a signal corresponding to a time-series sequence of multiple pulse signals. For example, the head-worn electronic device (520) can output an optical signal through a light-emitting element (522) based on the temporally modulated signal. For example, the optical signal may correspond to the temporally modulated signal.

[0196] For example, a head-worn electronic device (520) can output light through a light-emitting element (522). For example, the head-worn electronic device (520) can output a light signal (940) based on a binary on / off pattern. For example, the head-worn electronic device (520) can output a light signal (940) by using a light-emitting element (522) to output light of a specified pattern. For example, the head-worn electronic device (520) can identify a light signal corresponding to an on / off pattern. For example, the light signal (940) can be described as a modulated signal.

[0197] According to an embodiment, a light signal may be output through a light-emitting element that emits at least one of light of a wavelength corresponding to red, light of a wavelength corresponding to green, and light of a wavelength corresponding to blue. For example, a head-worn electronic device (520) may output a light signal through a light-emitting element (522) based on a time-series pattern consisting of colors of red, green, and blue. For example, the head-worn electronic device (520) may identify a light signal reflected from a reflector (930) based on a time-series pattern consisting of colors of red, green, and blue.

[0198] For example, an optical signal (940) may be reflected from a reflector (930). At least a portion of the optical signal (940) may be reflected from the reflector (930). At least a portion of the optical signal (940) reflected from the reflector (930) may be identified by a head-worn electronic device (520). The head-worn electronic device (520) may identify an optical signal (950) corresponding to at least a portion of the optical signal (940) reflected from the reflector (930). For example, the head-worn electronic device (520) may identify the reflected optical signal (950) through a camera. For example, the head-worn electronic device (520) may identify the optical signal (950) based on a plurality of images acquired through a camera (523).

[0199] According to an embodiment, the head-worn electronic device (520) can identify whether the optical signal (950) corresponds to the temporally modulated signal. For example, the temporally modulated signal may correspond to the optical signal (940). For example, at least a portion of the optical signal (950) may correspond to at least a portion of the temporally modulated signal. For example, the head-worn electronic device (520) can identify the optical signal (950) corresponding to the optical signal (940) by identifying the optical signal (950) corresponding to the temporally modulated signal.

[0200] For example, the camera (523) can be synchronized with the light-emitting element (522). For example, the time at which an image is acquired through the camera (523) can be synchronized with the time at which light is emitted through the light-emitting element (522). For example, a head-worn electronic device (520) can acquire an image through the camera (523) based on the time at which light is emitted through the light-emitting element (522).

[0201] For example, a head-worn electronic device (520) can obtain multiple images through a camera (523) for a reference time after outputting a light signal (940). Based on identifying a light signal (950) reflected from a reflector (930) within the multiple images obtained for the reference time, information about an image (960) representing a user (610) can be transmitted to the electronic device (510).

[0202] For example, a head-worn electronic device (520) can identify multiple visual objects within an image (960) based on information about the image (960). For example, the head-worn electronic device (520) can generate a segmentation map based on identifying multiple visual objects within the image (960). For example, the head-worn electronic device can generate a segmentation map using a model for identifying visual objects (e.g., a deep learning-based model). For example, the head-worn electronic device (520) can store information about the segmentation map in memory (526). For example, the head-worn electronic device (520) can transmit information about the segmentation map to an electronic device (510) through a communication circuit (524).

[0203] According to an embodiment, the electronic device (510) can identify a plurality of visual objects within an image (960) based on information about the image (960). For example, the electronic device (510) can generate a segmentation map based on identifying a plurality of visual objects within the image (960). For example, the electronic device (510) can store information about the segmentation map in memory (513).

[0204] According to an embodiment, the head-worn electronic device (520) may refrain from transmitting information about an image (960) to the electronic device (510) based on the fact that a light signal (950) is not identified in a plurality of images acquired during the reference time. For example, the head-worn electronic device (520) may provide a notification indicating that it has failed to perform authentication related to the head-worn electronic device (520) based on refraining from transmitting information about the image (960) to the electronic device (510). For example, the head-worn electronic device (520) may display a visual object indicating that it has failed to perform the authentication through a display (527). For example, the head-worn electronic device (520) may output an acoustic signal indicating that it has failed to perform the authentication through a speaker (525).

[0205] For example, the head-worn electronic device (520) can identify whether the optical signal (950) corresponds to the optical signal (940). For example, the head-worn electronic device (520) can identify whether the pattern corresponding to the optical signal (950) is identical to the pattern corresponding to the optical signal (940). For example, the head-worn electronic device (520) can identify whether the optical signal (950) corresponds to the optical signal (940) by identifying whether at least a portion (941) of the optical signal (950) corresponds to at least a portion (951) of the optical signal (940). For example, the head-worn electronic device (520) can identify that the head-worn electronic device (970) in the image (960) corresponds to the head-worn electronic device (520) by identifying the optical signal (950) corresponding to the optical signal (940). For example, a head-worn electronic device (520) can identify whether an optical signal (950) corresponds to an optical signal (940) by using dynamic time warping (DTW). For example, DTW can be described as an algorithm that obtains information about the similarity between two time series sequences of different speeds and / or lengths. For example, a head-worn electronic device (520) can identify whether an optical signal (950) corresponds to an optical signal (940) of a different length from the optical signal (950) by using DTW.

[0206] According to an embodiment, the head-worn electronic device (520) can identify whether the head-worn electronic device (970) in the image (960) corresponds to the head-worn electronic device (520) in order to enhance security, based on identifying a light signal (950) corresponding to a light signal (940). For example, the head-worn electronic device (520) can identify whether the shape of the head-worn electronic device (970) in the image (960) corresponds to the shape of the head-worn electronic device (520), based on identifying a light signal (950) corresponding to a light signal (940). For example, the head-worn electronic device (520) can identify that the head-worn electronic device (970) corresponds to the head-worn electronic device (520) by identifying that the shape of the head-worn electronic device (970) corresponds to the shape of the head-worn electronic device (520).

[0207] According to an embodiment, the head-worn electronic device (520) may emit a light signal (940) based on receiving an input requesting the creation of an avatar. For example, the input requesting the creation of an avatar may be described as a user's voice input (920). For example, the user's voice input (920) requesting the creation of an avatar may indicate "create an avatar."

[0208] For example, an input requesting the creation of an avatar can be described as the identification of a reflector (930) using a camera (523). For example, a head-worn electronic device (520) can acquire an image through the camera (523). The head-worn electronic device (520) can identify a reflector (930) within the acquired image. Based on identifying the reflector (930), the head-worn electronic device (520) can output an acoustic signal through a speaker (525) asking the user whether to create an avatar. For example, the acoustic signal asking whether to create an avatar can indicate, "You are standing in front of a mirror. Would you like to create an avatar?"

[0209] As described above, according to the method described in FIG. 9b, the head-worn electronic device (520) can identify whether the user (610) who requested the creation of the avatar is wearing the head-worn electronic device (520) before creating the avatar. For example, if multiple head-worn electronic devices are identified within the image (960), the avatar can be created based on the user (610) who is wearing the head-worn electronic device (520) that emits a light signal. For example, the head-worn electronic device (520) can prevent the act of deceiving others by unauthorizedly photographing the face of another person who is distinct from the user (610) and pretending to be said other person in a virtual space, by identifying, based on the light signal, whether the head-worn electronic device within the image corresponds to the head-worn electronic device (520) worn by the user (610).

[0210] FIG. 10 is a flowchart illustrating the operation of acquiring information about a face based on removing a head-worn electronic device from an image. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0211] Referring to FIG. 10, in operation 1001, the electronic device (510) can remove an object distinct from the user (610) within the image based on receiving information about the image (e.g., image (640) of FIG. 6) from the head-worn electronic device (520). For example, the electronic device (510) can obtain information about the image from which the object has been removed based on identifying an object distinct from the user included in the image. For example, the electronic device (510) can identify a plurality of objects distinct from the user within the image received from the head-worn electronic device (520). For example, the plurality of objects distinct from the user may include the head-worn electronic device (520), earrings, and / or nose rings. For example, a method for obtaining information about the image from which the shape of the head-worn electronic device (520) has been removed among the plurality of objects distinct from the user will be described later with reference to FIG. 12. For example, a method for obtaining information about an image from which other objects distinct from the user (e.g., accessories excluding head-worn electronic devices) have been removed will be described later with reference to FIG. 11.

[0212] In operation 1002, the electronic device (510) can obtain an image with distortion caused by a lens removed from the image based on information about an image from which an object distinguishable from the user (610) has been removed. For example, the electronic device (510) can obtain an image with distortion caused by a lens removed from the image based on information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can obtain an image with distortion caused by a lens removed from the image based on mesh data representing the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify information about a plurality of feature points included in the information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify a plurality of feature points of the head-worn electronic device (520) within the image. For example, the plurality of feature points identified within the image can be described as a plurality of feature points projected onto a plane. For example, the electronic device (510) can identify multiple feature points in three-dimensional space based on identifying multiple feature points of a head-worn electronic device (520) within an image. For example, the electronic device (510) can establish a 3D coordinate system based on identifying multiple feature points in 3D space. For example, one of the multiple feature points can be the origin of the 3D coordinate system. For example, after identifying the 3D coordinate system established based on the multiple feature points, the electronic device (510) can obtain an image with lens distortion removed from the image.

[0213] For example, the electronic device (510) can identify information about the distortion of the lens. For example, the electronic device (510) can identify information about the distortion of the lens stored in memory (513). For example, the lens may be included in a head-worn electronic device (520).

[0214] For example, the electronic device (510) can identify information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can obtain information about the position of the lens within the head-worn electronic device (520) based on information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can obtain information about the position of the lens within an image (e.g., image (640) of FIG. 6) based on information about the position of the lens within the head-worn electronic device (520). For example, the electronic device (510) can identify an area distorted by the lens within the image based on information about the position of the lens within the image.

[0215] For example, distortion caused by a lens may include refraction and / or reflection. For example, distortion caused by a lens may occur because light passing through an area corresponding to a lens within an image may be refracted by the lens. For example, distortion caused by a lens may occur because light passing through an area corresponding to a lens within an image may be reflected by the lens. For example, information regarding lens distortion may include information regarding the refractive index and / or reflectivity.

[0216] For example, the electronic device (510) can identify a visual object distinct from the user (610) in an area corresponding to a lens within the image based on information regarding reflectance. For example, the electronic device (510) can obtain an image from which the visual object has been removed. A method for obtaining an image from which the visual object has been removed will be described later with reference to FIG. 13.

[0217] For example, the electronic device (510) can identify, based on information regarding the refractive index, that the area corresponding to the lens within the image is distorted due to light refracted by the lens. For example, the electronic device (510) can obtain information about an image from which distortion due to refraction has been removed. A method for obtaining information about an image from which distortion due to refraction has been removed will be described later with reference to FIG. 13.

[0218] In operation 1003, the electronic device (510) can obtain information about the shape of the user (610)'s face (e.g., information about the shape of FIG. 8 (820)) based on information about the shape of the head-worn electronic device (520) (e.g., information about the shape of FIG. 8 (810)). For example, the electronic device (510) can identify landmarks on the face based on information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can obtain information about the shape of the face based on an image from which the head-worn electronic device (520) has been removed. For example, the electronic device (510) can obtain information about the shape of the face through a face feature point identification module (e.g., the face feature point identification module (541) of FIG. 5b). For example, the electronic device (510) can identify a plurality of facial feature points regarding the shape of the face through the facial feature point identification module (541). For example, the electronic device (510) can identify landmarks regarding the shape of the face through the facial feature point identification module (541). For example, the electronic device (510) can identify the relative position of the landmarks regarding the position of the head-worn electronic device through the facial feature point identification module (541). For example, the electronic device (510) can identify the relative positions of a plurality of facial feature points regarding a plurality of feature points regarding the shape of the head-worn electronic device. A method for obtaining information regarding the shape of the face based on information regarding the shape of the face-worn electronic device (520) will be described later with reference to FIG. 12.

[0219] According to an embodiment, the electronic device (510) may set a first threshold value for at least a portion of the face to be identified as a landmark. For example, the electronic device (510) may obtain a value corresponding to at least a portion of the face and associated with the identification of the landmark. For example, the electronic device (510) may identify said at least a portion of the face as a landmark based on identifying that the value corresponding to at least a portion of the face exceeds the threshold value. For example, if the user (610) does not properly (or correctly) wear the head-worn electronic device (520), the landmark of the user's (610) face may not be identified. In this case, the electronic device (510) may identify said at least a portion of the face as a landmark based on a second threshold value lower than the first threshold value.

[0220] In operation 1004, the electronic device (510) can generate an avatar based on information about the color of the face and information about the texture of the face. For example, the information about the face may include information about the skin of the face (e.g., information about the skin (822) of FIG. 8). The information about the skin may include information about the color of the face and information about the texture of the face. The electronic device (510) can generate an avatar that realistically depicts the user by using information about the color of the face and information about the texture of the face.

[0221] According to the embodiment, the order of operation 1001, operation 1002, and operation 1003 may be changed. For example, operation 1001, operation 1002, and operation 1003 may be performed concurrently. For example, the electronic device (510) may acquire information about the shape of the face based on information about the shape of the head-worn electronic device (520), and then acquire an image in which an object distinguishable from the user (610) within the image is removed. For example, the electronic device (510) may acquire an image in which an object distinguishable from the user (610) is removed after acquiring an image in which distortion caused by the lens is removed.

[0222] As described above, according to the operation described in FIG. 10, the electronic device (510) can generate an avatar that realistically depicts a user based on information about an image received from a head-worn electronic device (520). For example, the electronic device (510) can generate an avatar representing a user based on information about the appearance of the head-worn electronic device (520) (e.g., information about shape, information about size, and information about color).

[0223] According to an embodiment, the electronic device (510) can identify a plurality of face feature points based on information about an image received from a user's head-worn electronic device (520). For example, the plurality of face feature points can each be represented as a vector in 3D space. For example, the electronic device (510) can transfer (or map) the plurality of face feature points onto a single vector space. For example, the electronic device (510) can identify the locations of the transferred plurality of face feature points. For example, the electronic device (510) can obtain mesh data representing a face based on the locations of the plurality of face feature points transferred onto the vector space. For example, the electronic device (510) can deform the mesh data by adjusting the locations of the plurality of face feature points. For example, the electronic device (510) can map a texture generated through a texture generation module (546) onto the deformed mesh data. For example, the electronic device (510) can store information about the texture so as to be linked to the modified mesh data. For example, the electronic device (510) can generate an avatar representing a user based on the information about the texture and the mesh data.

[0224] Figure 11 illustrates a method for removing accessories.

[0225] Referring to FIG. 11, the electronic device (510) can identify an object (e.g., a head-worn electronic device (1110) and an earring (1120)) that is distinct from the user (610) within an image received from a head-worn electronic device (520). For example, the object distinct from the user (610) may be described as a wearable device and / or an accessory. For example, the object distinct from the user (610) may be an object representing a background distinct from the user (610).

[0226] For example, the electronic device (510) can remove an object that is distinct from the user (610) within the image. For example, the electronic device (510) can obtain an image (1130) from which the object distinct from the user (610) has been removed. For example, the image (1130) from which the object has been removed may correspond to the image (650) described with reference to FIG. 6. A method for obtaining an image from which the head-worn electronic device (1110) has been removed will be described later with reference to FIG. 12.

[0227] For example, the electronic device (510) can use an artificial intelligence (AI) model to identify an object within an image that is distinct from the user (610). For example, the electronic device (510) can use an AI model to obtain an image from which the object distinct from the user (610) has been removed. For example, the AI ​​model may be an on-device model. For example, the AI ​​model may be a model stored on a server. For example, the electronic device (510) can use a generative adversarial network (GAN) model to obtain an image from which the object distinct from the user (610) has been removed.

[0228] According to an embodiment, the electronic device (510) may receive information about a plurality of images from a head-worn electronic device (520). For example, the plurality of images may be captured by a camera (523) of the head-worn electronic device (520) at different angles. For example, the information about the plurality of images may include information about the direction in which the camera (523) is facing. For example, the electronic device (510) may identify an object distinct from the user (610) in each of the plurality of images based on information about the plurality of images according to various directions.

[0229] According to an embodiment, the electronic device (510) can identify information regarding a reference shape of a person's face. For example, the electronic device (510) can identify an object distinct from the user (610) based on information regarding the reference shape of the face. For example, the electronic device (510) can identify an object distinct from the user (610) by comparing the reference shape of the face with a shape representing the user's (610) face within an image.

[0230] FIG. 12 illustrates a method for obtaining information about the shape of a face based on the shape of a head-worn electronic device.

[0231] Referring to FIG. 12, the electronic device (510) can obtain information (1250) about the face based on an image (1210) representing a user wearing a head-worn electronic device (520). For example, the electronic device (510) can identify landmarks on the face based on the image (1210). The landmarks may be described as areas corresponding to the eyes, eyebrows, nose, mouth, and / or chin. The landmarks may be identified based on the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify a portion of the area within the lens of the head-worn electronic device (520) as the user's eyes. For example, the electronic device (510) can identify a portion of the area located on the top of the head-worn electronic device (520) as the user's eyebrows. For example, the electronic device (510) can map landmarks within the image and parts of the user's face (e.g., eyes, eyebrows, nose, mouth, and chin).

[0232] For example, the electronic device (510) can identify the shape of the head-worn electronic device (520) based on the image (1210). For example, the electronic device (510) can identify whether the user's face (or facial features) is present based on identifying the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify whether the user's face is present based on mesh data representing the shape of the head-worn electronic device (520).

[0233] For example, the electronic device (510) can obtain information about the shape of the face based on information about the shape of the head-worn electronic device (520). Information about the shape of the face can be described as information about facial feature points. For example, the electronic device (510) can obtain information about a plurality of facial feature points (1220) based on information about the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify a plurality of facial feature points using a principal components analysis (PCA) model based on information about the shape of the head-worn electronic device (520) and information about an image representing the user (610). For example, the plurality of facial feature points (1220) can be represented as three-dimensional coordinate data. For example, a plurality of facial feature points (1220) can be represented as coordinate data based on the location of the head-worn electronic device (520). For example, the electronic device (510) can identify a specified number (e.g., 3) of feature points (1230) among the plurality of feature points of the head-worn electronic device (520). For example, the electronic device (510) can establish a coordinate system based on the feature points (1230) with one of the feature points (1230) as the origin. For example, a plurality of facial feature points (1220) can be represented as three-dimensional coordinate data based on the coordinate system.

[0234] For example, the electronic device (510) can obtain mesh data representing the user's face based on information regarding a plurality of facial feature points (1220). For example, the electronic device (510) can generate an avatar of the user based on the mesh data. A method for obtaining mesh data based on a plurality of data sets including a plurality of facial feature points will be described later with reference to FIG. 15. A method for obtaining information about the face based on mesh data will be described later with reference to FIG. 16.

[0235] For example, the electronic device (510) can obtain an image (1240) with the head-worn electronic device (520) removed, based on information about the shape of the head-worn electronic device (520) and information about the distortion of the lens. For example, the electronic device (510) can obtain information about the face (1250) based on the image (1240).

[0236] FIG. 13 illustrates a method for correcting a part distorted by a lens.

[0237] Referring to FIG. 13, the electronic device (510) can identify distortion caused by a lens in an image (1310) representing a user (610) received from a head-worn electronic device (520). For example, distortion caused by a lens may include distortion due to refraction and distortion due to reflection. For example, the electronic device (510) can identify information about distortion of the lens stored in memory (513). For example, information about distortion may include information about the refractive index. For example, due to a difference in the thickness of the lenses, the refractive index of lens (1360-1) may be greater than the refractive index of lens (1360-2).

[0238] For example, the electronic device (510) can identify information regarding the position of a lens within the head-worn electronic device (520) based on information regarding the shape of the head-worn electronic device (520). For example, the electronic device (510) can identify the position of a lens within an image (e.g., image (1310) and image (1330)) based on information regarding the position of a lens within the head-worn electronic device (520). For example, the electronic device (510) can obtain an image (1350) from which distortion caused by the lens and the shape of the head-worn electronic device (520) have been removed based on the position of the lens within the image.

[0239] For example, the electronic device (510) can identify the refractive index of the lens included in the information regarding the distortion of the lens. For example, the electronic device (510) can remove distortion caused by refraction by the lens within the image (1310). For example, the electronic device (510) can obtain an image (1350) from which distortion caused by refraction by the lens and the shape of the head-worn electronic device (520) have been removed. For example, the area corresponding to the lens of an image representing a face may be distorted by the refraction of the lens. For example, the boundary of the face within the area corresponding to the lens may be discontinuous with the boundary of the face outside said area.

[0240] For example, the electronic device (510) can obtain an image (1350) in which distortion caused by refraction and the shape of the head-worn electronic device (520) are removed using an AI model. For example, the AI ​​model may include a generative AI model (e.g., a GAN (generative adversarial network) model and a diffusion-based model).

[0241] For example, the electronic device (510) can identify the shape of the head-worn electronic device (520) based on information regarding the segmentation map. For example, the segmentation map may correspond to the segmentation map described with reference to FIG. 9b. For example, the electronic device (510) can identify the shape of the head-worn electronic device (520) as noise based on information regarding the segmentation map. For example, the electronic device (510) can remove the shape of the head-worn electronic device (520) identified as noise from the image (1330). For example, the electronic device (510) can remove the shape identified as noise from the image (1330) using an AI model (e.g., a GAN (generative adversarial network) model). For example, information regarding the segmentation map may be described as information regarding the shape of the head-worn electronic device (520).

[0242] For example, the electronic device (510) can remove refraction caused by the lens based on information about the segmentation map. For example, the electronic device (510) can identify a visual object (1340) based on refraction caused by the lens based on information about the segmentation map. For example, the electronic device (510) can identify the visual object (1340) as noise based on information about the segmentation map. For example, the electronic device (510) can remove refraction caused by the lens using an AI model (e.g., a GAN (generative adversarial network) model) based on the visual object (1340) identified as noise. For example, the electronic device (510) can obtain an image (1350) from an image (1330) by removing refraction caused by the lens.

[0243] For example, the electronic device (510) can eliminate distortion caused by reflection. The electronic device (510) can identify a visual object (1320) that is distinct from the user (610) in an area corresponding to the lens within the image (1310). For example, the visual object (1320) can be described as a visual object reflected by the lens that is distinct from the user (610). For example, the electronic device (510) can remove the visual object (1320) within the image (1310). For example, the electronic device (510) can obtain an image (1330) from which the visual object (1320) has been removed.

[0244] For example, the electronic device (510) can identify a visual object (1320) that is distinct from the user (610) in a lens-corresponding area within the image (1310) based on information regarding the segmentation map. For example, the electronic device (510) can identify the visual object (1320) as noise based on information regarding the segmentation map. For example, the electronic device (510) can remove the visual object (1320) identified as noise from the image (1310) using an AI model (e.g., a GAN (generative adversarial network) model).

[0245] According to an embodiment, the electronic device (510) can remove distortion caused by refraction caused by the lens after removing distortion caused by reflection caused by the lens. For example, the electronic device (510) can remove distortion caused by refraction after removing an object that is distinguishable from the user (610) within the image (1310).

[0246] According to an embodiment, the electronic device (510) can remove distortion caused by reflection caused by the lens after removing distortion caused by refraction caused by the lens. For example, the electronic device (510) can obtain an image in which an object distinguishable from the user (610) within the image (1310) is removed based on removing distortion caused by refraction caused by the lens.

[0247] According to an embodiment, an operation to remove distortion caused by refraction by a lens and an operation to remove distortion caused by reflection by a lens can be performed in parallel.

[0248] FIG. 14 illustrates a method for providing guidance for acquiring images from different angles.

[0249] Referring to FIG. 14, the head-worn electronic device (520) may provide a notification (1430) to guide the acquisition of another image (1460) representing the face of the user (610). For example, the head-worn electronic device (520) may provide a notification (1430) before transmitting information about an image representing the face of the user (610) (e.g., image (640) of FIG. 6 and image (1440) of FIG. 14) to the electronic device (510).

[0250] For example, the notification (1430) can be described as a visual object. For example, a head-worn electronic device (520) can provide the notification (1430) by displaying a visual object to guide the acquisition of another image (1460) through a display (527).

[0251] For example, the notification (1430) can be described as an acoustic signal. For example, a head-worn electronic device (520) can provide the notification (1430) by outputting an acoustic signal to guide the acquisition of another image (1460) through a speaker (525).

[0252] For example, the head-worn electronic device (520) can obtain information about the direction in which the camera is facing. For example, the head-worn electronic device (520) may include an inertial measurement unit (IMU). For example, the IMU may include an accelerometer and / or a gyroscope. For example, the head-worn electronic device (520) can obtain information about the direction in which the camera is facing through the IMU.

[0253] For example, information regarding the direction in which the camera (523) is facing may include information regarding the position of the camera (523), information regarding the speed at which the camera (523) is moving, and information regarding the angle between the camera (523) and the reflector (1410). As an example, a head-worn electronic device (520) may obtain information regarding the position, information regarding the speed, and information regarding the angle through an IMU.

[0254] According to an embodiment, information regarding the direction may be described based on information regarding the shape of the head-worn electronic device (520) within an image (e.g., image (1440) or image (1460)). For example, the first shape of the head-worn electronic device (520) according to the first direction may differ from the second shape of the head-worn electronic device (520) according to the second direction. For example, the head-worn electronic device (520) may identify information regarding the shape of the head-worn electronic device (520) stored in memory (526).

[0255] For example, a head-worn electronic device (520) may request an image (1440) obtained according to a direction (1420) and then request an image according to a different direction (1450). For example, the head-worn electronic device (520) may obtain an image (1460) according to a different direction (1450) based on the request. For example, the head-worn electronic device (520) may transmit a plurality of images (e.g., image (1440) and image (1460)) to an electronic device (510) through a communication circuit.

[0256] According to an embodiment, the head-worn electronic device (520) may provide a notification (1430) based on receiving a signal to request another image (1460) from the electronic device (510). For example, the electronic device (510) may receive information about the image (1440) and information about the direction the camera is facing (e.g., direction (1420) and direction (1450)) from the head-worn electronic device (520) via a communication circuit. For example, the electronic device (510) may transmit a signal to the head-worn electronic device (520) via a communication circuit to request another image (1460) based on the information about the direction. For example, the electronic device (510) may transmit a signal to a head-worn electronic device (520) to request the acquisition of an image (e.g., image (1460)) taken according to a different direction (e.g., direction (1450)) based on information regarding a direction (1420) associated with an image (1440). For example, the electronic device (510) may receive information regarding the acquired image (1460) based on the signal from the head-worn electronic device (520) via a communication circuit.

[0257] FIG. 15 is a flowchart illustrating the operation of acquiring information about a face based on a plurality of facial feature points. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0258] Referring to FIG. 15, in operation 1501, the electronic device (510) can map multiple data sets containing multiple facial feature points in an image (e.g., image (1240) of FIG. 12) from which the head-worn electronic device (520) has been removed into one space.

[0259] For example, an electronic device (510) may receive information about a plurality of images representing a user (610) from a head-worn electronic device (520). For example, the electronic device (510) may obtain a data set containing a plurality of facial feature points for each of the plurality of images. For example, the electronic device (510) may obtain a plurality of data sets for the plurality of images. For example, the electronic device (510) may obtain mesh data based on the plurality of data sets. For example, the electronic device (510) may obtain mesh data based on a plurality of facial feature points included in a second data set among the plurality of data sets corresponding to a plurality of facial feature points included in a first data set among the plurality of data sets.

[0260] For example, the electronic device (510) can acquire multiple face feature points using a principal components analysis (PCA) model based on information about an image representing a user (610). For example, the electronic device (510) can acquire multiple data sets each containing multiple face feature points using a PCA model based on information about multiple images. For example, the electronic device (510) can acquire information about an average face feature point for each of the multiple face feature points included in the multiple data sets. For example, the electronic device (510) can acquire mesh data based on the average face feature point.

[0261] According to an embodiment, the electronic device (510) can identify information about a plurality of face feature points obtained through a size measurement module (e.g., the size measurement module (542) of FIG. 5b). For example, the electronic device (510) can identify the relative positional relationship between a plurality of face feature points based on the information about the plurality of face feature points. For example, the electronic device (510) can obtain mesh data representing a face using a PCA model based on identifying the relative positional relationship between a plurality of face feature points.

[0262] In operation 1502, the electronic device (510) can deform mesh data to create an avatar that realistically depicts the user based on information regarding the positions of a plurality of facial feature points. For example, the electronic device (510) can deform mesh data by changing the positions of a plurality of facial feature points representing one of the facial parts (e.g., eyes, eyebrows, nose, mouth, ears, and chin). For example, the electronic device (510) can enlarge the shape representing the user's (610) mouth by increasing the distance between a plurality of facial feature points representing the user's (610) mouth. A method for deforming mesh data according to an error value will be described later with reference to FIG. 16.

[0263] In operation 1503, the electronic device (510) can obtain information about the face based on mapping information about the color of the face and information about the texture of the face to mesh data. For example, the mesh data representing information about the shape of the face can be mapped to information about the color of the face and information about the texture of the face.

[0264] For example, the electronic device (510) can generate an avatar using information about a face obtained based on mesh data, information about the color of the face, and information about the texture of the face. For example, the avatar can be represented by mesh data representing the shape of the face, information about the color of the face, and information about the texture of the face.

[0265] As described above, according to the operation described in FIG. 15, the electronic device (510) can generate an avatar that realistically depicts the user (610). For example, the electronic device (510) can change the appearance of the avatar representing the user (610) by adjusting the mesh data.

[0266] FIG. 16 illustrates a method for obtaining information about a face based on an error value.

[0267] Referring to FIG. 16, the electronic device (510) can obtain an error value (1670) for obtaining information about a face based on a plurality of facial feature points. The error value (1670) can be described based on a 2D (two-dimensional) error value and a 3D error value. For example, the error value (1670) can be described as the sum of the 2D error value and the 3D error value.

[0268] For example, the electronic device (510) can adjust mesh data (1610) containing a plurality of facial feature points (1620) to generate an avatar that realistically depicts the user (610). For example, the mesh data (1610) can be adjusted according to an error value (1670). For example, the mesh data (1610) can be adjusted several times. The finally adjusted mesh data (1610) can correspond to information about the face.

[0269] For example, the electronic device (510) can identify a plurality of facial feature points (1641) obtained based on an image representing the user (610) (e.g., image (1640), image (1650), and / or image (1660)). For example, each of the images (1640), image (1650), and image (1660) may correspond to the image (650) described with reference to FIG. 6. For example, the electronic device (510) can identify a plurality of facial feature points (1641) using a PCA model based on an image representing the user (610). For example, the electronic device (510) can identify a plurality of facial feature points (1620) corresponding to each of the plurality of facial feature points (1641). For example, the electronic device (510) may obtain information regarding the distance between one of a plurality of face feature points (1641) and one of a plurality of face feature points (1620) corresponding to said face feature point. For example, the information regarding the distance may be described as information regarding the Euclidean distance. For example, the information regarding the distance may include information regarding the distance in a plane and / or information regarding the distance in 3D.

[0270] For example, a plurality of face feature points (e.g., a plurality of face feature points (1620) and a plurality of face feature points (1641)) may be projected onto a plane. For example, an electronic device (510) may obtain a plurality of face feature points (1601-1) projected onto a plane based on an image (1640). For example, an electronic device (510) may obtain a plurality of face feature points (1601-2) projected onto a plane based on an image (1650). For example, an electronic device (510) may obtain a plurality of face feature points (1601-3) projected onto a plane based on an image (1660).

[0271] For example, the electronic device (510) may project mesh data (1610) onto a plane according to the identified orientation based on identifying the user's orientation within the image (1640). For example, the electronic device (510) may obtain a plurality of face feature points (1620) projected onto a plane based on projecting the mesh data (1610) onto a plane. For example, the electronic device (510) may identify information regarding a mapping table between the plurality of face feature points (1620) projected onto a plane and the plurality of face feature points (1601-1) projected onto a plane. For example, the electronic device (510) may obtain information regarding the distance between the plurality of face feature points (1620) projected onto a plane and the plurality of face feature points (1601-1) projected onto a plane based on the information regarding the mapping table. The electronic device (510) can obtain information about the distance between a plurality of face feature points (1620) projected onto a plane and a plurality of face feature points (1601-2) projected onto a plane, as in the operation described above, and information about the distance between a plurality of face feature points (1620) projected onto a plane and a plurality of face feature points (1601-3) projected onto a plane. For example, the electronic device (510) can obtain information about the distance on a plane based on a plurality of face feature points (1601) projected onto a plane. For example, information regarding distances on a plane may include information regarding distances between a plurality of face feature points (1620) projected on a plane and a plurality of face feature points (1601-1) projected on a plane, information regarding distances between a plurality of face feature points (1620) projected on a plane and a plurality of face feature points (1601-2) projected on a plane, and / or information regarding distances between a plurality of face feature points (1620) projected on a plane and a plurality of face feature points (1601-3) projected on a plane.For example, information about distance on a plane can correspond to a 2D error value.

[0272] For example, multiple face feature points can be transferred into a three-dimensional space. For example, an electronic device (510) can obtain information about the distance between multiple face feature points transferred into a three-dimensional space. For example, the information about the distance between multiple face feature points transferred into a three-dimensional space can correspond to a 3D error value.

[0273] According to an embodiment, the electronic device (510) may obtain an error value (1670) based on a plurality of images representing a user (610) (e.g., image (1640), image (1650), and image (1660)). For example, the error value (1670) may be described as the sum of a first error value (1680-1) obtained based on a plurality of face feature points and a plurality of face feature points (1620) obtained by image (1640), a second error value (1680-2) obtained based on a plurality of face feature points and a plurality of face feature points (1620) obtained by image (1650), and a third error value (1680-3) obtained based on a plurality of face feature points and a plurality of face feature points (1620) obtained by image (1660).

[0274] For example, the electronic device (510) can adjust the mesh data (1610) based on reducing the error value (1670). For example, the electronic device (510) can adjust the mesh data (1610) based on reducing the sum of the 2D error value and the 3D error value. For example, the electronic device (510) can refrain from adjusting the mesh data (1610) based on identifying that the error value (1670) is smaller than a reference error value.

[0275] For example, an electronic device (510) can obtain information about a face (1690) based on adjusted mesh data (1610). For example, the information about a face (1690) may include information about the shape of the face corresponding to the adjusted mesh data. For example, the information about a face (1690) may include information about a plurality of facial feature points (1620) represented by the adjusted mesh data. For example, the electronic device (510) can generate an avatar of a user (610) using the information about a face (1690) obtained based on the adjusted mesh data (1610).

[0276] According to an embodiment, the electronic device (510) can generate an avatar of a user (610) through an avatar generation module (e.g., the avatar generation module (548) of FIG. 5b). For example, the electronic device (510) can generate an avatar based on a texture generated through a texture generation module (e.g., the texture generation module (546) of FIG. 5b) and mesh data obtained through a 3D Reconstruction module (e.g., the 3D Reconstruction module (543) of FIG. 5b). For example, the electronic device (510) can perform white balancing on the skin tone of the generated avatar using a white balancing module (e.g., the white balancing module (547) of FIG. 5b).

[0277] According to an embodiment, the electronic device (510) may receive information regarding an image representing the user (610) from the head-worn electronic device (520) after the avatar is created, and update the avatar based on the received information. For example, after the avatar is created, information regarding the user's face may be changed. For example, the electronic device (510) may change the avatar based on the information regarding the changed user's face. For example, the head-worn electronic device (520) may acquire an image representing the user (610) through a camera after the avatar is created. For example, the head-worn electronic device (520) may transmit information regarding the image representing the user (610) to the electronic device (510) through a communication circuit. For example, the electronic device (510) may identify whether the shape of the user's face has changed based on the information regarding the image received from the head-worn electronic device (520). For example, the electronic device (510) can change the avatar based on identifying that the shape of the user's face has changed.

[0278] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1 and the electronic device (510) of FIG. 5a) is provided. The electronic device may include a communication circuit (e.g., the communication circuit (512) of FIG. 5a), at least one processor (e.g., at least one processor (511) of FIG. 5a) including a processing circuit, and a memory (e.g., the memory (513) of FIG. 5a) which stores instructions and includes one or more storage media. The above instructions may cause the electronic device to receive, through the communication circuit, information about at least one image (e.g., image (460) of FIG. 6) representing the face of a user (e.g., user (610) of FIG. 6) obtained while at least a part of the user's face is obscured by the head-mounted electronic device, from a head-mounted display (HMD) device (e.g., head-mounted electronic device (520) of FIG. 5a) connected to the electronic device when executed individually or collectively by the at least one processor.

[0279] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain information about the user's face (e.g., information about the face of FIG. 6 (660)) based on the information about the at least one image and the information about the appearance of the head-worn electronic device (e.g., information about the appearance of the head-worn electronic device of FIG. 8 (810)).

[0280] For example, the above instructions may cause the electronic device to provide a service regarding the user’s avatar generated based on the information about the user’s face when executed individually or collectively by the at least one processor.

[0281] For example, the information regarding the face may include information regarding the shape of the face (e.g., information regarding the shape of the face in FIG. 8 (820)), information regarding the size of the face (e.g., information regarding the size of the face in FIG. 8 (821)), information regarding the color of the face and / or information regarding the texture of the face.

[0282] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify information regarding the shape of the head-worn electronic device included in the information regarding the external shape of the head-worn electronic device.

[0283] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain information regarding the shape of the user's face based on the information regarding the shape of the head-worn electronic device and the information regarding the at least one image.

[0284] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify that a shape identified based on the at least one image includes a part of the user's face.

[0285] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may cause the head-worn electronic device to transmit a signal through the communication circuit to request an image representing the remaining part of the face.

[0286] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to receive information about the image representing the remaining part of the face based on the signal from the head-worn electronic device through the communication circuit.

[0287] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to receive, through the communication circuit, information about a first direction in which the camera of the head-wearing electronic device is facing and information about the at least one image from the head-wearing electronic device.

[0288] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify that the shape identified based on the at least one image acquired according to the first direction includes the part of the user's face.

[0289] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify a second direction toward which the camera of the head-worn electronic device is facing for acquiring the image representing the remaining part of the face.

[0290] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to transmit the signal by transmitting information about the second direction to the head-worn electronic device through the communication circuit.

[0291] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify information regarding the size of the head-worn electronic device included in the information regarding the external shape of the head-worn electronic device.

[0292] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain information regarding the size of the face based on information regarding the shape of the head-worn electronic device and information regarding the size.

[0293] For example, the above at least one image may be at least one first image. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain at least one second image in which the head-wearable electronic device is removed within the at least one first image, based on the information regarding the shape of the head-wearable electronic device and the information regarding the at least one first image.

[0294] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the information about the user's face based on the information about the at least one second image and the information about the shape of the head-worn electronic device.

[0295] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify information regarding the position of a lens within the head-worn electronic device based on the information regarding the shape of the head-worn electronic device.

[0296] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain information about the position of the lens within the at least one first image based on the information about the position of the lens.

[0297] For example, the above instructions may cause the electronic device to identify information about the distortion of the lens when executed individually or collectively by the at least one processor.

[0298] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain information about at least one third image representing the user's face based on the information about the position of the lens and the information about the distortion within the at least one first image.

[0299] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to acquire the information about the face based on the information about the at least one third image and the information about the shape of the head-worn electronic device.

[0300] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify a visual object distinct from the user within an area corresponding to the lens in the at least one first image.

[0301] For example, the above instructions may cause the electronic device to acquire at least one fourth image based on removing the visual object within the at least one first image when executed individually or collectively by the at least one processor.

[0302] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the information regarding the at least one third image representing the user's face based on removing the head-worn electronic device within the at least one fourth image.

[0303] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to identify information regarding the color of the head-worn electronic device included in the information regarding the appearance of the head-worn electronic device.

[0304] For example, when the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain the information regarding the color of the face and the information regarding the texture of the face based on the information regarding the color of the head-worn electronic device and the information regarding the at least one image.

[0305] For example, the information regarding the shape of the face may include data indicating the three-dimensional shape of the face.

[0306] For example, the above data may include a plurality of feature points representing the shape of the face.

[0307] According to one embodiment, a head-worn electronic device is provided. The head-worn electronic device may include a camera (e.g., camera (523) of FIG. 5A) positioned toward the outside of the head-worn electronic device, a communication circuit (e.g., communication circuit (524) of FIG. 5A), at least one processor (e.g., at least one processor (521) of FIG. 5A) including a processing circuit, and a memory (e.g., memory (526) of FIG. 5A) that stores instructions and includes one or more storage media.

[0308] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may cause at least one image representing the user's face to be acquired through the camera while at least a portion of the user's face is obscured by the head-worn electronic device.

[0309] For example, the above instructions may cause the head-worn electronic device to transmit information about the at least one image to the electronic device through the communication circuit when executed individually or collectively by the at least one processor.

[0310] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may receive, through the communication circuit, a signal to request another image representing a part of the face from the electronic device.

[0311] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may provide a notification to guide the acquisition of information about another image representing the part of the face based on the signal.

[0312] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may cause the information about the other image acquired through the camera to be transmitted to the electronic device through the communication circuit.

[0313] For example, the above instructions may cause the head-worn electronic device to acquire information about the direction the camera is facing when executed individually or collectively by the at least one processor.

[0314] For example, the above instructions may cause the head-worn electronic device to transmit the information about the at least one image and the information about the direction to the electronic device through the communication circuit when executed individually or collectively by the at least one processor.

[0315] For example, the head-worn electronic device may include a light-emitting element. When the instructions are executed individually or collectively by the at least one processor,

[0316] Based on an input requesting the creation of the user's avatar, the head-wearable electronic device may be caused to emit a first light signal for identification of the head-wearable electronic device through the light-emitting element.

[0317] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may be caused to identify, through the camera, a second optical signal reflected from a reflector based on the first optical signal.

[0318] For example, the above instructions, when executed individually or collectively by the at least one processor, may cause the head-worn electronic device to acquire the at least one image representing the face through the camera based on identifying the second optical signal corresponding to the first optical signal.

[0319] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may be caused to identify information regarding a first shape of the head-worn electronic device stored in the memory.

[0320] For example, when the above instructions are executed individually or collectively by the at least one processor, the head-worn electronic device may be caused to identify the second optical signal corresponding to the first optical signal, and then identify whether the information regarding the second shape of the head-worn electronic device within the image acquired through the camera corresponds to the information regarding the first shape of the HMD.

[0321] For example, the head-worn electronic device may be configured to acquire the at least one image representing the face through the camera based on identifying the information regarding the second shape corresponding to the information regarding the first shape.

[0322] According to one embodiment, a method performed by an electronic device is provided. The method may include receiving, via a communication circuit, information regarding at least one image representing the face of a user, which is acquired while at least a portion of the user's face is obscured by the head-mounted electronic device, from a head-mounted display (HMD) device connected to the electronic device.

[0323] For example, the above method may include an operation of obtaining information about the user's face based on the information about the at least one image and the information about the appearance of the head-worn electronic device.

[0324] For example, the above method may include an operation of providing a service regarding the user's avatar generated based on the information regarding the user's face.

[0325] For example, the information regarding the face may include information regarding the shape of the face, information regarding the size of the face, information regarding the color of the face, and / or information regarding the texture of the face.

[0326] For example, the above method may include an operation of identifying information regarding the shape of the head-wearable electronic device included in the information regarding the external shape of the head-wearable electronic device.

[0327] For example, the above method may include the operation of obtaining information regarding the shape of the user's face based on the information regarding the shape of the head-worn electronic device and the information regarding at least one image.

[0328] For example, the above method may include an operation of identifying that a shape identified based on at least one image includes a part of the user's face.

[0329] For example, the above method may include the operation of transmitting a signal to the head-worn electronic device through the communication circuit to request an image representing the remaining part of the face.

[0330] For example, the above method may include the operation of receiving information about the image representing the remaining part of the face based on the signal from the head-worn electronic device through the communication circuit.

[0331] For example, the above method may include the operation of receiving, through the communication circuit, information regarding a first direction in which the camera of the head-wearing electronic device is facing and information regarding at least one image from the head-wearing electronic device.

[0332] For example, the above method may include an operation of identifying that the shape identified based on the at least one image obtained according to the first direction includes the part of the user's face.

[0333] For example, the above method may include an operation of identifying a second direction toward which the camera of the head-worn electronic device is facing for acquiring the image representing the remaining part of the face.

[0334] For example, the above method may include the operation of transmitting the signal by transmitting information about the second direction to the head-worn electronic device through the communication circuit.

[0335] For example, the above at least one image may be at least one first image. The method may include the operation of obtaining at least one second image in which the head-wearable electronic device is removed within the at least one first image, based on the information regarding the external shape of the head-wearable electronic device and the information regarding the at least one first image.

[0336] For example, the above method may include the operation of obtaining information about the user's face in contrast to information about at least one second image and information about the external shape of the head-worn electronic device.

[0337] According to one embodiment, a method is provided to be performed by a head-worn electronic device. The method may include the action of acquiring at least one image representing the face of a user through a camera while at least a portion of the user's face is obscured by the head-worn electronic device.

[0338] For example, the above method may include the operation of transmitting information about the at least one image to an electronic device through a communication circuit.

[0339] For example, the above method may include the operation of receiving, through the communication circuit, a signal for requesting another image representing a part of the face from the electronic device.

[0340] For example, the above method may include an operation to provide a notification to guide the acquisition of information about another image representing the part of the face based on the above signal.

[0341] For example, the above method may include the operation of transmitting the information regarding the other image acquired through the camera to the electronic device through the communication circuit.

[0342] According to one embodiment, a non-transient computer-readable storage medium is provided for storing one or more programs. For example, the one or more programs may include instructions that cause the electronic device to receive, through the communication circuit, information regarding at least one image representing the face of the user, which is acquired while at least a part of the user's face is obscured by the head-mounted electronic device, from a head-mounted display (HMD) device connected to the electronic device when executed by at least one processor of the electronic device having a communication circuit.

[0343] For example, the above one or more programs may include instructions that cause the electronic device to acquire information about the user's face based on the information about the at least one image and the appearance of the head-worn electronic device when executed by the at least one processor.

[0344] For example, the above one or more programs may include instructions that cause the electronic device to provide a service regarding the user's avatar generated based on the information regarding the user's face when executed by the at least one processor.

[0345] According to one embodiment, a non-transient computer-readable storage medium is provided for storing one or more programs. For example, the one or more programs may include instructions that cause the head-wearable electronic device to acquire at least one image representing the face of the user through the camera while at least a portion of the user's face is obscured by the head-wearable electronic device when executed by at least one processor of the head-wearable electronic device having a camera and a communication circuit.

[0346] For example, the above one or more programs may include instructions that cause the head-worn electronic device to transmit information about the at least one image to the electronic device through the communication circuit when executed by the at least one processor.

[0347] For example, the above one or more programs may include instructions that cause the head-worn electronic device to receive, through the communication circuit, a signal for requesting another image representing a part of the face from the electronic device when executed by the at least one processor.

[0348] For example, the above one or more programs may include instructions that cause the head-worn electronic device to provide a notification to guide the acquisition of information about another image representing the part of the face, based on the signal, when executed by the at least one processor.

[0349] For example, the above one or more programs may include instructions that cause the head-worn electronic device to transmit information about the other image acquired through the camera to the electronic device through the communication circuit when executed by the at least one processor.

[0350] According to the above-described embodiment, the electronic device (510) can identify the accessory and the head-worn electronic device by labeling the accessory (e.g., the part corresponding to the accessory in FIG. 8, the part corresponding to the head-worn electronic device) from an image obtained from the head-worn electronic device (520). For example, the electronic device (510) can generate an avatar of the user (610) based on obtaining an image from which the labeled parts have been removed. According to the above-described embodiment, the electronic device (510) can generate a realistic avatar based on the user's (610) daily actions. According to the above-described embodiment, the head-worn electronic device (520) can generate an avatar displayed on the execution screen of the application based on the user's (610) daily actions of running the application for video conferencing.

[0351] The electronic device according to the embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0352] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any component (e.g., the first) is referred to as "coupled" or "connected" to another component (e.g., the second), with or without the terms "functionally" or "communicationally," it means that said component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0353] In one embodiment of this document, the term “module” used may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0354] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0355] According to one embodiment, the method according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., CD-ROM (compact disc read-only memory)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0356] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, Communication circuit; At least one processor including a processing circuit, and It includes memory that stores instructions and includes one or more storage media, and When the above instructions are executed individually or collectively by the at least one processor, Information regarding at least one image representing the face of the user, obtained from a head-mounted electronic device connected to the electronic device while at least a part of the user's face is obscured by the head-mounted electronic device, is received via the communication circuit, and Based on the information regarding at least one image and the information regarding the appearance of the head-worn electronic device, information regarding the user's face is obtained, and The electronic device causing the service regarding the user's avatar generated based on the information regarding the user's face Electronic device.

2. In Claim 1, The above information regarding the above face is, Information regarding the shape of the face, information regarding the size of the face, information regarding the color of the face, and / or information regarding the texture of the face, Electronic device.

3. In claim 2, when the instructions are executed individually or collectively by the at least one processor, Identifying information regarding the shape of the head-wearable electronic device included in the information regarding the external appearance of the head-wearable electronic device, and Based on the information regarding the shape of the head-worn electronic device and the information regarding at least one image, the electronic device is to obtain the information regarding the shape of the user's face, thereby causing the electronic device to cause the electronic device. Electronic device.

4. In claim 3, when the instructions are executed individually or collectively by the at least one processor, Identifying that a shape identified based on at least one image includes a part of the user's face, and A signal for requesting an image representing the remaining part of the face is transmitted to the head-worn electronic device via the communication circuit, and Causing the electronic device to receive information about the image representing the remaining part of the face from the head-worn electronic device via the communication circuit based on the above signal, Electronic device.

5. In claim 4, when the instructions are executed individually or collectively by the at least one processor, From the head-wearing electronic device, information regarding a first direction in which the camera of the head-wearing electronic device is facing and information regarding at least one image are received through the communication circuit, and Identifying that the shape identified based on the at least one image obtained according to the first direction includes the part of the user's face, Identifying a second direction in which the camera of the head-worn electronic device is facing for acquiring the image representing the remaining part of the face, and By transmitting information regarding the second direction to the head-worn electronic device through the communication circuit, thereby causing the electronic device to transmit the signal, Electronic device.

6. In Claim 3, The information regarding the shape of the face includes data indicating the three-dimensional shape of the face, and The above data includes a plurality of feature points representing the shape of the face, Electronic device.

7. In claim 2, when the instructions are executed individually or collectively by the at least one processor, Identifying information regarding the size of the head-wearable electronic device included in the information regarding the external shape of the head-wearable electronic device, and Causing the electronic device to obtain information regarding the size of the face based on the information regarding the shape and the information regarding the size of the head-worn electronic device. Electronic device.

8. In Claim 1, The above at least one image is at least one first image, and When the above instructions are executed individually or collectively by the at least one processor, Based on the information regarding the external shape of the head-wearable electronic device and the information regarding the at least one first image, at least one second image is obtained in which the head-wearable electronic device is removed within the at least one first image, and An electronic device that causes the user to obtain information about the user's face based on information about at least one second image and information about the external shape of the head-worn electronic device. Electronic device.

9. In claim 8, when the instructions are executed individually or collectively by the at least one processor, Based on the information regarding the external shape of the head-worn electronic device, information regarding the position of a lens within the head-worn electronic device is identified, and Based on the information regarding the position of the lens, information regarding the position of the lens is obtained within at least one first image, and Identify information regarding the distortion of the above lens, and Based on the information regarding the position of the lens and the information regarding the distortion within the at least one first image, information regarding at least one third image representing the user's face is obtained, and Causing the electronic device to acquire the information about the face based on the information about at least one third image and the information about the external shape of the head-worn electronic device, Electronic device.

10. In claim 9, when the instructions are executed individually or collectively by the at least one processor, Identifying a visual object distinct from the user within an area corresponding to the lens in at least one first image, and Based on removing the visual object within the above at least one first image, at least one fourth image is obtained, and In the above at least one fourth image, the electronic device causes the information regarding the at least one third image representing the user's face to be obtained based on removing the head-worn electronic device. Electronic device.

11. In a head-worn electronic device, A camera positioned facing the outside of the head-worn electronic device; Communication circuit; At least one processor including a processing circuit; and It includes memory that stores instructions and includes one or more storage media, and When the above instructions are executed individually or collectively by the at least one processor, While at least a portion of the user's face is obscured by the head-worn electronic device, at least one image representing the user's face is acquired through the camera, and Information regarding at least one image is transmitted to an electronic device through the communication circuit, and A signal for requesting another image representing a part of the face from the electronic device is received through the communication circuit, and Based on the above signal, a notification is provided to guide the acquisition of information regarding another image representing the above part of the face, and Causing the head-worn electronic device to transmit the information about the other image acquired through the camera to the electronic device through the communication circuit, Head-worn electronic device.

12. In claim 11, when the instructions are executed individually or collectively by the at least one processor, Acquire information about the direction the above camera is facing, and Causing the head-worn electronic device to transmit the information regarding the at least one image and the information regarding the direction to the electronic device through the communication circuit, Head-worn electronic device.

13. In Claim 11, It includes a light-emitting device, and When the above instructions are executed individually or collectively by the at least one processor, Based on an input requesting the creation of the user's avatar, a first optical signal for identifying the head-worn electronic device is emitted through the light-emitting element, and Based on the first optical signal, the second optical signal reflected from the reflector is identified through the camera, and A head-worn electronic device that causes to acquire, through the camera, at least one image representing the face based on identifying the second optical signal corresponding to the first optical signal. Head-worn electronic device.

14. In claim 13, when the instructions are executed individually or collectively by the at least one processor, Identifying information regarding the first external shape of the head-worn electronic device stored in the memory, and After identifying the second optical signal corresponding to the first optical signal, determine whether the information regarding the second external shape of the head-worn electronic device within the image acquired through the camera corresponds to the information regarding the first external shape of the HMD. A head-worn electronic device that causes the acquisition of at least one image representing the face through the camera based on identifying the information regarding the second shape corresponding to the information regarding the first shape, Head-worn electronic device.

15. In a method performed by an electronic device, the method comprises: The operation of receiving, via a communication circuit, information regarding at least one image representing the user's face, which is acquired while at least a part of the user's face is obscured by the head-worn electronic device, from a head-mounted display (HMD) device connected to the electronic device; The operation of obtaining information about the user's face based on the information about at least one image and the information about the appearance of the head-worn electronic device; and The operation of providing a service regarding the user's avatar generated based on the information regarding the user's face, method.